Identified - The issue has been identified, a temporary fix is being put in place. We will continue to update this as we find out more from our temporary fix.
Jul 23, 23:19 CDT
Update - We are now seeing instability on the primary StudioNet uplinks, we are continuing to search for the root cause of this issue. No on air services are impacted at this time, however regions may notice intermittent internet connectivity issues.

Please report any additional issues to wpr-eng@wpr.org
Jul 23, 17:40 CDT
Investigating - We are investigating an issue with links failing over to backup and not returning properly causing audio outages. A temporary fix is in place to prevent failovers, however more work may need to be done after hours.
Jul 23, 11:06 CDT
Headend Audio Operational
Ideas Network Audio ? Operational
Music Network Audio Operational
HD2 (All Classical) Network Audio Operational
Madison Networking Partial Outage
90 days ago
99.89 % uptime
Today
WPR Core Firewall ? Operational
90 days ago
100.0 % uptime
Today
WPR Engineering Business Network ? Operational
90 days ago
100.0 % uptime
Today
StudioNet Primary ? Degraded Performance
90 days ago
100.0 % uptime
Today
StudioNet Secondary ? Partial Outage
90 days ago
99.18 % uptime
Today
Internal VoIP Phones ? Operational
DR Site Firewall ? Operational
90 days ago
100.0 % uptime
Today
CSSC Axia Primary ? Operational
90 days ago
100.0 % uptime
Today
CSSC Axia Secondary ? Operational
90 days ago
100.0 % uptime
Today
Vilas Audio Networking ? Operational
90 days ago
100.0 % uptime
Today
Primary Virtual Infrastructure Operational
90 days ago
99.98 % uptime
Today
WPR Primary Virtual Cluster ? Operational
90 days ago
99.98 % uptime
Today
WPR Primary Virtual Desktop Infrastructure ? Operational
90 days ago
99.98 % uptime
Today
VCenter Server 1 ? Operational
90 days ago
99.98 % uptime
Today
VCenter Server 2 ? Operational
90 days ago
99.98 % uptime
Today
VM Host 1 ? Operational
VM Host 2 ? Operational
VM Host 3 ? Operational
VM Host 4 ? Operational
VM Host 5 ? Operational
VM Host 6 ? Operational
VM Host 7 ? Operational
VM Host 8 ? Operational
VDI Host 1 ? Operational
VDI Host 2 ? Operational
VDI Host 3 ? Operational
VDI Host 4 ? Operational
Storage Infrastructure Operational
90 days ago
99.97 % uptime
Today
MSN Primary Array ? Operational
90 days ago
99.98 % uptime
Today
MSN Replication Array ? Operational
90 days ago
99.98 % uptime
Today
Archive Shared Drive Operational
90 days ago
99.96 % uptime
Today
Audio Common Shared Drive Operational
90 days ago
99.96 % uptime
Today
Wausau Partial Outage
90 days ago
99.25 % uptime
Today
Wausau Networking ? Operational
Wausau AudioVault Operational
Wausau Virtualization Operational
StudioNet Link Secondary Partial Outage
90 days ago
98.87 % uptime
Today
StudioNet Link Primary Degraded Performance
90 days ago
99.63 % uptime
Today
Eau Claire Partial Outage
Eau Claire Networking ? Partial Outage
Superior Partial Outage
90 days ago
99.6 % uptime
Today
Superior Axia ? Operational
Superior Networking ? Operational
Superior Virtualization Operational
Superior AudioVault Operational
Superior StudioNet Link Primary Degraded Performance
90 days ago
100.0 % uptime
Today
Superior StudioNet Link Secondary Partial Outage
90 days ago
99.2 % uptime
Today
Web Services Operational
Monitoring ? Operational
Ticketing System ? Operational
Docs / Wiki ? Operational
External Services Operational
Slack Login/SSO Operational
Slack Messaging Operational
Slack Notifications Operational
Slack Apps/Integrations Operational
Slack Workspace/Org Administration Operational
Opsgenie Voice Notification Delivery Operational
Opsgenie Email Notification Delivery Operational
Opsgenie Alert Flow Operational
Opsgenie Incident Flow Operational
Opsgenie Incoming Email Service Operational
Opsgenie Incoming Integration Flow Operational
Opsgenie Incoming Call Routing Operational
FortiClient VPN ? Operational
Newsboss Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Past Incidents
Jul 25, 2021

No incidents reported today.

Jul 24, 2021

No incidents reported.

Jul 23, 2021

Unresolved incident: Backup StudioNet link impacting services.

Jul 22, 2021

No incidents reported.

Jul 21, 2021
Postmortem - Read details
Jul 21, 21:14 CDT
Completed - The scheduled maintenance has been completed.
Jul 21, 21:00 CDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 21, 20:00 CDT
Scheduled - During this time we will be updating our core storage switches to resolve a bug in the switch OS. This should not impact any services, however we will be ready to respond in the event of any outages.
Jul 21, 14:24 CDT
Jul 20, 2021

No incidents reported.

Jul 19, 2021

No incidents reported.

Jul 18, 2021

No incidents reported.

Jul 17, 2021

No incidents reported.

Jul 16, 2021

No incidents reported.

Jul 15, 2021

No incidents reported.

Jul 14, 2021

No incidents reported.

Jul 13, 2021

No incidents reported.

Jul 12, 2021

No incidents reported.

Jul 11, 2021

No incidents reported.