Platform-Wide Outage

Incident Report for Progress Retail

Resolved

Service has been restored.
Posted Oct 29, 2025 - 15:39 CDT

Identified

Update from Microsoft Current Status:

We initiated the deployment of our ‘last known good’ configuration, which has now successfully been completed. Customers may have begun to see initial signs of recovery. We are currently recovering nodes and routing traffic through healthy nodes, and as we make progress in this workstream, customers will continue to see improvement.

Customer configuration changes will remain temporarily blocked while we continue mitigation efforts. We will notify customers once this block has been lifted.

Some customers may also have experienced issues accessing the Azure management portal. We have failed the portal away from AFD to mitigate these access issues. Customers should now be able to access the Azure portal directly, and while most portal extensions are functioning as expected, a small number of endpoints (e.g., Marketplace) may still experience intermittent loading problems.

At this stage, we anticipate full mitigation within the next four hours as we continue to recover nodes. This means we expect recovery to happen by 23:20 UTC on 29 October 2025. We will provide another update on our progress within two hours, or sooner if warranted.

Although we are seeing signs of recovery and have an estimated timeline, customers may also consider implementing failover strategies using Azure Traffic Manager to redirect traffic from Azure Front Door to their origin servers as an interim measure.

Learn more about Azure Front Door failover strategies for AFD: https://learn.microsoft.com/en-us/azure/architecture/guide/networking/global-web-applications/overview

This message was last updated at 19:22 UTC on 29 October 2025
Posted Oct 29, 2025 - 14:42 CDT

Update

Per Microsoft Azure, this is a global network infrastructure issue.

https://azure.status.microsoft/en-us/status
Posted Oct 29, 2025 - 12:18 CDT

Update

Microsoft is beginning to acknowledge the outage. DownDetector is at 18k reports and climbing.

"Starting at approximately 16:00 UTC, we began experiencing DNS issues resulting in availability degradation of some services. Customers may experience issues accessing the Azure Portal. We have taken action that is expected to address the portal access issues here shortly. We are actively investigating the underlying issue and additional mitigation actions. More information will be provided within 60 minutes or sooner."
Posted Oct 29, 2025 - 11:48 CDT

Update

Microsoft has acknowledged Azure Portal Access Issues, but has yet to acknowledge core infrastructure outage.

"We are investigating an issue with the Azure Portal where customers may be experiencing issues accessing the portal. More information will be provided shortly."
Posted Oct 29, 2025 - 11:22 CDT

Update

We are continuing to investigate this issue.
Posted Oct 29, 2025 - 11:13 CDT

Update

DownDetector has 15k outage reports and climbing. https://downdetector.com/status/windows-azure/

Microsoft has not yet claimed the outage.
https://azure.status.microsoft/en-us/status
Posted Oct 29, 2025 - 11:13 CDT

Investigating

Microsoft Azure services seem to be affected as all environments are not loading. There is a known Azure outage at this time.
Posted Oct 29, 2025 - 10:53 CDT
This incident affected: Learn (Learning Room, Learning Content Library, Course Designer), Task Management & Execution (Task Management Calendar, Task Management List View, Task Management Location View, My Tasks, Scheduled Tasks), Learning Analytics (Learning Analytics Overview, Learning Analytics: Courses Layer, Learning Analytics: Users Layer), Library (File Library), Administration (User Administation, Role Administration, Location Administration, Course Administration, Company Profile & Branding), Connect (Connect: Channels & Messaging), AI Sidekick & Search (AI Sidekick, AI Search), and Authentication, Notification Service, Google Cloud Platform Google Cloud SQL, Twilio SMS, APAC, Twilio SMS, APAC, Twilio SMS, APAC, Twilio SMS Toll-Free, North America, AWS rds-us-east-1.