Actions Job Delays

Summary

Actions Job Delays

On October 1, 2026, GitHub Actions experienced two periods of delayed and failed job starts on GitHub-hosted runners, beginning at approximately 14:15 UTC. Across the 14:15–17:56 UTC incident-response window, approximately 0.68% of workflow runs experienced start delays, and 0.196% of jobs failed to start. <br /><br />A routing change at our cloud provider concentrated API traffic on a single backend, causing an increase in rate-limit errors. Incorrect retry behavior in the provider’s inventory service added load to the affected APIs, compounding the rate limiting. This disrupted runner provisioning and reduced available capacity in affected regions. The resulting increase in demand in unaffected regions exposed a concurrency limit in our job-assignment system, delaying jobs even where runners were available. <br /><br />During the response, we increased job-assignment concurrency, scaled runner capacity, and worked with our cloud provider to reduce API load by excluding our runner subscriptions from its inventory service. By approximately 16:15 UTC, jobs were starting and executing as expected. We continued mitigation and monitoring until the incident was resolved at 17:56 UTC. <br /><br />Our cloud provider has since increased resource-management API rate limits for our workloads. To reduce the likelihood and impact of similar incidents, we are also adding more granular rate-limiting alerts and updating our job-assignment system to handle increased demand across fewer available regions.

• Started

  Oct 1, 2026 · 2:47 PM

• Resolved

  Oct 1, 2026 · 5:56 PM

• Duration

  3h 9m

• Severity

  Minor

• ◐

  Investigating

  Oct 1 · 2:47 PM GitHub

  We are investigating reports of degraded performance for Actions

• ◐

  Investigating

  Oct 1 · 2:54 PM GitHub

  We have identified the cause of increased Actions run start delays on Ubuntu runners and are actively deploying a fix. Customers may continue to experience intermittent delays while the mitigation rolls out and service metrics return to normal.

• ◐

  Investigating

  Oct 1 · 3:02 PM GitHub

  Run-start delays on Ubuntu runners have been resolved. We are investigating run-start delays on Windows runners and will share more information as it becomes available.

• ◉

  Monitoring

  Oct 1 · 3:10 PM GitHub

  The degradation affecting Actions has been mitigated. We are monitoring to ensure stability.

• ◐

  Investigating

  Oct 1 · 3:20 PM GitHub

  Actions is experiencing degraded performance. We are continuing to investigate.

• ◐

  Investigating

  Oct 1 · 3:29 PM GitHub

  We are seeing a reoccurrence in run-start delays, and are continuing to investigate to issue. Customers will potentially experience delays of up to ten minutes.

• ◐

  Investigating

  Oct 1 · 4:10 PM GitHub

  We have identified an issue with our upstream provider which is causing Actions requests to 429 which is creating the delays. We have escalated to the owning team and are investigating how to mitigate the 429s.

• ◐

  Investigating

  Oct 1 · 4:48 PM GitHub

  We are currently applying a mitigation and anticipate recovery within thirty minutes.

• ◐

  Investigating

  Oct 1 · 5:50 PM GitHub

  GitHub Actions experienced degraded performance for some hosted runners due to throttling within an upstream Azure dependency. Service capacity has recovered, and we are continuing to monitor while working with Azure on the underlying condition.

• ✓

  Resolved

  Oct 1 · 5:56 PM GitHub

  On October 1, 2026, GitHub Actions experienced two periods of delayed and failed job starts on GitHub-hosted runners, beginning at approximately 14:15 UTC. Across the 14:15–17:56 UTC incident-response window, approximately 0.68% of workflow runs experienced start delays, and 0.196% of jobs failed to start. <br /><br />A routing change at our cloud provider concentrated API traffic on a single backend, causing an increase in rate-limit errors. Incorrect retry behavior in the provider’s inventory service added load to the affected APIs, compounding the rate limiting. This disrupted runner provisioning and reduced available capacity in affected regions. The resulting increase in demand in unaffected regions exposed a concurrency limit in our job-assignment system, delaying jobs even where runners were available. <br /><br />During the response, we increased job-assignment concurrency, scaled runner capacity, and worked with our cloud provider to reduce API load by excluding our runner subscriptions from its inventory service. By approximately 16:15 UTC, jobs were starting and executing as expected. We continued mitigation and monitoring until the incident was resolved at 17:56 UTC. <br /><br />Our cloud provider has since increased resource-management API rate limits for our workloads. To reduce the likelihood and impact of similar incidents, we are also adding more granular rate-limiting alerts and updating our job-assignment system to handle increased demand across fewer available regions.