Increased task wait times for Docker Gen2

Summary

Increased task wait times for Docker Gen2

## Summary Since October 1, CircleCI customers have experienced multiple incidents which have caused delays and failures in customer pipeline execution. There are two \(unrelated\) causes, both of which the CircleCI engineering team is actively mitigating: * Available Cloud Computing Capacity * Internal Infrastructure ## Available Cloud Computing Capacity * **Problem**: Demand for high-performance cloud instances is rising extremely rapidly across the industry, which puts pressure on the cloud provider instance types that we use to execute customer jobs. * **Incidents**: * [Elevated wait times for machine jobs](https://status.circleci.com/incidents/jn756xtsy4xg) * [Increased task wait times for Docker Gen2](https://status.circleci.com/incidents/39rfrjmk5pvd) * [Elevated level of infra fails on customer jobs](https://status.circleci.com/incidents/0z31ldjd8m5v) * [Delay on starting Machine Job Tasks](https://status.circleci.com/incidents/tj6wdnfjwr64) * [Delays starting Gen 2 Docker Jobs](https://status.circleci.com/incidents/22mrv46g6n7g) * **Mitigations and Resolutions**: * We are expanding our set of cloud computing regions to include additional regions with available high-performance instances. * We are also working to secure additional guaranteed capacity from our cloud computing partners in our existing regions. * We are further expanding the set of instance types we can offer to customers. * We will publish a detailed Incident Report on these incidents on October 9, 2026. ## Internal Infrastructure * **Problem**: We assign customer workflows over multiple compute providers via a workflow orchestration service. This service is currently gated by the write throughput of a single database. * **Incidents:** * [Delays in pipelines, workflows, UI data, and notifications](https://status.circleci.com/incidents/z4hbq8h49j8c) * [Emergency Infrastructure Maintenance](https://status.circleci.com/incidents/g60yc9bvvkmx) * **Mitigations and Resolutions**: * By October 23, 2026, we plan to implement multiple redundant workflow orchestration service instances, each with its own database, which will enable us split the orchestration load between them. ## Our Commitment For avoidance of doubt, * These particular incidents are not related to ongoing outages at Github * We are not currently migrating our infrastructure or services * We are not currently under cyberattack Customer experience is our top priority, and we commit to continually improving the reliability of our systems to match the trust that our customers place in us. We thank our customers for their patience while our team worked to resolve these incidents and works to prevent future incidents. Please reach out to our support team with any questions or concerns.

• Started

  Oct 7, 2026 · 6:54 PM

• Resolved

  Oct 7, 2026 · 8:05 PM

• Duration

  1h 10m

• Severity

  Minor

• ◆

  Identified

  Oct 7 · 6:54 PM CircleCI

  There is an increased wait time for jobs on Docker Gen2.

• ◆

  Identified

  Oct 7 · 7:00 PM CircleCI

  Note: delays will be more noticeable for people running on Docker `xlarge` and above.

• ◉

  Monitoring

  Oct 7 · 7:52 PM CircleCI

  We're still seeing spikes in task wait times but on average wait times are looking better for all Docker Gen2.

• ✓

  Resolved

  Oct 7 · 8:05 PM CircleCI

  This incident has been resolved.

• ✓

  Postmortem

  Oct 8 · 5:54 PM CircleCI

  ## Summary Since October 1, CircleCI customers have experienced multiple incidents which have caused delays and failures in customer pipeline execution. There are two \(unrelated\) causes, both of which the CircleCI engineering team is actively mitigating: * Available Cloud Computing Capacity * Internal Infrastructure ## Available Cloud Computing Capacity * **Problem**: Demand for high-performance cloud instances is rising extremely rapidly across the industry, which puts pressure on the cloud provider instance types that we use to execute customer jobs. * **Incidents**: * [Elevated wait times for machine jobs](https://status.circleci.com/incidents/jn756xtsy4xg) * [Increased task wait times for Docker Gen2](https://status.circleci.com/incidents/39rfrjmk5pvd) * [Elevated level of infra fails on customer jobs](https://status.circleci.com/incidents/0z31ldjd8m5v) * [Delay on starting Machine Job Tasks](https://status.circleci.com/incidents/tj6wdnfjwr64) * [Delays starting Gen 2 Docker Jobs](https://status.circleci.com/incidents/22mrv46g6n7g) * **Mitigations and Resolutions**: * We are expanding our set of cloud computing regions to include additional regions with available high-performance instances. * We are also working to secure additional guaranteed capacity from our cloud computing partners in our existing regions. * We are further expanding the set of instance types we can offer to customers. * We will publish a detailed Incident Report on these incidents on October 9, 2026. ## Internal Infrastructure * **Problem**: We assign customer workflows over multiple compute providers via a workflow orchestration service. This service is currently gated by the write throughput of a single database. * **Incidents:** * [Delays in pipelines, workflows, UI data, and notifications](https://status.circleci.com/incidents/z4hbq8h49j8c) * [Emergency Infrastructure Maintenance](https://status.circleci.com/incidents/g60yc9bvvkmx) * **Mitigations and Resolutions**: * By October 23, 2026, we plan to implement multiple redundant workflow orchestration service instances, each with its own database, which will enable us split the orchestration load between them. ## Our Commitment For avoidance of doubt, * These particular incidents are not related to ongoing outages at Github * We are not currently migrating our infrastructure or services * We are not currently under cyberattack Customer experience is our top priority, and we commit to continually improving the reliability of our systems to match the trust that our customers place in us. We thank our customers for their patience while our team worked to resolve these incidents and works to prevent future incidents. Please reach out to our support team with any questions or concerns.