Incident with CodeQL, Webhooks, Notifications, and Slack Integration

Summary

Incident with CodeQL, Webhooks, Notifications, and Slack Integration

This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.

• Started

  May 12, 2026 · 2:38 PM

• Resolved

  May 12, 2026 · 5:43 PM

• Duration

  3h 5m

• Severity

  Minor

• ◐

  Investigating

  May 12 · 2:38 PM GitHub

  We are investigating reports of degraded performance for CodeQL

• ◐

  Investigating

  May 12 · 3:13 PM GitHub

  CodeQL actions are currently experiencing delays, which may result in those actions being stuck in a pending state or having failed due to a timeout.

• ◐

  Investigating

  May 12 · 3:42 PM GitHub

  We're continuing to investigate issues with CodeQL actions workflows. We're additionally seeing delays for notifications, webhooks, and the Slack integration.

• ◐

  Investigating

  May 12 · 3:44 PM GitHub

  Webhooks is experiencing degraded performance. We are continuing to investigate.

• ◐

  Investigating

  May 12 · 4:18 PM GitHub

  We've established that most delays are related to a queuing service and are working to scale out. Early signals from the scale-out are showing signs of recovery for some services. We'll provide an update when services are fully recovered.

• ◐

  Investigating

  May 12 · 4:28 PM GitHub

  Webhooks is operating normally.

• ◐

  Investigating

  May 12 · 4:29 PM GitHub

  Webhooks have fully recovered. Continuing to work on recovery for the other services.

• ◐

  Investigating

  May 12 · 4:59 PM GitHub

  CodeQL has fully recovered. We're continuing to work on recovery for the remaining impacted services.

• ◐

  Investigating

  May 12 · 5:43 PM GitHub

  All services have fully recovered.

• ✓

  Resolved

  May 12 · 5:43 PM GitHub

  This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.