Unexpected outage

Summary

Unexpected outage

**After a thorough analysis, we identified that one of our components had reached the maximum number of database connections it was allowed to establish.** As traffic continued to grow, our health check system marked the component as unavailable and automatically restarted it in an attempt to restore normal operation. It took approximately 30 minutes for the component to come back online with a sufficient number of available connections to handle the load. **Remediation:** We have increased the maximum number of allowed connections for this particular component. We sincerely apologize for any inconvenience this outage may have caused. Please rest assured that we are committed to providing the best service possible.

• Started

  Apr 29, 2025 · 10:00 AM

• Resolved

  Apr 29, 2025 · 10:00 AM

• Duration

  < 1m

• Severity

  Critical

• ✓

  Resolved

  Apr 29 · 10:49 PM Apwide

  Golive & Time Squad were unavailable this morning for a duration of approximately 30 minutes. We are currently conducting an investigation to identify the root cause.

• ✓

  Postmortem

  May 1 · 3:35 PM Apwide

  **After a thorough analysis, we identified that one of our components had reached the maximum number of database connections it was allowed to establish.** As traffic continued to grow, our health check system marked the component as unavailable and automatically restarted it in an attempt to restore normal operation. It took approximately 30 minutes for the component to come back online with a sufficient number of available connections to handle the load. **Remediation:** We have increased the maximum number of allowed connections for this particular component. We sincerely apologize for any inconvenience this outage may have caused. Please rest assured that we are committed to providing the best service possible.