Delayed and Timed-out requests to the Aptible LLM Gateway

Summary

Delayed and Timed-out requests to the Aptible LLM Gateway

Following the fixes deployed per the previous update, we have seen stable performance on the LLM Gateway since 7:12 EDT and are resolving the issue. The root cause of this issue was a severe performance degradation in 3 specific endpoints: • `GET /models` • `GET /v1/models` • `GET /metadata/models` ...all of which return a list of available models. The performance issue on these endpoints was severe enough to cause significant request queueing and degraded performance for other unaffected endpoints. The fix we deployed added caching on this endpoint to ensure fast responses. Since those fixes have been deployed, we are no longer seeing request queueing or impacts to other LLM Gateway API endpoints.

• Started

  Jul 16, 2026 · 6:31 PM

• Resolved

  Jul 17, 2026 · 1:04 PM

• Duration

  18h 32m

• Severity

  Minor

• ◆

  Identified

  Jul 16 · 6:31 PM Aptible

  We are aware of delayed and timed-out requests to the LLM Gateway. We have made some changes to help minimize the impact while engineers continue to address the root cause.

• ◉

  Monitoring

  Jul 16 · 8:02 PM Aptible

  We have increased resource capacity, and we are monitoring the results.

• ◉

  Monitoring

  Jul 17 · 8:14 AM Aptible

  After a regression in performance starting at 9:57 UTC (5:57 EDT) and lasting until 11:12 UTC (7:12 EDT), we've deployed a potential fix and updated our service scaling. We are continuing to monitor whether these changes fully resolve the issue before closing this incident.

• ✓

  Resolved

  Jul 17 · 1:04 PM Aptible

  Following the fixes deployed per the previous update, we have seen stable performance on the LLM Gateway since 7:12 EDT and are resolving the issue. The root cause of this issue was a severe performance degradation in 3 specific endpoints: • `GET /models` • `GET /v1/models` • `GET /metadata/models` ...all of which return a list of available models. The performance issue on these endpoints was severe enough to cause significant request queueing and degraded performance for other unaffected endpoints. The fix we deployed added caching on this endpoint to ensure fast responses. Since those fixes have been deployed, we are no longer seeing request queueing or impacts to other LLM Gateway API endpoints.