Pro MA flows not processing data.

Minor · Started Feb 10, 2025 · 12:59 PM

Duration

2d 4h 16m
Severity

Minor
Detection lead

—
User reports

—

Summary

Pro MA flows not processing data.

We do not see any indications that MA service is underperforming anymore. Incident has been resolved.

Started

Feb 10, 2025 · 12:59 PM
Resolved

Feb 12, 2025 · 5:15 PM
Duration

2d 4h 16m
Severity

Minor

Event timeline

How this incident unfolded

◐

Investigating
Feb 10 · 12:59 PM Apsis Pro

We are experiencing issues with profiles not triggering flows in Pro Marketing Automation. Investigation so far indicates that this is affecting all customers using Pro MA. Problem seems to also affect the processing of profiles within the flow, and some profiles are stuck. We continue investigating the cause and overall impact.
◐

Investigating
Feb 10 · 6:17 PM Apsis Pro

We are still investigating the issue - the response team have found possible solution but we need to make sure applying it won't trigger sending unwanted MA emails and sms and extending the incident impact.
◐

Investigating
Feb 11 · 6:48 AM Apsis Pro

We have made significant progress in understanding the problem. Unfortunately, our attempts to restore the service functionality were unsuccessful yesterday. The response team will continue its efforts today.
◉

Monitoring
Feb 11 · 10:44 AM Apsis Pro

Incident response team implemented mitigation actions, and the initial data suggest situation is contained - MA is processing data as expected. We are closely monitoring the situation.
◉

Monitoring
Feb 12 · 6:44 AM Apsis Pro

The state of the service seems to have normalized now - we are still closely monitoring it in search for abnormal behaviour
◉

Monitoring
Feb 12 · 2:52 PM Apsis Pro

We have made some changes to the configuration backend, that should improve the processing of traffic for Marketing Automation. We are preforming some tests to ensure that the service is processing jobs at a normal rate.
✓

Resolved
Feb 12 · 5:15 PM Apsis Pro

We do not see any indications that MA service is underperforming anymore. Incident has been resolved.