Inaccurate notifications being sent for legacy alerts

Summary

Inaccurate notifications being sent for legacy alerts

# Service Degradation - Legacy Alerts Notification ## Root Cause Analysis ### Duration of the incident Discovered: Aug 17, 2026 13:15 - UTC Resolved: Aug 21, 2026 14:12 - UTC ### Customer impact Some customers using Legacy Alerts experienced inaccurate alert notifications, including false-positive alerts and unintended email notifications. During remediation, some legacy alert processing was delayed, and alert visibility in the user interface temporarily lagged. V2 Alerts remained operational throughout the incident. ### Cause A product release intended to retire older default Legacy Alert configurations for new environments also removed supporting alert conditions that were still required by some existing customers with customized Legacy Alerts. Without the complete conditions, certain Legacy Alerts could trigger inaccurately. The remediation required reprocessing affected alerts, which temporarily increased processing load and contributed to delayed alert visibility, alert flapping, and excessive notifications. ### Effect Affected Legacy Alerts could trigger when the underlying condition was not present, re-trigger after being paused, or send notifications more broadly than intended. The resulting in-service lag also temporarily delayed displaying some alerts. Engineering restored the required alert configuration, removed the unintended broad email association, increased processing capacity, and proactively corrected additional affected customer environments. ### Future consideration\(s\) • Add monitoring for abnormal volumes of Legacy Alerts so unexpected alert activity can be detected earlier. • Expand pre-release and end-to-end testing for existing customized Legacy Alerts, including alert triggering, notification delivery, and post-change recovery behavior. • Strengthen canary-release review so alerting anomalies are investigated and understood before wider deployment.

• Started

  Aug 17, 2026 · 1:53 PM

• Resolved

  Aug 20, 2026 · 10:11 AM

• Duration

  2d 20h 17m

• Severity

  Minor

• ◐

  Investigating

  Aug 17 · 1:53 PM Auvik Networks

  We are investigating reports of a high volume of inaccurate notifications. Impact: Customers may experience improper alerts and notifications The following services are not affected: V2 alert notifications. Next Steps: Our team is working to identify contributing factors. We will share updates as more information becomes available.

• ◐

  Investigating

  Aug 17 · 3:04 PM Auvik Networks

  We are continuing to investigate the issues with incorrect notifications being sent. Impact: Customers may experience improper alerts and notifications The following services are not affected: V2 alert notifications. Next Steps: Our team is working to identify contributing factors. We will share updates as more information becomes available.

• ◆

  Identified

  Aug 17 · 3:45 PM Auvik Networks

  Our team has identified a suspected cause of the incorrect notifications and is taking steps to fix the issue. Impact: Customers may continue to experience false paused notifications The following services are not affected: V2 alerting Please report any related issues to Auvik Support so we can track and assist further. Next Steps: We are applying mitigation measures and will provide progress updates.

• ◉

  Monitoring

  Aug 17 · 4:31 PM Auvik Networks

  We have applied changes to address the issue. Alert notifications should no longer be sent to unintended users We are closely monitoring services for stability. Impact: Services should be operating normally, though new alerts may be delayed in the UI until services catch up; if you continue to encounter problems, please report them to Auvik Support. Next Steps: We will post a final update once we confirm the resolution.

• ◉

  Monitoring

  Aug 17 · 7:15 PM Auvik Networks

  We have applied additional changes to address the issue causing alerts to not appear in the UI. We are seeing alerts gradually recover and are continuing to closely monitor services to ensure a full recovery. Impact: Alert notifications should no longer be sent to unintended users. Alerts may still be delayed in appearing in the UI as services continue to recover. If you continue to encounter problems, please report them to Auvik Support. Next Steps: We will continue monitoring alert recovery and service stability and will provide a final update once we confirm the issue is fully resolved.

• ◉

  Monitoring

  Aug 17 · 9:47 PM Auvik Networks

  We have observed a full recovery of the affected services, and alerts are now appearing in the UI as expected across all clusters. We are continuing to closely monitor services to ensure stability.

• ◉

  Monitoring

  Aug 18 · 4:31 AM Auvik Networks

  We are continuing to monitor for any further issues.

• ◆

  Identified

  Aug 18 · 10:09 AM Auvik Networks

  We continue to receive reports of unexpected emails being sent by legacy alerting. Our team is actively investigating the issue. Next Steps: We will provide another update as more information becomes available.

• ◆

  Identified

  Aug 18 · 11:35 AM Auvik Networks

  We continue investigating the legacy alerting issue causing unexpected email notifications. Our investigation has made progress, and we believe we are moving toward a resolution. We are continuing to validate our findings and work through the remaining steps. Next Steps: We will provide another update as we confirm the resolution or as additional information becomes available

• ◆

  Identified

  Aug 18 · 2:46 PM Auvik Networks

  We continue investigating the legacy alerting issue causing unexpected email notifications. We are continuing to validate our findings and work through the remaining steps. Next Steps: We will provide another update as we confirm the resolution or as additional information becomes available

• ◆

  Identified

  Aug 18 · 5:30 PM Auvik Networks

  We have continued to see stable alerting behaviour following the changes made earlier, and alerting services remain fully recovered. We are continuing to monitor closely to ensure ongoing stability. Impact: Services are operating normally. If you experience any unexpected alerting behaviour or other outliers, please report them to Auvik Support so our team can investigate. Next Steps: We will continue monitoring services and will provide a final update once we are confident the issue remains fully resolved.

• ◉

  Monitoring

  Aug 19 · 5:31 AM Auvik Networks

  We have continued to see stable alerting behavior overnight, and alerting services remain fully recovered. We are continuing to monitor closely to ensure ongoing stability. Impact: Services are operating normally. If you experience any unexpected alerting behavior or other outliers, please report them to Auvik Support so our team can investigate.

• ◉

  Monitoring

  Aug 19 · 10:33 AM Auvik Networks

  We have continued to see stable alerting behavior. We are continuing to monitor closely to ensure ongoing stability. Impact: Services are operating normally. If you experience any unexpected alerting behavior or other outliers, please report them to Auvik Support so our team can investigate.

• ◉

  Monitoring

  Aug 19 · 3:23 PM Auvik Networks

  We have continued to observe stable alerting behavior across all systems. Our engineering team will continue active monitoring through the day to ensure continued performance. Current Impact: All services are operating normally. Critical Reminder: If you experience any unexpected alerting behavior or identify other outliers, please report them immediately to Auvik Support for investigation.

• ◉

  Monitoring

  Aug 20 · 7:18 AM Auvik Networks

  We have continued to observe stable alerting behavior across all systems. Our engineering team will continue active monitoring and address any reported issues. Current Impact: All services are operating normally. Critical Reminder: If you experience any unexpected alerting behavior or identify other outliers, please report them immediately to Auvik Support for investigation.

• ✓

  Resolved

  Aug 20 · 10:11 AM Auvik Networks

  The incident has been fully resolved, and all services are operating normally. Customers should no longer experience any related issues. If you continue to experience problems, please don't hesitate to contact Auvik Support. We will provide a Root Cause Analysis (RCA) once it is available.

• ✓

  Postmortem

  Aug 24 · 1:54 PM Auvik Networks

  # Service Degradation - Legacy Alerts Notification ## Root Cause Analysis ### Duration of the incident Discovered: Aug 17, 2026 13:15 - UTC Resolved: Aug 21, 2026 14:12 - UTC ### Customer impact Some customers using Legacy Alerts experienced inaccurate alert notifications, including false-positive alerts and unintended email notifications. During remediation, some legacy alert processing was delayed, and alert visibility in the user interface temporarily lagged. V2 Alerts remained operational throughout the incident. ### Cause A product release intended to retire older default Legacy Alert configurations for new environments also removed supporting alert conditions that were still required by some existing customers with customized Legacy Alerts. Without the complete conditions, certain Legacy Alerts could trigger inaccurately. The remediation required reprocessing affected alerts, which temporarily increased processing load and contributed to delayed alert visibility, alert flapping, and excessive notifications. ### Effect Affected Legacy Alerts could trigger when the underlying condition was not present, re-trigger after being paused, or send notifications more broadly than intended. The resulting in-service lag also temporarily delayed displaying some alerts. Engineering restored the required alert configuration, removed the unintended broad email association, increased processing capacity, and proactively corrected additional affected customer environments. ### Future consideration\(s\) • Add monitoring for abnormal volumes of Legacy Alerts so unexpected alert activity can be detected earlier. • Expand pre-release and end-to-end testing for existing customized Legacy Alerts, including alert triggering, notification delivery, and post-change recovery behavior. • Strengthen canary-release review so alerting anomalies are investigated and understood before wider deployment.