Postmortem -
Read details
Jun 17, 16:59 UTC
Resolved -
The fix was applied, we are updating the affected envelopes.
Jun 17, 16:18 UTC
Update -
We are continuing to monitor for any further issues.
Jun 17, 16:18 UTC
Monitoring -
We've restarted the affected workers and identified every affected envelope. Envelopes that failed in postprocessing will be recovered automatically — no action needed. Envelopes that failed in preprocessing were not created; please submit them again. We're monitoring to confirm full recovery.
Jun 17, 16:15 UTC
Identified -
We've identified the cause. During the upgrade, some queue workers and brokers kept connecting to the old cluster's addresses instead of the new ones. We're restarting them to restore normal processing.
Jun 17, 16:14 UTC
Investigating -
We're upgrading our queue system. During the upgrade, some envelopes are failing in preprocessing and postprocessing. We're investigating. Envelopes affected in postprocessing will be recovered
Jun 17, 16:09 UTC