Shipup - We're experiencing lag in carrier connections with Colissimo and Chronopost – Incident details

This incident has been fully resolved.

Description

Date: May 15th 2026

We experienced an issue affecting carrier event processing for Chronopost and Colissimo shipments.

The incident started on May 4th 2026, but was only detected later after reports of delayed tracking updates, as it increased over time. The issue originated from our Event Fetching V3 system, where carrier events were being processed with increasing delay.

On May 13th, we mitigated the issue by temporarily rerouting processing from Event Fetching V3 to our legacy Event Fetching V2 system, which reduced the lag and restored normal processing for Chronopost shipments.

On May 15th, a similar issue reoccurred and affected Colissimo shipments as well. The same mitigation was applied to reduce the impact.

The mitigations implemented during the incident were temporary measures. We are continuing to investigate the root cause and work on a permanent solution.

Business consequences of the issue

During the incident period, some merchants experienced delayed shipment tracking updates for Chronopost and Colissimo deliveries.

As a consequence:

• Shipment notifications could be delivered later than expected.

• In some cases, notifications may not have been sent if carrier events exceeded internal expiration thresholds.

• Shipment tracking visibility may have been temporarily degraded.

The issue primarily affected shipments using Chronopost and Colissimo integrations.

The absence of proper monitoring on event-processing lag delayed the detection of the issue and increased the duration of customer impact.

Chronology

• May 4th: Increased lag begins in carrier event processing on Event Fetching V3.

• May 12th: First reports of delayed tracking updates are received.

• May 13th: Investigation confirms lag affecting Chronopost event processing.

• May 13th: Mitigation applied by rerouting processing from Event Fetching V3 to Event Fetching V2.

• May 15th: Similar lag issue detected affecting Colissimo event processing.

• May 15th: Same mitigation strategy applied to reduce impact.

Learnings

• We lacked proper monitoring and alerting on carrier event-processing lag.

• The issue remained undetected for several days before customer reports triggered the investigation.

• Existing fallback procedures using the legacy V2 system helped reduce impact, but are not a sustainable long-term solution.

Short term actions

Description of the short-term actions that could help prevent recurrence:

• Add metrics and alerting on carrier event-processing lag

• Improve operational visibility on Event Fetching V3 processing queues

• Document and standardize mitigation procedures for rerouting processing traffic

Long term actions

Description of the long-term actions:

• Investigate and resolve the root cause of lag generation in Event Fetching V3

• Improve scalability and resilience of the event-processing pipeline

• Review notification expiration rules and evaluate fallback mechanisms for delayed events

• Reduce dependency on the legacy V2 system for incident mitigation

We took a mitigation action that will stabilise the tracking update latency. Trackers are getting up to date as we monitor.