Resolved -
This should now be resolved, and all non-duplicate webhooks sent out. The queue returned to normal around ~5:20 PM eastern.
Dec 11, 17:35 EST
Update -
We paused the retries to ensure no duplicates are sent, which has led to a new backlog. We are running a process to prevent duplicates, and the queue should be fully cleared in 20-30 minutes
Dec 11, 17:06 EST
Update -
As part of clearing that, we're seeing a temporary delay across webhooks. The queue will fully clear in the next 10-15 minutes.
Dec 11, 16:30 EST
Update -
From continued investigation, we are seeing this as specific to certain webhook types based on specific changed_field payloads holding these up. The queue should return to normal int he next 20-30 minutes
Dec 11, 16:16 EST
Investigating -
Our team is investigating an increased backlog in our webhooks queue. All job are enqueued and will be sent once the backlog is cleared out.
Dec 11, 16:07 EST