This should now be resolved, and all non-duplicate webhooks sent out. The queue returned to normal around ~5:20 PM eastern.
Posted Dec 11, 2024 - 17:35 EST
Update
We paused the retries to ensure no duplicates are sent, which has led to a new backlog. We are running a process to prevent duplicates, and the queue should be fully cleared in 20-30 minutes
Posted Dec 11, 2024 - 17:06 EST
Update
As part of clearing that, we're seeing a temporary delay across webhooks. The queue will fully clear in the next 10-15 minutes.
Posted Dec 11, 2024 - 16:30 EST
Update
From continued investigation, we are seeing this as specific to certain webhook types based on specific changed_field payloads holding these up. The queue should return to normal int he next 20-30 minutes
Posted Dec 11, 2024 - 16:16 EST
Investigating
Our team is investigating an increased backlog in our webhooks queue. All job are enqueued and will be sent once the backlog is cleared out.