No components marked as affected
Resolved
This should now be resolved, and all non-duplicate webhooks sent out. The queue returned to normal around ~5:20 PM eastern.
Investigating
We paused the retries to ensure no duplicates are sent, which has led to a new backlog. We are running a process to prevent duplicates, and the queue should be fully cleared in 20-30 minutes
Investigating
As part of clearing that, we're seeing a temporary delay across webhooks. The queue will fully clear in the next 10-15 minutes.
Investigating
From continued investigation, we are seeing this as specific to certain webhook types based on specific changed_field payloads holding these up. The queue should return to normal int he next 20-30 minutes
Investigating
Our team is investigating an increased backlog in our webhooks queue. All job are enqueued and will be sent once the backlog is cleared out.