Goal: accept webhooks securely and keep pipelines reliable even under retries, duplicates, and partial failures.
1) Verify signatures
- HMAC with SHA-256: compute the hex digest over the raw request body with your shared secret and compare to the provider's signature header (e.g., GitHub
X-Hub-Signature-256). - Do not mutate the payload before verification; use the exact raw bytes that were signed.
2) Prevent replay attacks
- Timestamp tolerance: require a signed timestamp header and reject events older than a small window (e.g., 5 minutes). Keep server clocks in sync (NTP).
- One-time acceptance: store recent event IDs or signatures and ignore duplicates inside the window.
3) Acknowledge fast, retry safely
- Respond 2xx quickly after basic validation; process heavy work asynchronously.
- Exponential backoff retries: retry only on transient errors (5xx/429/timeouts), with capped attempts and jitter. Do not retry hard 4xx.
4) Make handlers idempotent
- Idempotency keys / event IDs: persist processed IDs with TTL and skip repeats to achieve exactly-once effects over at-least-once delivery.
- Order of effects: verify signature & dedupe before side effects (emails, billing).
5) Operations & monitoring
- Audit trail: log attempt number, signature result, provider event ID, and processing outcome.
- Alerts: error rate spikes, retry depth, and DLQ size (if you enqueue failures).
Cheat sheet
- HMAC-SHA256 over raw body; compare constant-time.
- Signed timestamp; reject outside tolerance (e.g., 5 minutes).
- 2xx fast; retries with exponential backoff only for 5xx/429/timeouts.
- Idempotent consumer: store event IDs / keys with TTL.
Call to action
Want a hardened webhook blueprint? We can review your current handlers and ship a drop-in verification + retry + idempotency module.