Verify Webhook Signatures and Handle Duplicate Events

Webhook delivery is not the same as a normal user request. The sender may retry after a timeout, deliver events out of order, or resend an event you have already processed successfully. A reliable receiver treats authentication, durable acceptance, and business processing as separate responsibilities rather than putting everything inside one route handler.

Define what accepting an event means

Start by documenting the provider's signature scheme, delivery timeout, retry schedule, event identifier, and ordering guarantees. These are provider-specific. Never assume that a successful integration with one payment service proves your receiver is correct for another service with different signing rules.

Use the JSON Formatter to inspect a sanitized event fixture and identify the event identifier, event type, resource identifier, and schema version. Keep that readable fixture for developers, but preserve the raw request bytes separately when testing signatures. Reformatting JSON changes the signed content even when the parsed object appears identical.

Define acceptance as a durable handoff, not completion of every business action. Once the receiver has authenticated and safely stored the event for processing, it can usually return a prompt success response. Follow the provider's specific requirements for response status and timing. Returning success before durable storage risks losing an event permanently if the process stops immediately afterward.

Verify signatures before trusting the payload

Use the provider's maintained verification library whenever possible. It should handle canonicalization, multiple active signing secrets, signature decoding, and constant-time comparison according to the documented protocol. Do not replace an HMAC scheme with an unkeyed hash, and do not assume a generic JWT library can verify an unrelated webhook signature format.

The following route structure is deliberately provider-neutral. Its helper names represent integration code you must implement using the provider's official SDK and your own durable store.

app.post('/webhooks/provider', rawBodyMiddleware, async (req, res) => {
let event;
try {
  event = verifyProviderEvent({
    rawBody: req.body,
    signature: req.get('Provider-Signature'),
    secrets: activeWebhookSecrets
  });
} catch {
  return res.sendStatus(400);
}

await persistVerifiedEventOnce(event);
return res.sendStatus(204);
});

Configure middleware so a global JSON parser does not consume or transform this route's body first. Set a strict body-size limit before buffering it. The Stripe webhook documentation illustrates why raw-body handling and prompt acknowledgments matter, even if you use a different provider.

Add replay protection and durable deduplication

A valid signature authenticates content; it does not necessarily prove freshness. Where the protocol signs a delivery timestamp, enforce the provider's recommended tolerance and monitor server clock drift. An unsigned timestamp field inside JSON cannot independently supply replay protection.

Store the event under a unique constraint scoped by provider and account context. A repeated identifier should normally become a successful no-op at the ingestion boundary. Do not rely on a process-local set: it disappears on restart and cannot coordinate multiple receiver instances.

The insert and durable work handoff must have consistent failure semantics. An inbox table can serve as both the deduplication record and the worker's pending queue. Alternatively, insert an outbox record in the same database transaction and publish it afterward. Writing an event record and then publishing to a separate queue without recovery leaves a gap where accepted events can be stranded.

Return success for an authenticated duplicate already safely accepted. Return an appropriate failure when durable storage is unavailable so the provider can retry. Never turn all database errors into success merely to reduce noisy delivery dashboards.

Make business effects idempotent too

Deduplicating event identifiers is necessary but not always sufficient. Different events may describe the same business transition, such as several updates reporting that one invoice is paid. Enforce the relevant business invariant in the database: one fulfillment per paid invoice, one ledger entry per external transaction, or one entitlement transition per subscription version.

If a worker sends an email and crashes before marking its job complete, retrying may send the email twice. Use an external idempotency mechanism when the destination supports one, or accept and document a bounded duplicate risk where exactly-once effects are not feasible. A queue's delivery guarantee cannot by itself make arbitrary external side effects exactly once.

Handle out-of-order events explicitly. You might retrieve the provider's current resource state before updating a local projection, or compare a documented monotonic resource version. Arrival time is not a trustworthy business version. Avoid overwriting a newer local state merely because an older webhook arrived later.

The GitHub webhook best practices provide another useful operational reference. Keep provider-specific assumptions visible rather than burying them in generic receiver middleware.

Test retries, rotations, and recovery

Create tests for valid signatures, altered bytes, missing headers, stale signed timestamps, duplicate events, and invalid payload schemas. Test a signature generated over JSON with whitespace differences to catch accidental parsing before verification. Also test a valid signature with an unsupported event type; ignoring an irrelevant type should be an intentional policy.

Simulate a worker crash after each side effect. Confirm that retrying does not violate the business invariant and that stuck jobs become visible. Store enough diagnostic metadata to support replay, while applying retention limits and access controls to sensitive payloads. Avoid logging the entire event by default.

Support secret rotation with a deliberate overlap window matching provider behavior. Test old and new secrets, then remove retired secrets only after the transition is complete. Keep signing secrets separate between environments so test deliveries cannot authenticate against production.

Track accepted, rejected, duplicate, pending, and permanently failed events. An endpoint returning fast successes can still hide a stalled queue. Link recovery instructions from your backend operations reference, and use API Latency Tester only with a safe test route; arbitrary unsigned probes do not validate a real webhook flow. For broader event security concerns, consult the OWASP cheat sheets.

Key takeaways

• Verify the original bytes with the provider's documented signature protocol.
• Acknowledge only after a durable, deduplicated handoff succeeds.
• Protect business effects as well as event ingestion against duplicate execution.
• Test reordered deliveries, worker crashes, secret rotation, and operational replay.