A product that sends email for password resets, receipts, alerts and verification has an availability requirement for email whether or not anyone wrote it down. When mail stops, users cannot log in, orders look unconfirmed and support tickets spike. Yet many applications still send email inline in the request path, with a single provider, no retry policy and no way to tell whether a message was sent twice. This guide is for CTOs, architects and technical leads who want a design that survives the common failures: a slow or unavailable provider, a spike in volume, a bug that loops, a deploy that restarts workers mid-send and a delivery problem that appears only at one mailbox provider. It describes a reference architecture built around a queue, explains retries and idempotency, shows how to rate limit and separate streams, covers failover to a second provider and explains what to test. The patterns are provider-neutral. For choosing the provider itself, see the vendor scorecard.
Start with the failure you are designing for
| Failure | Symptom | Design response |
|---|---|---|
| Provider slow or unavailable | Timeouts, queue growth, stuck user flows | Queue, retry with backoff, circuit breaker, standby route |
| Traffic spike | Rate-limit errors, delays | Rate-limited workers, priority queues |
| Duplicate sends | Customers receive the same email twice | Idempotency keys and send records |
| Lost messages | No email, no error | Durable queue, outbox pattern, reconciliation |
| Runaway job | Thousands of unintended emails | Per-service quotas and kill switch |
| Bad credentials or key rotation | All sends fail with authentication errors | Alerting, separate keys, tested rotation procedure |
| Webhook or event loss | Bounces not recorded; repeated sends to dead addresses | Provider retries, idempotent handlers, reconciliation job |
| Delivery degradation at one mailbox provider | Complaints, deferrals for one domain group | Monitoring by receiving provider, throttling. See monitoring and SLOs |
Reference architecture
- Application code creates a "send request", not an email. It contains the recipient, template or content, a message type, a priority and a unique key.
- The request is written durably in the same database transaction as the business event (for example, the order). This is the transactional outbox pattern. It prevents the case where the order is saved but the email is lost, or the email is sent for an order that rolled back.
- A worker reads the outbox or queue and builds the final message.
- A sending adapter calls the provider by SMTP or API, wrapped with timeouts, retries and logging. Only this adapter knows provider details. See email API vs SMTP relay.
- The result is recorded: accepted, failed permanently, or retry scheduled, plus the provider's message identifier.
- Events from the provider (delivered, bounced, complained) arrive by webhook, update the record and feed suppression.
The key benefit is isolation. A provider problem slows the worker, not your checkout. A deploy restarts workers without losing messages. And you have one record per message that support can look up.
Retries and backoff
Classify outcomes before you retry.
| Outcome | Action |
|---|---|
| Network error, timeout, 5xx or temporary SMTP code (4xx) | Retry with exponential backoff and jitter, up to a maximum age |
| Rate-limit response | Back off for the indicated period and slow the worker |
| Validation error, permanent SMTP code (5xx) for a recipient | Do not retry; mark failed and suppress if appropriate |
| Authentication error | Stop and alert; retrying will not help |
| Unknown outcome (timeout after sending) | Use idempotency or check before retry |
Set a maximum age for retries that matches the message's usefulness. A one-time passcode that arrives an hour late is worse than no message, so expire it and let the user request another. A receipt can wait longer. For the meaning of SMTP codes, see SMTP bounce codes explained.
Idempotency: avoid sending twice
The hard case is the timeout after the provider has accepted the message but before you receive the response. A naive retry sends a second email. Defences, in order of preference:
- Provider idempotency keys, if the provider supports them. Send a stable key per logical message.
- Your own send record. Before sending, mark the message as "sending" with a unique key. After a timeout, check the provider's log by your message identifier or custom header before retrying.
- Idempotent consumers upstream. Make the event that triggers email (for example, "order paid") processed exactly once, using a unique event identifier.
Accept that no design gives perfect exactly-once delivery. Aim for at-least-once with strong de-duplication, and design messages so an occasional duplicate is harmless, for example by including a clear order number.
Rate limiting and priorities
- Priority queues. Separate time-critical mail (passcodes, password resets) from bulk or digest mail so that a large batch never delays them.
- Per-service quotas. Give each internal service a limit, and alert when it is approached. This catches loops.
- Worker concurrency. Set it based on provider limits and observed latency, not CPU.
- Kill switch. A configuration flag that pauses a message type immediately.
Keep marketing and transactional streams on separate credentials, domains and, where justified, IP pools. See separating the streams and dedicated vs shared IPs.
Failover to a second provider
For mail that must arrive, such as authentication codes, consider a standby route. The design decisions are:
- Trigger. Fail over on sustained errors or latency measured by a circuit breaker, not on one failure.
- Authentication. The standby provider needs your domain's SPF and DKIM set up in advance. Test it regularly; an untested standby is not a standby. See multiple vendors on one domain.
- Reputation. A standby that never sends has no reputation. Send a small steady share of traffic, for example internal or low-risk mail, through it.
- Suppression. Both providers need the same suppression list. Keep it in your own database and push it to each.
- Cost and complexity. Weigh against the cost of an outage. For many products, a durable queue and a quick manual switch are enough.
Keep your provider code behind an interface so the standby is another adapter. This also makes a future migration easier; see the migration checklist.
Handling events and feedback
Process provider events with idempotent handlers, since providers may deliver an event more than once or out of order. Verify signatures. On a hard bounce or complaint, update your suppression list before any further send. Run a periodic reconciliation job that compares your records with the provider's logs, to catch events you missed. See webhooks.
Testing
- Unit test the retry classification with fake provider responses.
- In staging, make the provider fail on purpose (wrong credentials, blocked network, artificial delay) and observe the queue, alerts and recovery. See testing email in staging.
- Kill a worker mid-send and verify that no message is lost or doubled.
- Replay a webhook event twice and check the effect.
- Rotate credentials and confirm the process works.
- Run a load test at ten times your normal peak against a sandbox.
Common mistakes
- Sending email inside the HTTP request and failing the user's action when mail fails.
- Retrying everything, including permanent failures, which hurts reputation.
- No maximum age on retries.
- One shared queue for all mail types.
- An untested standby provider.
- Treating accepted by the API as delivered. See bounce rate guidance.
Build on a relay designed for this Per-service credentials, STARTTLS on port 587, delivery events and webhooks to support a queue-based design.
References and further reading
Frequently asked questions
Why not send email inside the web request?
A slow or failing provider would slow or break the user's action. A queue decouples sending from the request and lets workers retry safely.
What is the outbox pattern for email?
Write the send request in the same database transaction as the business event, then have a worker send it. This prevents lost emails and emails for rolled-back events.
How do I avoid duplicate emails on retry?
Use provider idempotency keys, keep your own send records and check logs after an unknown outcome, and make upstream event handling idempotent.
Which failures should I retry?
Temporary errors such as timeouts, 5xx responses and 4xx SMTP codes with backoff and a maximum age. Do not retry validation errors, permanent recipient failures or authentication errors.
Do I need a second email provider?
Only for mail that must arrive and where an outage is costly. A standby must be authenticated and tested in advance and share your suppression list.