Engineering organisations measure the latency, error rate and availability of their APIs, databases and queues. Email, which is often just as critical, tends to be measured by vibes: someone notices customers saying they did not get a message, or the open rate looks off. This guide is for CTOs, technical leads and platform teams who want to run email like any other production dependency. It defines the metrics that matter, explains the difference between technical delivery and inbox outcomes, proposes service level indicators and objectives (SLIs and SLOs) you can actually measure, shows how to build a dashboard and alerting that catches problems before customers do, and describes a lightweight incident process. The advice is provider-neutral and works with any sending service, including ours. We do not suggest universal target numbers for bounce or complaint rates, because acceptable levels depend on your mail and the receiving providers' current guidance; we show how to derive your own from baselines. For provider requirements that set hard limits, see Gmail and Yahoo bulk sender requirements.
The email delivery funnel
An email passes through stages, and each needs its own measurement.
| Stage | Question | Where the data comes from |
|---|---|---|
| Created | Did the application create the send request? | Your outbox or queue |
| Submitted | Did the provider accept it? | API or SMTP response, your sending adapter |
| Sent | Did the provider hand it to the recipient's server? | Provider logs and events |
| Delivered (accepted) | Did the recipient's server accept it? | Delivery events |
| Bounced or deferred | Was it rejected or delayed? | Bounce and deferral events. See bounce codes |
| Inbox placement | Did it reach the inbox or the spam folder? | Not directly observable. Estimated from engagement, seed tests and provider dashboards |
| Engagement | Did the user act, for example click the reset link? | Your application events; opens and clicks where tracked |
| Complaint | Did the recipient mark it as spam? | Feedback loops and provider dashboards |
The most important distinction is between delivered and in the inbox. A provider accepting your message does not tell you where it landed. Placement is inferred from indirect signals, which is why outcome metrics from your own application are valuable.
Service level indicators for email
A good SLI is a ratio of good events to valid events over a window. Define them per message type, since a password reset and a weekly digest have different needs.
- Submission success ratio. Messages accepted by the provider divided by messages submitted, excluding permanent validation failures.
- Time to submit. The proportion of messages handed to the provider within a target time from creation. This measures your queue.
- Time to deliver. The proportion of messages accepted by the recipient's server within a target time of submission. For time-critical mail, this is the key latency SLI.
- Delivery ratio. Delivered messages divided by sent messages, per receiving provider.
- Bounce rate. Hard and soft bounces divided by sent. See bounce rate.
- Complaint rate. Complaints divided by delivered messages, where the data is available.
- Deferral rate. Messages delayed by the recipient's server, an early sign of reputation trouble.
- Outcome rate. For example, the share of password reset emails followed by a completed reset within 15 minutes. This is a business-level SLI that catches spam placement, which no delivery metric can see.
From SLIs to SLOs
An SLO is a target for an SLI over a window, for example "99% of one-time-passcode emails are delivered within 60 seconds of creation, measured over 30 days". To choose realistic targets:
- Measure first. Collect 4 to 8 weeks of data for each SLI without setting targets.
- Look at the distribution, not the average. Latency in particular has a long tail.
- Consider the user need. A passcode that arrives after two minutes has failed its purpose, even if it eventually arrives.
- Check provider requirements. Mailbox providers publish thresholds for spam complaint rates and other practices. Treat those as limits you must stay well within, and read their current guidance. See Postmaster Tools setup.
- Set an error budget. If your SLO allows 1% of passcodes to be late, that is your budget for the window. Use it to decide when to pause risky changes.
Separate SLOs by message type and by stream. See separating the streams.
What to put on the dashboard
| Panel | Why |
|---|---|
| Volume by message type and by credential | Spot loops, leaked keys and unexpected changes |
| Queue depth and age of oldest message | Detect backlogs before users complain |
| Submission errors by type | Authentication, rate limits and provider problems |
| Delivery latency percentiles | The key SLI for time-critical mail |
| Bounces by category and receiving provider | Distinguish list problems from provider problems |
| Deferrals by receiving provider | Early signal of throttling or reputation trouble |
| Complaint rate where available | Compare with provider guidance |
| Outcome rates for key flows | Reset completion, verification completion |
| Authentication results (SPF, DKIM, DMARC) | Catch DNS drift. See authenticating your sending domain |
| Reputation indicators | Blocklists and provider dashboards. See reputation monitoring |
Alerting that is useful
- Page on symptoms users feel: time-critical mail latency over its SLO, queue age over a threshold, authentication failures to the provider.
- Ticket on slower signals: rising bounce rates, increased deferrals at one mailbox provider, gradual drops in outcome rates.
- Use burn-rate alerts on your error budget: a fast burn pages; a slow burn creates a ticket.
- Alert on volume anomalies per credential, since sudden spikes are an early sign of compromise or a loop.
- Alert on DNS changes to SPF, DKIM and DMARC records.
Each alert should link to a runbook. A page that says only "bounce rate high" wastes the first ten minutes of the incident.
Feeding data back to the system
Monitoring is wasted if events do not reach your systems. Use webhooks to record delivery, bounce and complaint events against your message records, suppress hard bounces and complainers automatically and reconcile with the provider's logs periodically. See webhooks and designing reliable email sending.
A lightweight incident process
- Detect: alert fires, or support reports missing email.
- Scope: which message types, which receiving providers, since when, what changed (deploy, DNS, volume, template).
- Check the basics: provider status, credentials, queue, authentication records, blocklists. See why emails get blocked and blocklist checks.
- Mitigate: pause a runaway job, fail over a critical stream, reduce volume to a struggling provider, roll back a change.
- Communicate: tell support and affected teams what users should do, for example request another code.
- Review: record the cause, the detection time and what would have caught it sooner.
Common mistakes
- Reporting open rate as a delivery metric. Opens are unreliable because of image proxying and privacy features.
- Looking only at averages and missing the tail latency.
- No separation by receiving provider, which hides a problem at one large mailbox provider.
- Targets copied from a blog post instead of your baseline.
- Alerts without runbooks.
- No outcome metric for key flows, so spam placement goes unnoticed.
If you want to extend monitoring to deliverability reviews, see what a deliverability service includes and our deliverability page.
Get delivery data into your systems Use webhooks and analytics to feed delivery, bounce and complaint events into your monitoring.
References and further reading
Frequently asked questions
What is the difference between delivered and inbox placement?
Delivered means the recipient's server accepted the message. Inbox placement means it reached the inbox rather than spam, which is not directly observable and is estimated from engagement, seed tests and provider dashboards.
What SLIs should I use for email?
Submission success, time to submit, time to deliver, delivery ratio per receiving provider, bounce, complaint and deferral rates and an outcome rate for key flows such as completed password resets.
How do I set an email SLO?
Measure each SLI for several weeks, look at the distribution, consider the user need and provider requirements, then set a target and an error budget per message type.
What should I alert on?
Page on user-visible symptoms such as latency over SLO, queue age and provider authentication failures. Ticket on slower signals such as rising bounces, deferrals at one provider and falling outcome rates.
Why track deferrals?
Deferrals are an early sign that a receiving provider is throttling you, often before bounces or complaints rise.