Email began as a seven-bit ASCII system. Letters with accents, non-Latin scripts, emoji and even typographic quotes were not part of the original design, and the standards that added them did so in layers that still show through today. When you see a subject line turned into =?UTF-8?B?...?= in a log, a name shown as question marks or an apostrophe replaced by three strange characters, you are looking at the seams between those layers. This guide explains the pieces in order: the character set of the body, the transfer encoding that carries it through older servers, the special encoding for headers, display names and addresses, and the newer extensions that allow raw UTF-8 and internationalised addresses. It ends with a diagnostic table and a set of rules that avoid most problems. The advice applies to any language and any sending method, whether you use SMTP or an API.
Three separate things
Most confusion comes from mixing three different concepts. The character set says which numbers stand for which characters; UTF-8 is the universal choice. The transfer encoding says how bytes are carried through mail systems that may be limited to seven-bit text; it is declared with the Content-Transfer-Encoding header. The header encoding is a separate mechanism, because headers cannot use the MIME body rules. When any of the three is wrong or missing, the receiver guesses, and a wrong guess produces mojibake, the garbled text that results from decoding bytes with the wrong character set.
The body: charset and transfer encoding
A well-formed text part declares both:
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<p>Gr=C3=BC=C3=9Fe aus Z=C3=BCrich</p>The charset parameter tells the client how to interpret the bytes. If it is absent, the standard falls back to US-ASCII, and clients will guess, often wrongly. Add the same information inside the HTML, with <meta charset="utf-8">, as a second line of defence for viewers that ignore the header. The transfer encoding is chosen from a small set:
| Encoding | How it works | Best for |
|---|---|---|
| 7bit | Plain ASCII only, lines under 998 characters | English text without special characters |
| 8bit / 8BITMIME | Raw 8-bit bytes sent as they are, if both servers allow it | Mostly supported, but relies on every hop |
| quoted-printable | Non-ASCII bytes written as =XX; ASCII stays readable | Text with few non-ASCII characters; keeps the source legible |
| base64 | All bytes encoded as a 64-character alphabet; about 33 per cent larger | Languages with many non-ASCII characters; binary attachments |
Quoted-printable is the common choice for HTML mail in Western languages; base64 is efficient for text in Chinese, Japanese, Korean, Arabic or Hindi, where almost every character needs several bytes. Libraries choose for you, usually correctly. Problems arise when you build the message by hand or when a gateway re-encodes it. See MIME, multipart and content types.
Headers: RFC 2047 encoded words
The Subject, display names in From and To, and other text headers may not contain raw non-ASCII bytes in the classic format. They use encoded words: =?charset?encoding?text?=, where the encoding is B (base64) or Q (a variant of quoted-printable).
Subject: =?UTF-8?B?UsOpc3Vtw6kgZGUgbOKAmWFubsOpZQ==?=
From: =?UTF-8?Q?Z=C3=BCrich_Team?= <[email protected]>The first example decodes to a French subject with accented letters and a typographic apostrophe. A single encoded word may not exceed 75 characters, so long subjects are split into several words, and a decoder joins them. Do not split inside a multi-byte character; this is a frequent bug in hand-written encoders and produces broken characters at the join. Always use a library function to encode headers: email.header.Header in Python, the mailer classes in PHP, Nodemailer and Java Mail all handle it. Emoji work in subjects the same way, since they are ordinary UTF-8 characters, but clients and filters vary in how they display them, so test.
Display names and addresses
The display name can be encoded with RFC 2047, as shown above. The address itself is stricter. Its domain may be an internationalised domain name such as a name in Cyrillic or Chinese, and in SMTP it travels in its ASCII form, called punycode, which begins with xn--. Your library or provider normally converts it for you. The local part (before the @) with non-ASCII characters needs an extension called SMTPUTF8 (RFC 6531), which both the sending and receiving servers must support. Support has grown but is not universal, so a message to such an address can fail with a message like 553 ... non-ASCII address not supported or a similar code; see SMTP error 553. If you must send to such addresses, check the support of your provider and expect some bounces. For normal use, advertise ASCII addresses.
SMTPUTF8 and raw UTF-8 headers
With the SMTPUTF8 extension, a client can send headers in raw UTF-8, with no encoded words. This only works when every server on the path agrees, and a server that does not support it must reject or downgrade the message. For that reason, the safest approach for outgoing mail remains the classical encoding for headers, which is understood everywhere, with SMTPUTF8 reserved for cases where an address demands it.
Typical symptoms and fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| é shown as é in the body | UTF-8 bytes decoded as Latin-1: charset missing or wrong | Declare charset=UTF-8 in the header and the HTML |
| Question marks instead of characters | Text converted to ASCII or Latin-1 before sending, or the database column is not UTF-8 | Use UTF-8 end to end, including the database connection |
| Subject shows =?UTF-8?B?...?= literally | Subject encoded twice, or the encoded word is malformed | Encode once, using the library, and do not pre-encode |
| Characters broken at the end of a long subject | Multi-byte character split between two encoded words | Use a library; avoid manual splitting |
| Apostrophes and dashes look wrong | Word-processor "smart" punctuation pasted in a non-UTF-8 pipeline | Use UTF-8 or replace with plain characters |
| Attachments with non-ASCII file names are renamed | File name parameter not encoded (RFC 2231) | Let the library encode filename* |
| Plain-text part garbled but HTML part fine | Charset set on one part only | Set the charset on every text part |
| Bounce says non-ASCII address not supported | Receiving server does not support SMTPUTF8 | Use an ASCII alias for that recipient |
Practical rules
- Use UTF-8 for everything. Template files, database tables and connections, application strings and the final message.
- Declare the charset in both places. The MIME header and the HTML meta tag.
- Let a library build the message. Do not concatenate headers and bodies yourself.
- Include a plain-text part and set its charset too. Multipart messages with both parts are better for deliverability, see HTML email templates.
- Test in several clients. Send a message containing accented letters, a non-Latin script, an emoji and a curly apostrophe to test accounts at the major providers, in both subject and body.
- Keep subjects reasonable. Long subjects in scripts that need many bytes per character are cut earlier than you expect; see subject line length.
- Inspect the raw source when something looks wrong. Compare the headers with the examples above; the guide on reading headers helps.
Personalisation and templates
Merge fields bring user-provided text into your message, and user text is where encoding errors most often enter. A name containing an accent, an apostrophe or a character from another script must pass through your template engine, your queue and your sending library without being converted. Test with real names from your user base, including long ones, and avoid truncating strings by bytes, which can cut a character in half. If your tool offers fallback values, make sure the fallback is also UTF-8. See personalisation and localisation.
Encoding problems look mysterious but have a small set of causes. Make UTF-8 the only character set in your system, let a library handle the MIME layers and test with the characters your audience uses. Then your subject lines, names and body text will arrive as you wrote them.
Send multilingual email reliably Use an SMTP relay or API that preserves UTF-8 content and headers.
References and further reading
Frequently asked questions
Why does my email show strange characters?
The receiving client decoded the bytes with the wrong character set, usually because the charset header is missing or the text was not UTF-8 before sending.
Should I use quoted-printable or base64?
Quoted-printable suits text with few special characters, such as French or German. Base64 is more efficient for scripts where nearly every character is non-ASCII.
How do I put emoji or accents in a subject line?
Let your mail library encode the subject as an RFC 2047 encoded word in UTF-8. Do not encode it by hand or twice.
Can I send to an address with non-ASCII characters?
Only if both servers support SMTPUTF8. Domain names are sent as punycode, which is widely supported.
Do I need the meta charset tag in HTML email?
It is a useful second declaration. The Content-Type header is the main one, and both should say UTF-8.