Many MUA don't recognize charset "eucgb2312_cn" in email header

[tommylikehu]

BPO	44560
Nosy	@warsaw, @terryjreedy, @bitdancer, @corona10

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-07-04.08:32:11.133>
labels = ['type-bug', 'expert-email', '3.9']
title = 'Many MUA don\'t recognize charset "eucgb2312_cn" in email header'
updated_at = <Date 2021-07-09.19:33:45.681>
user = 'https://bugs.python.org/tommylikehu'

bugs.python.org fields:

activity = <Date 2021-07-09.19:33:45.681>
actor = 'terry.reedy'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['email']
creation = <Date 2021-07-04.08:32:11.133>
creator = 'tommylikehu'
dependencies = []
files = []
hgrepos = []
issue_num = 44560
keywords = []
message_count = 3.0
messages = ['396939', '397045', '397210']
nosy_count = 5.0
nosy_names = ['barry', 'terry.reedy', 'r.david.murray', 'corona10', 'tommylikehu']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44560'
versions = ['Python 3.9']

Linked PRs

• gh-88726: Stop using non-standard charset names eucgb2312_cn and big5_tw in email #149959
• [3.15] gh-88726: Stop using non-standard charset names eucgb2312_cn and big5_tw in email (GH-149959) #150491
• [3.14] gh-88726: Stop using non-standard charset names eucgb2312_cn and big5_tw in email (GH-149959) #150492
• [3.13] gh-88726: Stop using non-standard charset names eucgb2312_cn and big5_tw in email (GH-149959) #150493

[tommylikehu]

Email module is used for email message decode and encode, if the header content is gb2312 encoded for example "中文", by design we would finally have a rfc-2047 encoded header as below:

=?eucgb2312_cn?b?1tDOxA==?=

the test script is as below:

from email import header, charset

h = header.make_header([(str("中文").encode("gb2312"),
                         charset.Charset("gb2312"))])
print(h.encode())

My question is why don't we use "gb2312" as the charset in rfc-2047 encoded string, considering the "eucgb2312_cn" is only python awareness.

Thanks

[bitdancer]

I can't tell tell for sure if this behavior is intentional or not from a quick glance at the code (though like you I wouldn't think it would be).

That's part of the legacy api, at this point. The new api will just use utf8:

from email.message import EmailMessage

m = EmailMessage()
m['Subject'] = '中文'

print(bytes(m))

results in

b'Subject: =?utf-8?b?5Lit5paH?=\n\n'

The fix, assuming it is correct, would be to add the line:

'eucgb2312_cn': 'gb2312',

to the CODEC_MAP in email/charset.py, and then specify the internal codec name in your Charset call. I'm not sure that's right, though...once upon I time I think I understood the logic behind the charset module, but I no longer remember the details.

I'd recommend just using the new API and not the legacy API.

[terryjreedy]

Anything before 3.9 only gets security patches.