Formatting of email responses in NewDot isn't pretty

[mvtglobally]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

@greeny was sending messages back/forth to a business partner

Expected Result:

When the partner replied via email that the formatting would like nice and there wouldn't be extra line breaks

Actual Result:

Formatting is a bit janky, inc. extra line breaks

Workaround:

unknown

Platform:

Where is this issue occurring?

• Web
• iOS
• Android
• Desktop App
• Mobile Web

Version Number: 1.1.41-3
Reproducible in staging?: need repro
Reproducible in production?: Y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: @mallenexpensify
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1646931708047189

View all open jobs on GitHub

[MelvinBot]

Triggered auto assignment to @MariaHCD (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

[MariaHCD]

I think this could be an issue on the backend when parsing the response (and thus, possibly not something an external contributor could work on)

@zsgreenwald Would you mind DM'ing me more details (eg. the user's email) so I can investigate further via the logs and possibly on Mailgun as well?

[zsgreenwald]

On it!

[MariaHCD]

Looks like we ran into something similar here: https://github.com/Expensify/Expensify/issues/142335

Looking at the logs for the user that @zsgreenwald sent over via DM, looks like we defaulted to mailgun's stripped HTML content instead of parsing it ourselves:

Email request is either using VML/XML namespaces or is from an unknown provider, relying on mailgun's stripped HTML content

cc: @thienlnam Not sure if we can do anything here?

[thienlnam]

Yeah, not much we can do here unless we try to strip it ourselves which leads to endless things we'd have to account for. Since mailgun email stripping has been unreliable we can look into using another library for this except we'd need to test multiple platforms to make sure it works and isn't stripping out text content. We can also strip out whitespace after stripped HTML but that also leads to some unintended changes in other email providers

[MariaHCD]

Thanks, @thienlnam! So it sounds like:

Option 1: Strip the content ourselves (which would be tricky due to the endless number of mail providers that we'd have to account for)

Option 2: Investigate usage of another library (which would entail lots of tests across multiple platforms to ensure the text content isn't being stripped)

Option 3:

We can also strip out whitespace after stripped HTML but that also leads to some unintended changes in other email providers

Could you elaborate on those unintended changes? Maybe we can trim any whitespaces before and after the stripped HTML?

[thienlnam]

Yup those options are correct, I think option 2 would actually be worthwhile to do

Could you elaborate on those unintended changes?

It's been a while so I might not be remembering correctly, but IIRC stripping whitespace messed up the email formatting inside the email and made (yahoo?) emails look especially bad. I think we had this enabled at one point but decided to revert it.

Maybe we can trim any whitespaces before and after the stripped HTML?

Yeah this could help a bit, though it looks like in the linked issue the spacing is bad between text and before/after the text too and so it would still look bad

[MariaHCD]

Great, thanks, @thienlnam! So moving forward, sounds like the best option would be to investigate usage of another library and this is something that can be handled internally as it is a backend change.