Revisit Compatibility

[stasm]

(Previous discussions: #133, #269)

Summary

I'd like to start a long-term clean-up project which aims at the following:

• Simplify the build pipeline and reduce the number of dev dependencies.
• Focus on natively supporting ES modules in browsers and in Node.
• Increase the usefulness of CI testing.
• Don't break the current compatibility of compat.js builds.

The Status Quo and Its Problems

• We currently don't support ES modules in Node.js.
  • As of Node.js 13.2, only files ending in .mjs or files under a folder hierarchy with a package.json with {"type": "module"} can be imported rather than required.
  • We have neither.
• Our build pipeline is complex.
  • We use Rollup to bundle our files.
    • It's not really needed for using Fluent in Node.js and as a dependency for larger projects with their own build pipelines.
    • OTOH, a single-file package is easy to include Fluent in jsfiddle, codepen etc.
      • Then again, all of modern browsers support <script type="module">.
  • We use Babel to transpile the compat.js build.
    • We use preset-env to define what should be transpiled. The browser matrix was chosen not based on usage data, but based on the JS features we didn't want to transpile. That's kind of backwards of how preset-env should work.
  • In Migrate to TypeScript? #376 we're talking about adding TypeScript to the mix, which would further complicate the pipeline, testing, debugging etc.
• We write tests using ES modules and make them work in Node by patching require.
  • This is done using @babel/plugin-transform-es2015-modules-commonjs or esm.
• We test code in different version of Node (good), but we do it by transpiling our unit tests with a Babel setup which is different from the one which we use for shipping (bad).

Short and Mid-Term Ideas

I'd like the clean-up to be careful and incremental. In no particular order:

• Add integration tests.
  • Create a new test suite using our compat.js builds and run it in different versions of Node and older browsers without any further transpilation.
  • Switch our unit tests to only run in the currently active version of Node.js.
• Drop preset-env and transpile to ES2017 which we effectively do right now anyways.
• Ship ES modules next to CommonJS files.
  • There isn't an established good practice about how exactly to do it right now.
  • Node.js has some docs about proposed approaches.
  • The blog post (same as above) mentions January 2020 as the target date for finalizing the official docs about this.
• Maybe drop Rollup?
  • Once we fully support ES modules, will there still be a strong case of bundling our files in a UMD package?
• Maybe migrate to TypeScript?
  • See Migrate to TypeScript? #376 for the discussion about TypeScript.
  • One more tool in the pipeline, but..
  • ...we could possibly drop Babel thanks to TypeScript's target compiler flag.

Long-Term Ideas

Some further ideas for when all Node LTS versions support ES Modules without flags (from mid-2022 forward).

• Drop CommonJS
  • Node 12 will be EOLed in April 2022.
• Drop esm and @babel/plugin-transform-es2015-modules-commonjs when running tests.
  • This requires that we use a test runner which natively supports ES modules. Work is underway in both mocha (which we currently use) and jest to enable this.
  • It might still be months away today, as it likely requires the "custom loaders" API to stabilize in Node.js. See the bottom of this blog post.

[stasm]

I re-organized the list of ideas in my comment above and split them in two groups: short and mid-term ideas, and long-term ones. The long term is related to the timeline of enabling ES module support in Node without any flags. ES modules supported was unflagged in Node 13, which means that the first LTS to have it will be Node 14. Sadly, this also means that for the next two years we'll also have to deal with Node 10 and Node 12. At the same time, it's not a big issue and the esm module works wonders even right now.

In the shorter term, I'd like to focus on simplifying the build system. In #376 I've been tracking my progress migrating @fluent packages to TypeScript. Today, only @fluent/dom hasn't been ported, and in #470 I've suggested we do it later this year. Thanks to TypeScript, we could remove Babel from the list of dev dependencies, and instead take advantage of tsconfig's target field. In fact, I think we could also take this opportunity to remove compat.js artifacts completely. I filed #472 about this.

I don't think we should completely drop Rollup, as suggested in one of the ideas in my earlier comment. It's useful to be able to bundle all code into a single file and use that for a quick experimentation, or for a project which doesn't use a build system.

I think we could also use that single file as a CommonJS-specific entry point for Node apps. If we called it index.cjs rather than index.js, old Node (10, 12) wouldn't care, and Node 13+ would automatically recognize it as a CommonJS file. We could then mark the rest of the package as type: "module".

Finally, I've been thinking about building a suite of integration tests. It appears that there are paid-for solutions like BrowserStack which could be useful for running the suite on multiple versions of browsers. Or, we could try to set up dockerized Selenium ourselves. But I do wonder if that wouldn't be an overkill. I haven't seen such extensive compatibility testing done for library projects.