Make testing a dependency, not built in

[ryanflorence]

You can't introduce testing w/o introducing religion, unfortunately. For example, I refuse to test my UI in an environment where I can't actually render it in a browser and step through problems in real browsers to fix bugs. Asking folks to eject because they don't like your religion seems a little harsh.

Seems like for testing we ought to be able to have a module on npm called react-scripts-jest, require it in the app's package.json, and then have npm test be something like:

{
  "scripts": {
    "test": "react-scripts-jest"
  }
}

That would be the default or create-react-app, but other folks can create react-scripts-karma or just roll their own inside the project.

[gaearon]

Who would maintain react-scripts-karma?
If we make breaking changes to react-scripts that break it, whose job will it be to fix it?

[kennetpostigo]

This sorta seems familiar to hyperterm plugins. Which most people seem to like.

[gaearon]

I understand the allure of this, I’m just not sure what the plan is with regards to maintenance.

[gaearon]

Also, I’m curious: would you rather prefer we don’t ship testing than we ship browserless testing?

[gaearon]

It seems that if something like react-scripts-karma is maintained by the community, they could implement this proposal even today.

Is changing "test": "react-scripts-jest" to "test": "react-scripts-karma" better in some way than changing "test": "react-scripts test" to "test": "react-scripts-karma"? Since in both cases it’s a community effort and not supported out of the box, I don’t really see the difference, but maybe I’m missing something and I’d love to learn what you think.

[AvivRubys]

As I understand this, there's nothing stopping any community effort from replacing "test": "react-scripts test with "test": "some-community-testing-package".
The only benefit switching create-react-app to a multi package model would give you, is being able to opt out of jest being installed in your project when you're in development. Which doesn't seem like a very big problem.
At the end of the day, it's an opinionated vs configurable sort of issue, but in this case the opinion is easily overridable.

[cpojer]

I think what a lot of people are missing is that we don't have a single line of config inside of the created app that is related to testing, except of course for the scripts entry to run react-script's test command. Anyone can just simply install whatever they like and update the test command with no additional overhead. I think this is reasonable.

[mxstbr]

I agree with @cpojer, I don't understand what the fuss here is about the above is possible already – replace a single command in the package.json and you can use your own test runner!

[ForbesLindesay]

The problem (in as much as there is a problem) is that you cannot easily access the babel/webpack config from another test runner. Any test runner would need to be specialised to react-scripts/create-react-app and know to look up config by doing require('react-scripts/config') which would be super brittle.

I think the solution to this would be something akin to the "configurator" field suggested by @gaearon in #215. With something like this in place, we could easily make other test runners compatible (and make it easy to share code with node.js apps or use eslint plugins in your IDE etc.)

[gaearon]

I think that with time, as things shake out, we might start exporting readonly Babel config for interested tools. Not now though.

[jeffbski]

I agree with making testing a dependency not a built-in. You don't need testing to start creating an app, but it is something that you eventually want to add in. It would be nice that when you get ready to add it, that you could add the suite of preference (like mocha, karma-mocha, jest, ava, ...).

Having the ability to use your favorite test environment is a huge win. If something is built-in and you have to eject to change it that's a terrible experience, plus you now have to cleanup all the unwanted dependencies that it may have taken 10 minutes to download and compile.

Each test environment community could support keeping their dependency up to date, that wouldn't be a burden put on create-react-app. You simply need to define how the dependency/plugin needs to work and those who want to use it can build it and maintain it.

PS. If you still decide to make jest a built-in, please allow for a way to opt-out.

[gaearon]

@jeffbski

Have you had a chance to read through my earlier replies? I don’t really see what the issue is, and unfortunately @ryanflorence hasn’t had time to clarify his concerns yet.

You can use another test runner. There won’t be any code that checks your dependencies and punishes you for using Mocha 😄 . I’m just saying we intend to ship Jest by default, and then it’s up to you if you want to use something else—just replace test command with what you want and fully configure it yourself. You can even extract that to a package and maintain compatibility yourself.

Is there something I’m missing that extracting Jest integration to a separate package would do for your use case? As I said above I don’t see the difference except for weird package layout where some things are in one dependency, and other things are in another dependency. If you ask us to maintain alternative integrations, sorry, we won’t do it—it’s more work than we could put on this project. But I don’t understand how splitting Jest integration into another package can possibly affect you (either positively or negatively) if you want to use Mocha. npm test is yours, you can change your scripts to do whatever.

As @ForbesLindesay noted, you would have to replicate our Babel config (or rely on an internal module that might change), but this would be a problem in any other case except if we start exposing it via an official API. Is this what this issue is about? I’m not sure because it seems focused on us shipping Jest integration via a separate package, rather than us exposing a Babel config.

What is it that you would like to see, exactly?

[gaearon]

[...] that wouldn't be a burden put on create-react-app. You simply need to define how the dependency/plugin needs to work [...]

Implementing a dependency/plugin system is a burden we are not comfortable taking right now. This is exactly what Webpack, Babel, ESLint, and all those other tools do. This is where configuration, incompatibilities, and painful upgrade paths come from. Plugin systems are hard.

In an ideal world we would love to have a plugin system, but we just aren’t there with the default experience yet. Plugin system makes it hard for us to freely change internals, add features without worrying about breaking some third party setup, swap tools for better ones, et cetera. We may do this in a year, but definitely not in the first month.