Create a testing process

[acjh]

• To formalise testing, and for regression testing.
• Preferably automated.
• Possibly a different repo.

Currently, there are no tests.

[nicholaschuayunzhi]

Perhaps we could have a separate test site repository that contains multiple sub-sites (which test for different components/features) where we can test markbind build/include for each of them. We then diff against previous site built and see if there are any changes.

[acjh]

Yes, that could be a starting point.

Some thoughts:

• We should differentiate between "issues" (reproducing bugs) and "PRs" (regression testing).
• A tester should not have to use multiple build commands. Fewer sub-sites, more pages?
• There should be an index with a brief description of each page, not rely on GitHub issues.

[nicholaschuayunzhi]

A tester should not have to use multiple build commands. Fewer sub-sites, more pages

Maybe we can use a script to execute multiple build commands?

[acjh]

Can you explain your preference for sub-sites?

While a script can do that, sub-sites seems unnecessary (unless testing for submodules specifically) and involves duplication of files.

[nicholaschuayunzhi]

Can you explain your preference for sub-sites?

My idea was to treat each sub site as a unit test so that we can test each feature in isolation. This could help with debugging when we make changes to code base.

We can also have one site where we integrate the features together.

[nicholaschuayunzhi]

Additionally when a new component is added, the dev has to provide a separate sub site and also add his feature into the site with integrated features.

[acjh]

My idea was to treat each sub site as a unit test so that we can test each feature in isolation.

Pages do not affect one another, as far as this measure is concerned.

Additionally when a new component is added, the dev has to provide a separate sub site and also add his feature into the site with integrated features.

Those would be demo sites, rather than test sites, and maintenance will burden future developers.
Even if we were to have a demo site, one is enough — like vue-strap.
It is reasonable to expect a feature PR to add a page, but to add a functional site seems too much.

To reiterate, the goal of a testing process is to formalise testing.

[nicholaschuayunzhi]

Pages do not affect one another, as far as this measure is concerned

Oh my bad. Thanks for clarification. In that case i agree that we can split into pages.

[Gisonrg]

Compare site by site could be large (consider it as integration test )
Some unit test would also help, for example, construct an example site folder in the project. Create a Site object based on that, and see if it can read Site.json correctly, list correct number of assets, or read sub site.json correctly. (We don't need to compile and render stuff). Similar to Page object.
Similarly, if we want to test rendering, set up unit test in markbind and see if we could produce expected fragment would be good. (like the <= case)

[acjh]

Resolved by:

• Add a test site markbind-cli#27 Add a test site
• Add Travis CI for automated linting and testing markbind-cli#36 Add Travis CI for automated linting and testing
• Add jest for unit testing #178 Add jest for unit testing