Integrate Markdown linter to enforce consistency

[tlylt]

Is there an existing issue for this?

• I have searched the existing issues

Any related issues?

No response

What is the area that this feature belongs to?

No response

Is your feature request related to a problem? Please describe.

Inspired by a recent review comment on my PR that some of the markdown I wrote weren’t compliant with the recommended style, perhaps would be good to have a linter to enforce it instead of relying on manual checking.

Describe the solution you'd like

Integrate a Markdown linter both locally as part of linting, as well as CI check.
For possible packages to use, there’s https://github.com/DavidAnson/markdownlint which seems to be quite promising.

Describe alternatives you've considered

No response

Additional context

No response

[ang-zeyu]

Implementation wise this might be a little difficult as MarkBind extends commonmark, but maybe there's some configuration options in there that could fix this 🤔

Inspired by a recent review comment on my PR that some of the markdown I wrote weren’t compliant with the recommended style, perhaps would be good to have a linter to enforce it instead of relying on manual checking.

Which PR is this? 👀
(not aware that we had a recommended style)

Just my two cents,

I don't think linting documentation would turn out to be quite worth the maintainence costs of introducing such linters into our toolchain.

I'm also personally somewhat against introducing silly stringent standards like "ensure paragraphs stay on one line, enable word wrap in your ide" which introduces more trouble for contributors, and does not produce any real tangible readability benefits like eslint does with code.

This is not to say we shouldn't aim to make our documentation look consistent to the reader (which, most of the syntactic rules in https://github.com/DavidAnson/markdownlint do not help with), but I think this may be better achieved through PR reviews as we've been doing so far.

E.g. I do not think it poses a readability issue to developers if you use either of these. (much less, to the reader)

* list

- list

*Nor to say it shouldn't be at least somewhat readable to developers (which I think documentation has a really low bar for)

e.g. this is blatantly not ok (I do not think linters would be able to catch such cases either):

This is an
overly wrapped
sentence that
looks like a
poem

[tlylt]

Which PR is this? 👀 (not aware that we had a recommended style)

#1872 (comment)

Mentioned here https://markbind.org/devdocs/devGuide/styleGuides.html

[ang-zeyu]

A simple analogy to the point on documentation source files having a really low bar might be that we are able to read microsoft word documents from anyone 😂
(although slightly inappropriate - as the word document itself is both the documentation and the source file)

[ang-zeyu]

Which PR is this? 👀 (not aware that we had a recommended style)

#1872 (comment)

Mentioned here https://markbind.org/devdocs/devGuide/styleGuides.html

Thanks for pointing this out. I missed this in #1246

pinging @damithc for discussion

[damithc]

As the amount of our documentation grows, it's nice if it's follows a consistent style, although we don't want to be overly prescriptive.
Some style rules actually have tangible benefits and help to educate the writer (the writer may not be aware of those benefits or those syntax options). e.g., using 1. for every item in numbered lists
We can start by enforcing just a few rules that we think have value and pilot it for a while. If it's getting in the way, we can get rid of it.
We are not in dire need of a linter right now though (i.e., the expected benefit is not that high). So, it depends on how much effort is required for adopting one too (i.e., the cost shouldn't be high either).

[ang-zeyu]

it's nice if it's follows a consistent style, although we don't want to be overly prescriptive.

Some style rules actually have tangible benefits and help to educate the writer (the writer may not be aware of those benefits or those syntax options). e.g., using 1. for every item in numbered lists

We are not in dire need of a linter right now though (i.e., the expected benefit is not that high). So, it depends on how much effort is required for adopting one too (i.e., the cost shouldn't be high either).

Agreed as well, if there are certain rules with real readability benefits, we can look into it.

This issue would be very low priority too, I'll leave this open for investigation if anyone is interested.

Implementation wise this might be a little difficult as MarkBind extends commonmark, but maybe there's some configuration options in there that could fix this 🤔

To give one example, we modify the markdown/html parser to allow parsing things like <puml>, which can contain almost anything. The linter likely won't be able to accomodate this as nicely.

---

Just adding some anecdotal experience from my past 2 years reviewing PRs here too 😅, I've never really encountered any issues with readability in documentation source files syntax from anyone, even the most trivial comments are only related to grammar / wording.

I think there is a natural tendency to follow ⬇️ even without a linter (also rather surprised markdown linters exist as such):

it's nice if it's follows a consistent style, although we don't want to be overly prescriptive.

Nevertheless, ⬇️

Agreed as well, if there are certain rules with real readability benefits, we can look into it.

[tlylt]

Implementation wise this might be a little difficult as MarkBind extends commonmark, but maybe there's some configuration options in there that could fix this 🤔

To give one example, we modify the markdown/html parser to allow parsing things like <puml>, which can contain almost anything. The linter likely won't be able to accomodate this as nicely.

Some custom configuration will be necessary, and perhaps not too difficult as well (something like configuring Eslint options, which is quite typical)

Just adding some anecdotal experience from my past 2 years reviewing PRs here too 😅, I've never really encountered any issues with readability in documentation source files syntax from anyone, even the most trivial comments are only related to grammar / wording.

The downside of that is now we do have some inconsistencies and while some reviewers may not look at markdown styling, some do (and in the future too). This means time is spent on giving such feedback and making the adjustments. The time that could be avoided if there's help for the authors to write the document consistently in the first place. One important fact is that updates to the existing documentation is going to mirror what has been done. If things were inconsistent, the documentation will grow more so in the future. Low impact still, but not very nice.

Linters do more than linting nowadays in our IDE. They provide warnings, links to related rules, an explanation of the rationale behind such rules, and auto fixing that can be of great help for anyone who's lazy (practically everyone). Some rules besides aesthetic, does more help in terms of informing the writer before he needs to serve the site and obverse the error in the browser. Those squggly lines can save debugging time, especially when you happen to casually miss a blank line.

Again, some rules may not be easily enforced and some may not be very useful, but some do:
(Just taking one example that I saw)
https://github.com/DavidAnson/markdownlint/blob/v0.25.1/doc/Rules.md#md042---no-empty-links

I think there is a natural tendency to follow ⬇️ even without a linter (also rather surprised markdown linters exist as such):

it's nice if it's follows a consistent style, although we don't want to be overly prescriptive.

To rely on memory or personal grasp of the markdown formats in my opinion is just not going to cut it, because rules when not enforced, cease to exist.

The point is to make the product better, which sometimes involves a trade-off between the quality of the product and the effort required. I agree on that due to the low significance, we should make such improvements that do not make the developers less productive when writing docs.

As a first-step/low hanging fruit, we can document the existence of such tools and provide a MarkBind friendly config file in our DG, for anyone who's interested. This is similar to the debugger settings that we currently have for VScode/webstorm. This is focusing on helping the developers while not enforcing the rules via strict linter checks before push/merge.

I might do just that when I got the time...(I have the markdownlint extension installed in my VScode, but haven't got the time to play around with it 😓 )

[ang-zeyu]

Some custom configuration will be necessary, and perhaps not too difficult as well (something like configuring Eslint options, which is quite typical)

one example, in a "vanilla" markdown parser like markdown it:

(if we don't want lists in trees to be validated since they aren't really trees)
https://github.com/DavidAnson/markdownlint/blob/v0.25.1/doc/Rules.md#md032---lists-should-be-surrounded-by-blank-lines

foobar  // markdown paragraph start
<tree>  // markdown continue
* Item 1  // parsed as list (try in the github renderer for example)
  * Nested Item 1
  * Nested Item 2
</tree>

or issues with markdown-it-attrs, etc.

Not quite sure if these would have configuration options, as they require rather significant markdown parser modifications.

[damithc]

Ultimately, I'm weighing my opinion on this based on what I've observed in these past 2 years vs the cons of a markdown standard. Which, I really believe everyone has been doing a good job, and that "proof is in the pudding" here. I'd much rather we invest our time and 🧠 into the goal of reader communication in documentation. Things like this https://www.gatsbyjs.com/contributing/gatsby-style-guide.

Yes, this kind of guidance is indeed good and more important.

I'm going to controversially suggest therefore we remove the markdown standard from the developer guide at all, which we've been doing ok without.

I have no violent objections to removing it. I value consistency in style however. If it turns out future PR authors are not being consistent with the existing style we can consider bringing it back.

[ang-zeyu]

*In case anyone's looking for 2 comments, I deleted a largely similar comment with one very glaring assumption, and decided to revisit this when I have a little more time to think through. Backtracked shortly and decided it was worth to post my current 2 cents for discussion anyway.

This comment goes chronologically before #1883 (comment). Thanks for the speedy input @damithc

*This comment is in view of my concluding controversial suggestion, not just whether to have a linter:

I'm going to controversially suggest therefore we remove the markdown standard from the developer guide at all, which we've been doing ok without.

Linters do more than linting nowadays in our IDE. They provide warnings, links to related rules, an explanation of the rationale behind such rules, and auto fixing that can be of great help for anyone who's lazy (practically everyone). Some rules besides aesthetic, does more help in terms of informing the writer before he needs to serve the site and obverse the error in the browser. Those squggly lines can save debugging time, especially when you happen to casually miss a blank line.

Again, some rules may not be easily enforced and some may not be very useful, but some do:
(Just taking one example that I saw)
https://github.com/DavidAnson/markdownlint/blob/v0.25.1/doc/Rules.md#md042---no-empty-links

Agree things like this could benefit. (in this particular case though, implementing hash checking might be the right direction)

My argument was for syntactic consistency, per the issue title.

The majority of rules like this https://github.com/DavidAnson/markdownlint/blob/v0.25.1/doc/Rules.md#md009---trailing-spaces, I'm really having a hard time believing would have any real benefits. Or, when they do pose a tangible readability issue to the reader (e.g. https://github.com/DavidAnson/markdownlint/blob/v0.25.1/doc/Rules.md#md001---heading-levels-should-only-increment-by-one-level-at-a-time), are already being dealt with seldomly in our PRs.

The downside of that is now we do have some inconsistencies and while some reviewers may not look at markdown styling, some do (and in the future too). This means time is spent on giving such feedback and making the adjustments. The time that could be avoided if there's help for the authors to write the document consistently in the first place. One important fact is that updates to the existing documentation is going to mirror what has been done. If things were inconsistent, the documentation will grow more so in the future. Low impact still, but not very nice.

There definitely are inconsistencies, and it is definitely "not as nice". It is not a win-for-all decision. My argument and analogy on the point with microsoft word is that these inconsistencies do not affect developer readability in any meaningful way, and / or are already enforced to a "good enough" degree by a natural tendency to adhere to the majority style in our documentation.

The biggest case in point I am raising therefore is that we have been doing ok for the past 2 years (and then likely even before that for 3 years) (MarkBind is ~5 years old) without a markdown syntactic consistency standard.

There are also anecdotally many successful and much larger projects that do not have such standards imposed. And even when such standards are imposed, there is often disagreement on which ones have the winning rationale. In which case I really believe we don't bother imposing such subjective standards at all. E.g.

• https://se-education.org/guides/conventions/markdown.html (line wrapping)
• https://docs.saucelabs.com/contributing/style-guide/#content

This means time is spent on giving such feedback and making the adjustments.

Do not forget:

• The time spent by N contributors over P PRs and K standards correcting things (which, even with a linter, adds up)
• The increased setup work needed to integrate linting tools for contributors and on our end
• The increased barrier to entry to getting contributors at all, which is paramount for an open source project
• The cognitive overhead for reviewers to impose such standards (which, even with a linter, adds up), distracting from reader readability issues that do matter in documentation writing

some reviewers may not look at markdown styling

no, this assumes we agree on imposing such standards at all. Inconsistent stylistic reviewing standards is the last thing an open source project should be having.

To rely on memory or personal grasp of the markdown formats in my opinion is just not going to cut it, because rules when not enforced, cease to exist.

Yes, it won't. See my concluding suggestions.

Ultimately, I'm weighing my opinion on this based on what I've observed in these past 2 years vs the cons of a markdown standard. Which, I really believe everyone has been doing a good job, and that "proof is in the pudding" here. I'd much rather we invest our time and 🧠 into the goal of reader communication in documentation. Things like this https://www.gatsbyjs.com/contributing/gatsby-style-guide.

This is not something like code that far too often runs into severe consistency issues that really do affect code readability, in which case, eslint does help.

---

@damithc @tlylt

I'm going to controversially suggest therefore we remove the markdown standard from the developer guide at all, which we've been doing ok without.

To be clear as well, I'm still welcoming PRs like https://github.com/MarkBind/markbind/pull/1829/files as and when needed. But these things are few and far between and self-evident without having to explicitly enforce a markdown standard.

My argument and analogy on the point with microsoft word is that these inconsistencies do not affect developer readability in any meaningful way, and / or are already enforced to a "good enough" degree by a natural tendency to adhere to the majority style in our documentation.

"proof is in the pudding"

Pinging for a few more opinions as well if this is really the case or just "in my head" @ryoarmanda @jonahtanjz @wxwxwxwx9 @raysonkoh

As a first-step/low hanging fruit, we can document the existence of such tools and provide a MarkBind friendly config file in our DG, for anyone who's interested. This is similar to the debugger settings that we currently have for VScode/webstorm. This is focusing on helping the developers while not enforcing the rules via strict linter checks before push/merge.

I am also not against this to be clear (especially the more functional rules in there). Making it optional will alleviate most of the concerns. But not as a "first-step", but a final one.

[tlylt]

I think I have put forth my case in enough detail in my previous comment, so I am just going to add a few points for completeness. (Should have spent this time revising for finals, but oh well)

Which, I really believe everyone has been doing a good job, and that "proof is in the pudding" here.

I don't mean to be overly critical here. We certainly have generated a ton of good content, by no means sub-standard. I am pointing out the room for improvement, which in my opinion is always something worth bringing up, especially in open source projects.

My argument was for syntactic consistency, per the issue title.

While it is important to make documentation consistent FOR the reader, I am seeing a slightly less, but nonetheless valid concern about the consistency of our documentation code (don't forget, markdown is a lightweight markup language, which I feel comfortable enough to respect it in some regard, as "code"). Again, this is an open-source project, a collaboration done among multiple developers, across a totally different time frame. Important context.

no, this assumes we agree on imposing such standards at all. Inconsistent stylistic reviewing standards is the last thing an open source project should be having.

This depends on what we do with https://markbind.org/devdocs/devGuide/styleGuides.html, because I believe while not explicitly stated as a reviewer's responsibility, this is enough for a developer to assume that there is a markdown code style that we ought to follow.

My argument and analogy on the point with microsoft word is that these inconsistencies do not affect developer readability in any meaningful way, and / or are already enforced to a "good enough" degree by a natural tendency to adhere to the majority style in our documentation.

The problem with this analogy is that, for Microsoft Word/PowerPoint, the reference is missing the important context we have here: "multiple actors, across time". And just to make one case in this equal analogy: in Microsoft words, one can use multiple spaces to serve as a linebreak (pushing the content from one page to the next). While this works fine, whenever some content is added in the front, the spaces push the content on the next page further down. If a linebreak is used, the content on the second page will remain on top until the previous page is filled. Non-obvious choice, no immediate impact, could manifest in the future, when someone else is editing the document.

Things like this gatsbyjs.com/contributing/gatsby-style-guide.

Even though Gatsby does not place an emphasis on this, others do: See examples here. The point is not to find projects that do adopt this, the point is show that it can be done in reasonable fashion that other projects find it worthwhile.

Do not forget:

The time spent by N contributors over P PRs and K standards correcting things (which, even with a linter, adds up)
The increased setup work needed to integrate linting tools for contributors and on our end
The increased barrier to entry to getting contributors at all, which is paramount for an open source project
The cognitive overhead for reviewers to impose such standards (which, even with a linter, adds up), distracting from reader > readability issues that do matter in documentation writing

The point of having such linting tools is exactly to make these pain points go away. For example with a slight barrier, we eliminate/greatly reduces the other efforts required. Hence this list is a little misleading in that they are not necessarily a package. (Some trade-offs may be required, but those trade-offs could be worth it)

I am also not against this to be clear (especially the more functional rules in there). Making it optional will alleviate most of the concerns. But not as a "first-step", but a final one.

To be clear, I am not in favor of the argument over - or * when writing a list. I am open to exploring the existing rules and if they are useful (functional or otherwise), I am open to adopting them. If they are not, then we shouldn't be. In any case, we should not make it a false dichotomy.

I thank @ang-zeyu for providing your insights into the problem, which are indeed enlightening. From a project direction perspective, I'm in no position to push anything forward, and hence I should state here that this issue is more for discussion than anything (at least for me). Pinging @ong6 @kaixin-hc @jovyntls for thoughts and comments since they have been working on this project lately, and have written enough documentation to chime in.

[ong6]

I feel we should add a formatter first given that most people use auto format as a means to fix their basic layouts. But there's also no harm in adding a simple linter. I personally markdownlint as well.

Though the value of a linter may be limited (compared to a formatter) as different pages might want to have different layouts and a linter could be quite annoying by stopping user freedom.