Configure external link recognition (and handling)

[chrisjsewell]

Currently, if a link ([text](link)) does not match a URL scheme (e.g. 'http://...') then it is treated as an internal cross-reference (to a reference target or sphinx document, etc):

MyST-Parser/myst_parser/docutils_renderer.py

Lines 412 to 415 in 3d5ae4f

	url_check = urlparse(destination)
	# If there's not a url scheme (e.g. 'https' for 'https:...' links),
	# or there is a scheme but it's not in the list of known_url_schemes,
	# then assume it's a cross-reference

In some use cases a configuration option could be useful, such that links with certain extensions (or regexes) are converted to external links, rather than attempting to resolve them as internal links

Originally posted by @chrisjsewell in jupyter-book/jupyter-book#823 (comment)

[jpmckinney]

Same as #215?

[chrisjsewell]

yep I guess so, and more or less #361

[chrisjsewell]

although with #215 it is more than just leaving it as an external link, but turning it into a download link.
the focus though is: more configurable handling of the value of the (link)
seomthing like mapping the extension/regex to whether it is handled as an external link or a certain role, or some other logic

[Griatch]

recommonmark provided a url_resolver feature, where one could provide a custom handler to pre-process whatever is parsed as the url.

def url_resolver(url):
    ...
    return url

This made it very easy to make custom urls be parsed into whatever end result. Including stuff like [mylink](github:issues) being parsed into an external link to the github issue page, or [foobar](api:/foo/bar) becoming the correct full link to the api documentation page.

[chrisjsewell]

recommonmark provided a url_resolver feature, where one could provide a custom handler to pre-process whatever is parsed as the url.

errr, it may be possible to do that here.
But, I maybe fell you are stepping away from what a Markdown link actually is, and there are other ways to achieve this.
For example, you could use a substitution for the link to github issues, like:

myst_substitutions = {
  "gh_issues": "[GitHub Issues](https://github.com/executablebooks/MyST-Parser/issues)"
}

Or you could create a new sphinx role for special links: {api}`foo/bar`

[Griatch]

@chrisjsewell
While one can achieve the outcomes like you suggest, it requires quite a lot of changes to (in my case) vast amounts of already written Markdown documentation using recommonmark. Even though I suppose I could write scripts to try to convert, it's not really an ideal solution:

• Neither {{ gh_issues }} nor {api}`foo/bar` are familiar Markdown for people. And while I as the maintainer can learn MyST syntax for advanced things, I have to explain this to a lot of relatively inexperienced contributors that should ideally be able to contribute something as simple as a link with familiar syntax. Lowering the threshold for contributing is an important thing.
• Being able to write [Issues](github:issues) or [FooBar class](api:foo.bar#FooBar) (where "github:issues" and other urls are just strings I get to parse and modify as I like at build-time) is just a lot easier to work with IMO.

The url_resolver is a very powerful tool and seems to be a much more flexible solution for overriding defaults than trying to provide some way to 'recognize external links' as suggested in this issue. IMO of course. :)

[jpmckinney]

I personally just use Sphinx's extlinks extension, which I assume is essentially what @chrisjsewell is referring to as new Sphinx roles.

Sure, you might prefer the syntax to be [hyperlink text](issue:123) instead of {issue}`hyperlink text<123>`, but the latter works better for non-hyperlink roles (there are a lot in Sphinx), and it requires no extra code.

[Griatch]

@jpmckinney I don't expect [hyperlink text](issue:123) to carry any particular meaning at all out of the box. With url_resolver it would be up to me to parse the string "issue:123" and figure out that it means an url pointing to a particular github issue. What styling/shortcut to use would be up to me.

For Myst to be a replacement for recommonmark, this functionality is pretty critical though.