feat: add support for pattern matching

[theHamsta]

https://www.python.org/dev/peps/pep-0636/

Fixes #116

[theHamsta]

I need to solve this somehow with patterns (including "or" and "as" patterns)

[theHamsta]

or call this alias_expression?

[theHamsta]

The patterns in a match case are a subset of expressions. I simplified using expression here since I didn't want to replicated all the new patterns that basically follow the same rules as expressions.

[theHamsta]

Test fail for uses of match not as a keyword (django loves match)

match = 4

Match is a soft keyword, the python parser would backtrack when seeing a : and start parsing a match statement
https://www.python.org/dev/peps/pep-0622/#id69

[theHamsta]

A possible hacky resolution would be to have a common parent for assignments that assign to "match" and match_statement

        _maybe_match_statement: $ => seq(
            choice($.match_statement,
                alias(seq(alias("match", $.identifier),
                    choice(
                        seq('=', field('right', $._right_hand_side)),
                        seq(':', field('type', $.type)),
                        seq(':', field('type', $.type), '=', field('right', $._right_hand_side))
                    )), $.assignment))),

I can't make maybe_match_statement hidden due to super node requirement in compound statement. _maybe_match_statement can be a third category next to _compound_statement and _simple_statements.

[tmke8]

Maybe there could also be a test case for a variable called match with a type annotation:

match: int = 3

[theHamsta]

@thomkeh sure, but right now it doesn't parse the simple case correctly, also for match, other = 1, 2

[GBeauregard]

if you're looking for weird match test cases you can go through the ones in CPython: https://github.com/python/cpython/blob/main/Lib/test/test_patma.py

There's plenty of weird ones https://github.com/python/cpython/blob/eb483c46d62707bdf705491f76cf1fa9642fb47e/Lib/test/test_patma.py#L1205

[theHamsta]

Thanks @GBeauregard I hope I fixed a few edge cases in test_patma.py . The file now parses successfully.

[GBeauregard]

Test fail for uses of match not as a keyword (django loves match)

match = 4

Match is a soft keyword, the python parser would backtrack when seeing a : and start parsing a match statement https://www.python.org/dev/peps/pep-0622/#id69

I think you figured it out (and I don't think it mattered for this comment anyway), but to clarify the PEP 622 you linked here is not the accepted pattern matching specification (note the status is superseded); the final/accepted specification differs in a few aspects and was put in PEP 634: https://www.python.org/dev/peps/pep-0634/

Thanks @GBeauregard I hope I fixed a few edge cases in test_patma.py . The file now parses successfully.

no problem, thanks for implementing parsing for the fun new feature :)

[GBeauregard]

One more comment, this and the nvim-treesitter PR seem to be missing the new _ soft keyword. See https://docs.python.org/3.10/reference/lexical_analysis.html#soft-keywords for reference on it being considered a keyword and python/cpython#25851 where the CPython built-in IDLE highlighting considers and highlights this as a soft keyword.

See also https://docs.python.org/3.10/reference/compound_stmts.html#wildcard-patterns

This can be useful for users as well, since the syntax highlighting difference will hint that _ has a different behavior from a capture variable named _ (can be used multiple times, etc etc). (To the extent this isn't obvious is all the more reason to highlight it!)

[theHamsta]

This PR just regards all of the patterns introduced by pattern matching as expressions. To treat _ special only in patterns one would have to implement all the introduced patterns. Not sure whether that's worth it since _ is only semantically different from an identifier (it can bind to multiple values). For highlighting, one could just always highlight _ as wildcards.

[bellini666]

Hey guys. Just wondering when this will get merged :)

[dcreager]

This looks great, @theHamsta, thanks for taking this on!

You'll need to rebase onto the latest master since I just merged a couple of other PRs. Also, can you add match and case as new keywords in the syntax highlighting file? And ideally some highlighting tests to test/highlight/keywords.py. Then I think this is good to merge!

[theHamsta]

@dcreager rebased and added a highlight test