Surrounding pair problem in error python code

[AndreasArvidsson]

def funk(words:list<str>):
take pair in <str> doesn't work

[auscompgeek]

Isn't that invalid Python? list[str] would be a valid type hint.

[pokey]

Yes, but the idea behind surrounding pairs is that they should work no matter what

[pokey]

The problem here is that tree-sitter is seeing this < as a less-than operator. We have code that tries to discard < as a paired delimiter if it is being used as the less-than operator, so that you don't get trapped by things like if (foo < bar).

We do avoid doing this if the < token is an error node, but in this case tree-sitter actually doesn't treat this < as an error node; it doesn't decide there's an error until it sees the >

The example described in this issue is obviously fairly odd-ball code, but @AndreasArvidsson maybe it would make sense to just take a swing at implementing #673 using a new experimental modifier type. Then it's well-contained and feature flagged, so you can see if you can get something robust working. I do think you'll need to think about the less-than case, but who knows, maybe there's a stack of heuristics that will end up being more predictable than what we have today. As long as it's in its own modifier stage and protected by a feature flag, I'm happy for you to go wild. One of the benefits of the new, more modular approach

You might want to wait for the newer scope type setup, though, so you don't have to worry about ranges. I'm hoping we can get that working in #629