Support generic comma-separated lists

[pokey]

High-level

We'd like to support arbitrary list-like syntactic structures which consist of opening delimiters, closing delimiters, and separators. Examples include the following:

• Argument lists
• List literals
• Dictionary literals
• Enclosed import lists
• Template parameters
• Probably lots of other things we haven't thought of, but that we can get for free

We'd like to use the same / similar code-path as surrounding pairs, so that it leverages parse tree if it has it, but works on text if it doesn't, and works generically without needing to manually specify acceptable tree-sitter types.

The idea would be that it would select anything between an opening / closing delimiter and a comma, as long as that comma isn't nested between delimiters not containing the input. For example

foo(bar(baz, bongo), hello)

If we said "take item risk", it should skip over the first comma because it is nested

Algorithm

Note: This algorithm could likely be simplified a lot if we decide to go with #484

• Add a new delimiter side called "separator"
• Potentially also need to have delimiters that create a level of nesting, so we ignore separators between them, but that shouldn't count as a container for a single item. Good example would be ": should ignore commas between the quotes but not return its contents as an item, in contrast to parentheses, where if we found no commas between the parentheses, we'd just yield between them as a single-item list

In generateUnmatchedDelimiters

• we yield separators if and only if we're at top-level, ie sum(Object.values(delimiterBalances)) === 0. Otherwise we just ignore them

In findDelimiterPairContainingSelection

• After rightward pass, we look to see whether the yielded delimiter is a separator or a closing delimiter
• If right delimiter is a closing delimiter, then:
  • Sweep leftwards looking for either the matching left delimiter or any separator. We need the generateUnmatchedDelimiters function to keep track of all delimiter pairs even though it's only looking for the matching left delimiter. This way we properly ignore nested separators. Could do this either by having a trackedDelimiters arg that contains all the delimiters, or by just looping in findDelimiterPairContainingSelection until we get a matching delimiter.
  • If we hit either the opening delimiter we want or a separator, then we check if it is left of selection
    • If left of selection, return it
    • If not,
      • if it is the opening delimiter, we go back to the outer loop
      • if it is a separator, we continue left
• If right delimiter is a separator, then:
  • Sweep leftwards looking for any delimiter or separator,
  • If yielded delimiter is left of selection, return it
  • If not, check whether the yielded delimiter is a separator or an opening delimiter
    • If separator, continue leftward
    • If opening delimiter,
      • sweep rightward looking only for the matching closing delimiter, ignoring separators or other delimiters. I think can just set acceptableDelimiters for that one. Though depending how much we care about mismatched delimiters, it might be safer to leave acceptableDelimiters and just loop checking the yielded delimiter
      • once we hit the matching closing delimiter, we just go back to the outer loop

In findDelimiterPairAdjacentToSelection

• If it is an opening delimiter, we look to the right for its closing delimiter or for a separator, making sure to use trackedDelimiters to ensure that we still properly ignore nested separators
• If it is a closing delimiter, we look to the left for its opening delimiter or for a separator
• If it is a separator, we look to the left for matching separator or any opening delimiter

Returning selection

• We consider any whitespace to the right of the left delimiter, or to the left of the right delimiter, to be part of the delimiter
• We don't support interiorOnly / excludeInterior
• On selectionContext, set trailing / leading delimiter and inDelimitedList in the same way we do for args / items today

Extra credit

Unenclosed lists

Examples:

• CSVs
• unenclosed import lists, like in Python
• |-separated type disjunctions in Typescript

We might be able to inject dummy zero-length opening / closing delimiter pair by leveraging a bit of language-specific context here:

• For the Python unenclosed import lists, inject a dummy delimiter after import and before end of statement
• For Typescript |, any node where child is union_type but parent is not, inject dummy delimiters before and after union_type
• For the CSV example, inject dummy delimiters at start and end of line

Alternately, could just limit scope of search instead of injecting dummy delimiters, but then we'd need to indicate to lower level that it's ok to yield even if it didn't find one or both sides, which feels weird