Improve parsing of responses from the AI

[PankajBhojwani]

Summary of the Pull Request

Instead of manually parsing out code blocks from the response we receive, use the cmark library.

Validation Steps Performed

Responses are parsed as expected.

PR Checklist

• Closes #xxx
• Tests added/passed
• Documentation updated
  • If checked, please file a pull request on our docs repo and link it here: #xxx
• Schema updated (if necessary)

[PankajBhojwani]

I initially wanted to directly use the RichTextBlock we get from MarkdownToXaml::Convert added in #17585, but because we want to allow the user to click individual code blocks to send it to the shell we cannot just present one RichTextBlock - the palette needs to know which items are code and which items are plaintext and present them individually. The code here uses the same logic though.

[PankajBhojwani]

Don't like that this is the exact same as lines 227-231 but making a helper function for 3 lines of code seems silly...

[DHowett]

the palette needs to know which items are code and which items are plaintext and present them individually

does it? could we just put all the output in a single speech bubble? i'd rather avoid reimplementing the cmark->xaml parser that mike just wrote if at all possible.

[zadjii-msft]

mine had a play button, chat needs a copy. that might make the reuse a little trickier

[PankajBhojwani]

could we just put all the output in a single speech bubble?

I considered this but there are enough times where the AI sends a large response with multiple code blocks in it that it felt like the palette needs to do its own parsing. For example:

If we didn't do our own parsing, there'd be one big bubble with 2 code blocks in it and we wouldn't know which one to use when the user clicks the bubble to send the output to the shell

[lhecker]

mine had a play button, chat needs a copy. that might make the reuse a little trickier

Why not both? Having a copy button in your markdown pane seems useful as well. It would also mirror GitHub which also has a copy button on each code block.

[PankajBhojwani]

Closing this in favour of #18220