llama : speed-up grammar sampling

[jakexcosme]

Note: This issue was copied from ggml-org#4218

Original Author: @ggerganov
Original Issue Number: ggml-org#4218
Created: 2023-11-25T17:04:06Z

---

There have been a few reports where the grammar sampling can significantly degrade the performance.
It would be nice to profile and optimize the implementation - there should be room for improvements.

Already on-going efforts:

• reserve space in decode_utf8 ggml-org/llama.cpp#4210
• Allow reusing results from llama_token_to_piece when sampling grammars ggml-org/llama.cpp#4213

Probably worth looking in multi-threading the implementation as well.