### 馃殌 The feature, motivation and pitch For accuracy reasons, mamba caches need to kept in float32. In vllm users specify cache dtype using `--mamba_ssm_cache_dtype` flag ### Alternatives _No response_ ### Additional context _No response_ ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and checked the [documentation](https://nvidia.github.io/TensorRT-LLM/) and [examples](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples) for answers to frequently asked questions.
馃殌 The feature, motivation and pitch
For accuracy reasons, mamba caches need to kept in float32. In vllm users specify cache dtype using
--mamba_ssm_cache_dtypeflagAlternatives
No response
Additional context
No response
Before submitting a new issue...