Our monkey-patched demollm runtime experiences some inconsistencies in token generation compared to the full runtime. This is from a qualitative assessment of the example prompts+responses on our dashboard.
Let's dig a little deeper to understand if there is a systemic error in our demo/debug runtime. It might also just be due to the very simple sampling logic we have implemented
Our monkey-patched demollm runtime experiences some inconsistencies in token generation compared to the full runtime. This is from a qualitative assessment of the example prompts+responses on our dashboard.
Let's dig a little deeper to understand if there is a systemic error in our demo/debug runtime. It might also just be due to the very simple sampling logic we have implemented