Contexts are typically used via a MelleaSession. In that setting, contexts are "bound" to a specific model_id. When we know the model_id, we can lookup the max context length and then use this context length to determine when the Context's sliding window needs to move. We should implement a DynamicLinearContext that provides this behavior, or maybe even just change the LinearContext to do this dynamic window management based upon models' max context length.
Contexts are typically used via a
MelleaSession. In that setting, contexts are "bound" to a specificmodel_id. When we know themodel_id, we can lookup the max context length and then use this context length to determine when the Context's sliding window needs to move. We should implement aDynamicLinearContextthat provides this behavior, or maybe even just change the LinearContext to do this dynamic window management based upon models' max context length.