The tutorial "jumps right in" too quickly.
We should start with a simple one-function example. We should demonstrate how you can swap out the model and how to specify model_options. Sketch below.
How it Works
@generative
def my_function(...) -> ...:
""" .... """
results in a GenerativeSlot object, which has the following call signature (all the uppercase letters are generic types):
def __call__(
self, m, model_options: dict | None = None, *args: P.args, **kwargs: P.kwargs
) -> R:
[[TODO: point to code repo]]
where m: MelleaSession. Sessions hold a Backend which has a model_id. You can pass whatever session you want into a generative slot. So, via the first parameter to any generative slot, you are able to specify the LLM.
The model_options dict allows you to additionally control params.
We should start more gadually in the tutorial, pointing out these options.
The tutorial "jumps right in" too quickly.
We should start with a simple one-function example. We should demonstrate how you can swap out the model and how to specify model_options. Sketch below.
How it Works
results in a GenerativeSlot object, which has the following call signature (all the uppercase letters are generic types):
[[TODO: point to code repo]]
where
m: MelleaSession. Sessions hold a Backend which has a model_id. You can pass whatever session you want into a generative slot. So, via the first parameter to any generative slot, you are able to specify the LLM.The model_options dict allows you to additionally control params.
We should start more gadually in the tutorial, pointing out these options.