From 3b0dc8d8f166869cd3fd129ecf4655f31fbf8fe1 Mon Sep 17 00:00:00 2001 From: Jake LoRocco Date: Wed, 20 May 2026 08:52:23 -0400 Subject: [PATCH 1/2] docs: add tutorial for mellea byoa Signed-off-by: Jake LoRocco --- tutorials/README.md | 2 + .../guides/configure_your_own_adapter.md | 3 - .../guides/mellea_bring_your_own_adapter.md | 84 +++++++++++++++++++ .../guides/mellea_with_granite_switch.md | 3 +- 4 files changed, 87 insertions(+), 5 deletions(-) delete mode 100644 tutorials/guides/configure_your_own_adapter.md create mode 100644 tutorials/guides/mellea_bring_your_own_adapter.md diff --git a/tutorials/README.md b/tutorials/README.md index 2a23da5..5cd224f 100644 --- a/tutorials/README.md +++ b/tutorials/README.md @@ -22,8 +22,10 @@ Step-by-step walkthroughs covering adapter invocation, pipeline construction, an |-------|-------------| | [Using Mellea with Granite Switch](guides/mellea_with_granite_switch.md) | Connect Mellea to a Granite Switch model | | [Bring Your Own Adapter](guides/bring_your_own_adapter.md) | Train, compose, and use custom adapters | +| [Bring Your Own Adapter with Mellea](guides/mellea_bring_your_own_adapter.md) | Connect Mellea to a Granite Switch model with custom adapters | | [Compare Inference Throughput](guides/compare_inference_throughput.md) | Compare LoRA vs aLoRA based models in an inference race setup | + ## Learning Paths ### Composing Models diff --git a/tutorials/guides/configure_your_own_adapter.md b/tutorials/guides/configure_your_own_adapter.md deleted file mode 100644 index 388a3e1..0000000 --- a/tutorials/guides/configure_your_own_adapter.md +++ /dev/null @@ -1,3 +0,0 @@ -# Configure Your Own Adapter - -This guide explains how to configure your own adapter with Mella to be used by Granite Switch model. diff --git a/tutorials/guides/mellea_bring_your_own_adapter.md b/tutorials/guides/mellea_bring_your_own_adapter.md new file mode 100644 index 0000000..9fb83ee --- /dev/null +++ b/tutorials/guides/mellea_bring_your_own_adapter.md @@ -0,0 +1,84 @@ +# Bring Your Own Adapter with Mellea + +This guide explains how to configure your own adapter with Mellea to be used by Granite Switch model. + +## Overview + +Together, Mellea + Granite Switch + vLLM provide a production-ready inference stack for adapter-based AI applications that can utilize custom adapters. +- See [Mellea With Granite Switch](mellea_with_granite_switch.md) for a detailed explanation of how granite-switch and Mellea work together. +- See [Bring Your Own Adapter](bring_your_own_adapter.md) for info on how to train your own adapter. +- See Mellea's [Lora and aLoRA adapters](https://docs.mellea.ai/advanced/lora-and-alora-adapters) for info on how to train your own custom adapters using Mellea. + +## Prerequisites + +1. **A composed Granite Switch model** with the adapters you need +2. **A running vLLM server** serving that model + +## Installation + +```bash +pip install mellea +``` + +## Quick Example + +By default, calling custom adapters requires utilizing lower level interfaces in Mellea: + +```python +import json + +from mellea.backends.model_options import ModelOption +from mellea.backends.openai import OpenAIBackend +from mellea.stdlib.context import ChatContext +from mellea.stdlib.components import Message, Intrinsic +import mellea.stdlib.functional as mfuncs + +# 1. Initialize the Mellea Backend +backend = OpenAIBackend( + model_id="path/to/your/granite-switch-model", # Local files or Huggingface model id + base_url="http://localhost:8000/v1", # vLLM server + api_key="unused", # vLLM doesn't require auth by default + load_embedded_adapters=True +) + +# By default, load_embedded_adapters will autoload the adapters for the provided switch model. +# If you need to explicitly load adapters from another location (that are supported by the running vLLM +# server / model), you can use `backend.register_embedded_adapter_model` as shown in the "Mellea With Granite Switch" +# example. + +# 2. Use Your Custom Adapter +# Your custom adapter likely requires certain inputs. Here, the example assumes a simple +# chat / conversation is enough. +context = ChatContext().add(Message("assistant", "Hello there, how can I help you?")) +action = Intrinsic("") + +out, _ = mfuncs.act( + action, + context, + backend, + model_options={ModelOption.TEMPERATURE: 0.0}, + strategy=None, +) + +# Adapter / Intrinsic processing in Mellea utilizes the io.yaml format forcing the output +# to be a json. See the "Bring Your Own Adapter" linked example above. +result = json.loads(str(out)) +print(result) +``` + +If you want to define helper functions so that your adapters operate similarly to the high-level +intrinsic wrapper functions in Mellea, you can do the following: + +```python +# Continuing with imports / code from above. +from typing import Any + +from mellea.backends.adapters import AdapterMixin +from mellea.stdlib.components.intrinsic._util import call_intrinsic +# Importing from a private API is fragile. You can imitate the same functionality below without it as well. + +# Replace the return type with the actual type. +def your_custom_functionality(context: ChatContext, backend: AdapterMixin) -> Any: + result_json = call_intrinsic("", context, backend) + return result_json[""] +``` diff --git a/tutorials/guides/mellea_with_granite_switch.md b/tutorials/guides/mellea_with_granite_switch.md index 9beeb37..cba5888 100644 --- a/tutorials/guides/mellea_with_granite_switch.md +++ b/tutorials/guides/mellea_with_granite_switch.md @@ -38,8 +38,7 @@ See [PREREQUISITES.md](../PREREQUISITES.md) for detailed setup instructions. ## Installation ```bash -# Install Mellea from source -pip install "git+https://github.com/generative-computing/mellea.git@main" +pip install mellea ``` ## Quick Example From 6ce7d43e7ec89f0b68ca7ddf694d9005f523a42d Mon Sep 17 00:00:00 2001 From: antonp Date: Wed, 20 May 2026 16:48:55 +0300 Subject: [PATCH 2/2] Place mellea adapter guide instead of configure you own adapter --- tutorials/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tutorials/README.md b/tutorials/README.md index 5cd224f..1dce494 100644 --- a/tutorials/README.md +++ b/tutorials/README.md @@ -22,7 +22,6 @@ Step-by-step walkthroughs covering adapter invocation, pipeline construction, an |-------|-------------| | [Using Mellea with Granite Switch](guides/mellea_with_granite_switch.md) | Connect Mellea to a Granite Switch model | | [Bring Your Own Adapter](guides/bring_your_own_adapter.md) | Train, compose, and use custom adapters | -| [Bring Your Own Adapter with Mellea](guides/mellea_bring_your_own_adapter.md) | Connect Mellea to a Granite Switch model with custom adapters | | [Compare Inference Throughput](guides/compare_inference_throughput.md) | Compare LoRA vs aLoRA based models in an inference race setup | @@ -62,7 +61,7 @@ Best for: Seeing how adapters compose into multi-step applications Best for: Custom adapter development 1. [Bring Your Own Adapter Guide](guides/bring_your_own_adapter.md) -2. [Configure Your Own Adapter Guide](guides/configure_your_own_adapter.md) +2. [Configure Your Own Adapter Guide](guides/mellea_bring_your_own_adapter.md) 3. [Compose Your Checkpoint](notebooks/compose_granite_switch.ipynb)