From 55ac5473f56e3b71138bf4c89f8c8ec4bdc32a3d Mon Sep 17 00:00:00 2001 From: Alon Freund Date: Sun, 24 May 2026 14:12:03 +0300 Subject: [PATCH] Add T4 GPU prerequisite to hello_adapter and granite_switch_with_hf notebooks - Updated hello_adapter.ipynb to specify T4 or better GPU requirement - Updated granite_switch_with_hf.ipynb to specify T4 or better GPU requirement - Consistent with hello_mellea.ipynb which was previously tested on T4 - Makes tutorials more accessible by clarifying minimum GPU requirements --- .../notebooks/granite_switch_with_hf.ipynb | 26 ++++++++++++++++++- tutorials/notebooks/hello_adapter.ipynb | 26 ++++++++++++++++++- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/tutorials/notebooks/granite_switch_with_hf.ipynb b/tutorials/notebooks/granite_switch_with_hf.ipynb index c8b68d5..6bcd3ca 100644 --- a/tutorials/notebooks/granite_switch_with_hf.ipynb +++ b/tutorials/notebooks/granite_switch_with_hf.ipynb @@ -3,7 +3,31 @@ { "metadata": {}, "cell_type": "markdown", - "source": "# Granite Switch with HuggingFace\n\n**Duration:** ~10 min (after model download)\n\nA Granite Switch checkpoint bundles a base model with many LoRA experts. You pick one per forward pass by passing its name to the chat template.\n\n*Why HuggingFace:* this notebook uses the `transformers` backend for familiarity - every call is a standard `model.generate()`. Production workloads should switch to vLLM for 10-20x speedup; see [`rag_101.ipynb`](./rag_101.ipynb).\n\n**What you'll build:** one growing conversation about *Horizon 2055 Target Date Fund* (a fictional fund whose prospectus is the retrieved context), where each natural turn demonstrates a different embedded adapter function.\n\n**What you'll learn:**\n- How to load a composed Granite Switch checkpoint via `AutoModelForCausalLM` - no `trust_remote_code=True`.\n- How to invoke any embedded adapter function with `tokenizer.apply_chat_template(..., adapter_name=...)`.\n- The two parts of every adapter call: the LoRA switch, and the adapter-specific content protocol (criteria strings, control tokens, tagged sentences).\n- How guardian-family adapter functions act as *judges* over a side conversation without polluting the main chat history.\n\n**Adapters used:** adapters from the [Core](https://huggingface.co/ibm-granite/granitelib-core-r1.0) library (`context-attribution`, `uncertainty`, `requirement-check`) and the [Guardian](https://huggingface.co/ibm-granite/granitelib-guardian-r1.0) library (`guardian-core`, `policy-guardrails`, `factuality-detection`, `factuality-correction`).\n\n## Prerequisites\n\n1. **Install dependencies** (GPU recommended; CPU works but slow):", + "source": [ + "# Granite Switch with HuggingFace\n", + "\n", + "**Duration:** ~10 min (after model download)\n", + "\n", + "A Granite Switch checkpoint bundles a base model with many LoRA experts. You pick one per forward pass by passing its name to the chat template.\n", + "\n", + "*Why HuggingFace:* this notebook uses the `transformers` backend for familiarity - every call is a standard `model.generate()`. Production workloads should switch to vLLM for 10-20x speedup; see [`rag_101.ipynb`](./rag_101.ipynb).\n", + "\n", + "**What you'll build:** one growing conversation about *Horizon 2055 Target Date Fund* (a fictional fund whose prospectus is the retrieved context), where each natural turn demonstrates a different embedded adapter function.\n", + "\n", + "**What you'll learn:**\n", + "- How to load a composed Granite Switch checkpoint via `AutoModelForCausalLM` - no `trust_remote_code=True`.\n", + "- How to invoke any embedded adapter function with `tokenizer.apply_chat_template(..., adapter_name=...)`.\n", + "- The two parts of every adapter call: the LoRA switch, and the adapter-specific content protocol (criteria strings, control tokens, tagged sentences).\n", + "- How guardian-family adapter functions act as *judges* over a side conversation without polluting the main chat history.\n", + "\n", + "**Adapters used:** adapters from the [Core](https://huggingface.co/ibm-granite/granitelib-core-r1.0) library (`context-attribution`, `uncertainty`, `requirement-check`) and the [Guardian](https://huggingface.co/ibm-granite/granitelib-guardian-r1.0) library (`guardian-core`, `policy-guardrails`, `factuality-detection`, `factuality-correction`).\n", + "\n", + "## Prerequisites\n", + "\n", + "**GPU runtime** (T4 or better). Go to *Runtime -> Change runtime type -> T4 GPU*.\n", + "\n", + "1. **Install dependencies:**" + ], "id": "d5ed1e5ac8582c60" }, { diff --git a/tutorials/notebooks/hello_adapter.ipynb b/tutorials/notebooks/hello_adapter.ipynb index d733067..c039261 100644 --- a/tutorials/notebooks/hello_adapter.ipynb +++ b/tutorials/notebooks/hello_adapter.ipynb @@ -3,7 +3,31 @@ { "metadata": {}, "cell_type": "markdown", - "source": "# Hello Adapter - Granite Switch with HuggingFace\n\n**Duration:** ~5 min\n\nMinimal example of invoking an **embedded LoRA adapter** inside a **Granite Switch** model, using the HuggingFace backend. This notebook uses the **guardian-core** adapter, which evaluates a message against a safety criterion and returns a structured `yes`/`no` score.\n\n**What you'll learn:**\n- How to build a single guardian-core call that scores a user message against a safety criterion and prints a parsed `harmful`/`safe` verdict.\n- How to load a composed Granite Switch checkpoint with `transformers`.\n- How to activate an adapter function function by passing `adapter_name=...` to `apply_chat_template`.\n- The Guardian prompt protocol - how to frame a criterion so the adapter returns a parseable score.\n\n**Adapters used:** `guardian-core` from the [Guardian](https://huggingface.co/ibm-granite/granitelib-guardian-r1.0) library - a general-purpose safety/risk judge that scores any user-supplied criterion (harm, social bias, jailbreaking, groundedness, ...) as `yes`/`no`.\n\nFor the recommended inference path (mellea + vLLM), see [`hello_mellea.ipynb`](./hello_mellea.ipynb). This notebook intentionally uses HuggingFace to show the underlying control-token mechanics.\n\n## Prerequisites\n\n**1 * A composed Granite Switch checkpoint** with the `guardian-core` adapter function. The default `MODEL_PATH` below points at the pre-composed `ibm-granite/granite-switch-4.1-3b-preview` on HuggingFace (drawn from the [IBM Granite 4.1 collection](https://huggingface.co/collections/ibm-granite/granite-41-language-models)). To compose your own checkpoint instead, see [`./compose_granite_switch.ipynb`](./compose_granite_switch.ipynb) and point `MODEL_PATH` at its output directory.\n\n**2 * Dependencies** (CUDA GPU required):", + "source": [ + "# Hello Adapter - Granite Switch with HuggingFace\n", + "\n", + "**Duration:** ~5 min\n", + "\n", + "Minimal example of invoking an **embedded LoRA adapter** inside a **Granite Switch** model, using the HuggingFace backend. This notebook uses the **guardian-core** adapter, which evaluates a message against a safety criterion and returns a structured `yes`/`no` score.\n", + "\n", + "**What you'll learn:**\n", + "- How to build a single guardian-core call that scores a user message against a safety criterion and prints a parsed `harmful`/`safe` verdict.\n", + "- How to load a composed Granite Switch checkpoint with `transformers`.\n", + "- How to activate an adapter function function by passing `adapter_name=...` to `apply_chat_template`.\n", + "- The Guardian prompt protocol - how to frame a criterion so the adapter returns a parseable score.\n", + "\n", + "**Adapters used:** `guardian-core` from the [Guardian](https://huggingface.co/ibm-granite/granitelib-guardian-r1.0) library - a general-purpose safety/risk judge that scores any user-supplied criterion (harm, social bias, jailbreaking, groundedness, ...) as `yes`/`no`.\n", + "\n", + "For the recommended inference path (mellea + vLLM), see [`hello_mellea.ipynb`](./hello_mellea.ipynb). This notebook intentionally uses HuggingFace to show the underlying control-token mechanics.\n", + "\n", + "## Prerequisites\n", + "\n", + "**1 * A composed Granite Switch checkpoint** with the `guardian-core` adapter function. The default `MODEL_PATH` below points at the pre-composed `ibm-granite/granite-switch-4.1-3b-preview` on HuggingFace (drawn from the [IBM Granite 4.1 collection](https://huggingface.co/collections/ibm-granite/granite-41-language-models)). To compose your own checkpoint instead, see [`./compose_granite_switch.ipynb`](./compose_granite_switch.ipynb) and point `MODEL_PATH` at its output directory.\n", + "\n", + "**2 * GPU runtime** (T4 or better). Go to *Runtime -> Change runtime type -> T4 GPU*.\n", + "\n", + "**3 * Dependencies:**" + ], "id": "97c76dcca207b140" }, {