feat: add embedded adapters (granite switch) to openai backend

[jakelorocco]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes N/A

Adds support for granite switch models for OpenAIBackends.

Changes:

• OpenAIBackends can utilize EmbeddedIntrinsicAdapters with any of the regular intrinsic functionality.
• Adapters are loaded by default when load_embedded_adapters=True at init.
• Added a placeholder IBM_GRANITE_SWITCH_4_1_8B that points to the temp repo for now.
• Added documentation, examples, tests.
• When generating from an intrinsic, we now only grab temperature and seed from model options. This is also correctly noted.
• call_intrinsic function now works with both OpenAIBackends and LocalHFBackends

Please see most recent comment below for additional context: #881 (comment)

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[jakelorocco]

I'm still in the process of testing some of these changes from an e2e perspective. I also still need to add documentation for how to use embedded models along with examples.

These changes do seem to work though and allow utilizing intrinsics with an embedded adapter / granite switch model.

[lastras]

Bug: model_options temperature is silently dropped for intrinsic calls

The seed parameter is correctly forwarded to api_params at line 639, but temperature (and any other model_options entries) are not. They are written into request_json at line 617, which gets absorbed into the ChatCompletion object by rewriter.transform(), but never extracted back out when building api_params. The result is that callers passing model_options={ModelOption.TEMPERATURE: 0.0} get the model's default temperature instead — causing non-deterministic outputs even when greedy decoding is explicitly requested.

Fix: mirror the seed pattern immediately after line 640:

if ModelOption.TEMPERATURE in model_options:
    api_params["temperature"] = model_options[ModelOption.TEMPERATURE]

Verified: with this fix, 5 back-to-back pipeline runs (guardian → rewrite → answerability → QC → base model, no sleep between calls) produce bit-for-bit identical outputs including long-form base model answers. Without the fix, outputs vary run-to-run despite temperature=0.0 being passed by the caller. The warning at line 557 ("some model options may be overwritten / ignored") is technically accurate but understated — temperature is always silently dropped, not just sometimes.

[nrfulton]

@lastras Your comment states that all model options get dropped but that the fix is to add (another) special case for temperature in model_options.

It sounds from your comment like there's an underlying bug here that should be fixed. Can you clarify?

[jakelorocco]

@lastras @nrfulton, I am going to move this convo to slack for faster resolution.

[jakelorocco]

I fixed a few random issues and also the issues we talked about (call_intrinsic and seed/temperature setting). I'm working on adding some documentation and minor examples.

The functional parts of the code should be stable now. I have additional things that I'd like to clean up, but I will delay them until after we make sure everything is working as intended and post-release, so I don't impact any functionality.

[jakelorocco]

Updated the PR description. Fixed a few additional errors / bugs. Added documentation and examples. We should be good to start reviewing. I need to run one more round of tests with the latest from the granite-switch team.

I have plans to open a few additional issues to clean things up after this PR is merged.

Additional changes that should be made before the official release:

• Update IBM_GRANITE_SWITCH_4_1_3B to point to the correct model and update all references to the model
• Update our documentation to point to the official granite switch documentation when published

[psschwei]

Docs pointer

Docs should not be a blocker for merging this btw, just something we should have in advance of the release.

[planetf1]

LGTM - I know there's some placeholders here such as for the repo -- but I wonder if the doc update should be deferred until later, since this will get published on our next minor release (Wed) and would have a dead line (openai.md and glossary.md) ?

[jakelorocco]

We can defer doc updates. There was concern about the model_identifier being left in with the placeholder value. I'm open to merging with both the docs and placeholder model (Grizlee-Ber/...) in the code, but was hoping we could fix those before merging.

[ajbozarth]

I'm open to merging with both the docs and placeholder model (Grizlee-Ber/...) in the code, but was hoping we could fix those before merging.

I agree that waiting for the fixes was my preference, but if leaving this unmerged is causing code drift we may want to merge now then fix those before the next release

[lastras]

We have a name already. We can talk in slack. I believe the name will be stable, and all will be ok.

…

On Mon, Apr 27, 2026 at 2:38 PM Alex Bozarth ***@***.***> wrote: *ajbozarth* left a comment (generative-computing/mellea#881) <#881 (comment)> I'm open to merging with both the docs and placeholder model (Grizlee-Ber/...) in the code, but was hoping we could fix those before merging. I agree that waiting for the fixes was my preference, but if leaving this unmerged is causing code drift we may want to merge now then fix those before the next release — Reply to this email directly, view it on GitHub <#881 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYZ5XAQFFK5A4GPLFDAOVL4X6SLLAVCNFSM6AAAAACX5ODHBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DGMRZGU2DONJQGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jakelorocco]

@planetf1 @ajbozarth @avinash2692, can I please get a review from one (or more) of y'all? We are going to merge as is (assuming you don't spot any issues with the code). Then, pre-release, we will update the model id and doc links.

[ajbozarth]

Two non-blocking nits below, otherwise LGTM.

[ajbozarth]

import os is never used inside this function (os is already imported at module level on line 7). Ruff will flag this as F401 — safe to remove.

[jakelorocco]

Will address in a follow up PR. Thank you.

[ajbozarth]

This fires even when the caller passes only recognized options like {ModelOption.TEMPERATURE: 0.0}. Consider narrowing to unrecognized options only so it's not a false alarm:\npython\nrecognized = {ModelOption.TEMPERATURE, ModelOption.SEED}\nif any(k not in recognized for k in model_options):\n MelleaLogger.get_logger().info(...)\n

[jakelorocco]

Will address in a follow up PR. Thank you.