Skip to content

feat(model_registry): enhance model registration and refresh mechanisms#2106

Merged
luispater merged 1 commit intodevfrom
model
Mar 13, 2026
Merged

feat(model_registry): enhance model registration and refresh mechanisms#2106
luispater merged 1 commit intodevfrom
model

Conversation

@hkfires
Copy link
Collaborator

@hkfires hkfires commented Mar 13, 2026

No description provided.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the model registration and refresh capabilities of the system. It transitions from a static, startup-only model catalog to a dynamic, periodically refreshed one. The core improvement lies in the introduction of a callback mechanism that allows the service to react to changes in model definitions by automatically re-registering clients. This ensures that the system's understanding of available models and client-specific states, like quota cooldowns, is always current and responsive to external updates.

Highlights

  • Periodic Model Refresh: The model catalog now refreshes periodically (every 3 hours) in the background, rather than only once on startup. This ensures that the service stays up-to-date with the latest model definitions.
  • Model Refresh Callback Mechanism: A new callback system (SetModelRefreshCallback) has been introduced to notify the service when model definitions for specific providers change during a refresh. This allows for dynamic updates to client registrations.
  • Dynamic Client Re-registration: Upon detection of changed model definitions for a provider, the service automatically re-registers affected authentication clients. This process intentionally clears transient scheduling states, such as quota cooldowns or suspensions, ensuring clients operate with the most current model availability.
  • Centralized Quota Exceeded Window: The hardcoded 5 * time.Minute duration for model quota expiration has been replaced with a named constant modelQuotaExceededWindow, improving maintainability and clarity.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • internal/registry/model_registry.go
    • Added modelQuotaExceededWindow constant to centralize the quota expiration duration.
    • Included a comment clarifying that re-registering an existing client/model binding resets transient scheduling states.
    • Replaced direct 5 * time.Minute literals with the modelQuotaExceededWindow constant in quota-related logic.
  • internal/registry/model_updater.go
    • Introduced modelsRefreshInterval constant for periodic model updates.
    • Modified StartModelsUpdater to run the updater in a goroutine, enabling background refresh.
    • Implemented periodicRefresh function to handle scheduled model catalog updates.
    • Refactored model fetching into tryStartupRefresh, tryPeriodicRefresh, and a generalized tryRefreshModels.
    • Defined ModelRefreshCallback type and SetModelRefreshCallback function for external notification of model changes.
    • Added detectChangedProviders, modelSectionChanged, notifyModelRefresh, and mergeProviderNames functions to manage and report changes in model definitions.
    • Updated fetchModelsFromRemote to return parsed data and the source URL, and to include catalog validation.
  • sdk/cliproxy/service.go
    • Added registerResolvedModelsForAuth helper function to encapsulate client model registration logic.
    • Integrated registry.SetModelRefreshCallback to listen for model catalog changes and trigger re-registration for affected clients.
    • Implemented refreshModelRegistrationForAuth to re-apply model registrations for an auth, clearing previous transient states.
    • Added latestAuthForModelRegistration to retrieve the most current auth snapshot.
    • Updated existing calls to GlobalModelRegistry().RegisterClient to use the new s.registerResolvedModelsForAuth helper.
Activity
  • The pull request introduces new features related to model registration and refresh mechanisms.
  • The changes are implemented by user hkfires.
  • The pull request description is empty, so all context is derived from the code changes and title.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a periodic model refresh mechanism and enhances model registration. The changes are well-structured, introducing a callback system for model updates and refactoring existing code to be more robust and maintainable. I've found a couple of areas with opportunities for minor refactoring to reduce code duplication and improve efficiency. Overall, this is a solid enhancement to the model registry functionality.

Comment on lines +272 to +295
seen := make(map[string]struct{}, len(existing)+len(incoming))
merged := make([]string, 0, len(existing)+len(incoming))
for _, provider := range existing {
name := strings.ToLower(strings.TrimSpace(provider))
if name == "" {
continue
}
if _, ok := seen[name]; ok {
continue
}
seen[name] = struct{}{}
merged = append(merged, name)
}
for _, provider := range incoming {
name := strings.ToLower(strings.TrimSpace(provider))
if name == "" {
continue
}
if _, ok := seen[name]; ok {
continue
}
seen[name] = struct{}{}
merged = append(merged, name)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The two for loops for processing existing and incoming provider lists are identical. This code can be simplified to reduce duplication and improve maintainability by combining the slices and iterating once.

seen := make(map[string]struct{}, len(existing)+len(incoming))
merged := make([]string, 0, len(existing)+len(incoming))
for _, provider := range append(existing, incoming...) {
	name := strings.ToLower(strings.TrimSpace(provider))
	if name == "" {
		continue
	}
	if _, ok := seen[name]; !ok {
		seen[name] = struct{}{}
		merged = append(merged, name)
	}
}

Comment on lines +571 to +586
for _, item := range auths {
if item == nil || item.ID == "" {
continue
}
auth, ok := s.coreManager.GetByID(item.ID)
if !ok || auth == nil || auth.Disabled {
continue
}
provider := strings.ToLower(strings.TrimSpace(auth.Provider))
if !providerSet[provider] {
continue
}
if s.refreshModelRegistrationForAuth(auth) {
refreshed++
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The call to s.coreManager.GetByID(item.ID) inside the loop is redundant. s.coreManager.List() already returns a slice of *coreauth.Auth objects (clones), so you can use the item from the loop directly instead of fetching it again by ID. This will improve performance by avoiding an unnecessary map lookup and lock acquisition.

for _, auth := range auths {
	if auth == nil || auth.ID == "" || auth.Disabled {
		continue
	}
	provider := strings.ToLower(strings.TrimSpace(auth.Provider))
	if !providerSet[provider] {
		continue
	}
	if s.refreshModelRegistrationForAuth(auth) {
		refreshed++
	}
}

@luispater luispater merged commit 1db2397 into dev Mar 13, 2026
2 checks passed
@luispater luispater deleted the model branch March 13, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants