Skip to content

[dead-code] chore: remove obsolete modelsdev catalog pricing API #48739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Copilot wants to merge 2 commits into from
+7 −461
Closed

[dead-code] chore: remove obsolete modelsdev catalog pricing API #48739

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7fca7e5
Initial plan
Copilot Jul 28, 2026
e50b840
chore(modelsdev): remove dead catalog pricing API
Copilot Jul 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 6 additions & 20 deletions pkg/modelsdev/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,17 @@
# modelsdev Package

The `modelsdev` package provides model pricing lookup backed by the public `models.dev` catalog.
The `modelsdev` package provides provider and model identifier normalization helpers.
Comment thread
pelikhan marked this conversation as resolved.

## Overview

This package downloads and parses `https://models.dev/catalog.json`, normalizes provider/model identifiers, and exposes per-token pricing for callers that need cost-aware behavior.

The catalog is loaded once per process via a singleton cache (`syncutil.OnceLoader`) to avoid repeated network fetches.
This package normalizes provider/model identifiers so callers can perform consistent model matching.

## Public API

### Functions

| Function | Signature | Description |
|----------|-----------|-------------|
| `FindPricing` | `func(ctx context.Context, provider, model string) (map[string]float64, bool)` | Returns normalized per-token pricing for a provider/model pair. Falls back to cross-provider matching when provider lookup fails. Returns `(nil, false)` when no pricing is available |
| `NormalizeProvider` | `func(provider string) string` | Normalizes provider aliases such as `github`, `copilot`, and `github_models` to `github-copilot`, and lower-cases other provider identifiers |
| `NormalizeComparableModelID` | `func(value string) string` | Lower-cases a model identifier, trims surrounding whitespace, and replaces `.` and `_` with `-` so equivalent model IDs compare consistently |

Expand All @@ -23,31 +20,20 @@ The catalog is loaded once per process via a singleton cache (`syncutil.OnceLoad
```go
import "github.com/github/gh-aw/pkg/modelsdev"

pricing, ok := modelsdev.FindPricing(ctx, "github", "gpt-4.1")
if !ok {
// pricing unavailable
return
}

inputUSD := pricing["input"] // per token
outputUSD := pricing["output"] // per token
_ = inputUSD
_ = outputUSD
provider := modelsdev.NormalizeProvider(" github_models ")
comparableModel := modelsdev.NormalizeComparableModelID(" GPT_4.1-mini ")
_, _ = provider, comparableModel
```

## Dependencies

**Internal**:
- `github.com/github/gh-aw/pkg/logger` — debug logging
- `github.com/github/gh-aw/pkg/syncutil` — one-time catalog load/cache primitive
- None

## Design Notes

- Provider aliases such as `github`, `copilot`, and `github_models` are normalized to `github-copilot`.
- Comparable model matching normalizes separators (`.` and `_` to `-`) to improve lookup robustness.
- Numeric catalog costs are interpreted as per-million-token values and converted to per-token units.
- String catalog costs are treated as already normalized per-token values.
- Network or parsing failures degrade gracefully to an empty cache so callers can continue without pricing data.

## Source Synchronization

Expand Down
213 changes: 1 addition & 212 deletions pkg/modelsdev/catalog.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,222 +1,11 @@
package modelsdev

import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"

"github.com/github/gh-aw/pkg/logger"
"github.com/github/gh-aw/pkg/syncutil"
)

const (
fetchTimeout = 5 * time.Second
maxBodyBytes = 4 * 1024 * 1024 // 4 MiB safety cap
)

// catalogURL is a variable so tests can override it with a local HTTP server.
var catalogURL = "https://models.dev/catalog.json"
import "strings"

// modelIDReplacer normalizes separator characters in model IDs so that IDs
// differing only in ".", "_", or "-" compare equal.
var modelIDReplacer = strings.NewReplacer(".", "-", "_", "-")

var pkgLog = logger.New("modelsdev:catalog")

// rawCatalog mirrors the top-level models.dev catalog JSON structure.
type rawCatalog struct {
Providers map[string]rawProvider `json:"providers"`
}

type rawProvider struct {
Models map[string]rawModel `json:"models"`
}

type rawModel struct {
// Cost values are per-million-token numbers (or pre-normalized strings) in the catalog.
Cost map[string]json.RawMessage `json:"cost"`
}

// pricingCache maps normalizedProvider → normalizedModel → per-token pricing.
type pricingCache = map[string]map[string]map[string]float64

var (
catalogCache syncutil.OnceLoader[pricingCache]

// httpClientFactory is overridable for tests.
httpClientFactory = func() *http.Client {
return &http.Client{Timeout: fetchTimeout}
}
)

// FindPricing looks up per-token pricing for the given provider/model from the downloaded
// models.dev catalog. Returns (nil, false) when the catalog is unavailable or the model
// is not found.
func FindPricing(ctx context.Context, provider, model string) (map[string]float64, bool) {
catalog := ensureCatalog(ctx)
if len(catalog) == 0 {
return nil, false
}

normalizedProvider := NormalizeProvider(provider)
trimmedModel := strings.TrimSpace(model)
if trimmedModel == "" {
return nil, false
}
normalizedModel := strings.ToLower(trimmedModel)
comparableModel := NormalizeComparableModelID(normalizedModel)

pkgLog.Printf("FindPricing: looking up provider=%q model=%q", normalizedProvider, normalizedModel)

// Provider-scoped exact match.
if normalizedProvider != "" {
if providerModels, ok := catalog[normalizedProvider]; ok {
if pricing, ok := providerModels[normalizedModel]; ok {
pkgLog.Printf("FindPricing: provider-scoped exact match for %q/%q", normalizedProvider, normalizedModel)
return pricing, true
}
// Comparable (dot/underscore-normalized) model ID match.
for mn, pricing := range providerModels {
if NormalizeComparableModelID(mn) == comparableModel {
pkgLog.Printf("FindPricing: provider-scoped comparable match %q for %q", mn, normalizedModel)
return pricing, true
}
}
}
}

// Cross-provider fallback (when provider is unknown or empty).
for _, providerModels := range catalog {
if pricing, ok := providerModels[normalizedModel]; ok {
pkgLog.Printf("FindPricing: cross-provider fallback match for model %q", normalizedModel)
return pricing, true
}
for mn, pricing := range providerModels {
if NormalizeComparableModelID(mn) == comparableModel {
pkgLog.Printf("FindPricing: cross-provider comparable match %q for %q", mn, normalizedModel)
return pricing, true
}
}
}

pkgLog.Printf("FindPricing: no pricing found for provider=%q model=%q", normalizedProvider, normalizedModel)
return nil, false
}

// ensureCatalog downloads and normalizes the models.dev pricing catalog at most once per
// process. Network failures are logged and result in an empty (non-nil) cache so
// subsequent calls are instant no-ops.
func ensureCatalog(ctx context.Context) pricingCache {
downloaded, _ := catalogCache.Get(func() (pricingCache, error) {
downloaded, err := downloadAndParseCatalog(ctx)
if err != nil {
pkgLog.Printf("models.dev catalog download failed (pricing fallback unavailable): %v", err)
return pricingCache{}, nil
} else {
total := 0
for _, models := range downloaded {
total += len(models)
}
pkgLog.Printf("Downloaded models.dev catalog: %d providers, %d total models", len(downloaded), total)
}
return downloaded, nil
})
return downloaded
}

func downloadAndParseCatalog(ctx context.Context) (pricingCache, error) {
reqCtx, cancel := context.WithTimeout(ctx, fetchTimeout)
defer cancel()

req, err := http.NewRequestWithContext(reqCtx, http.MethodGet, catalogURL, nil)
if err != nil {
return nil, fmt.Errorf("creating request: %w", err)
}

resp, err := httpClientFactory().Do(req)
if err != nil {
return nil, fmt.Errorf("GET %s: %w", catalogURL, err)
}
defer resp.Body.Close()

if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("unexpected HTTP %d from %s", resp.StatusCode, catalogURL)
}

body, err := io.ReadAll(io.LimitReader(resp.Body, maxBodyBytes))
if err != nil {
return nil, fmt.Errorf("reading response: %w", err)
}

return parseCatalog(body)
}

// parseCatalog parses the raw models.dev catalog JSON and normalizes pricing to per-token
// float64 values. Numeric catalog values are in USD per-million tokens and are divided by
// 1,000,000; string values are treated as already per-token.
func parseCatalog(data []byte) (pricingCache, error) {
var raw rawCatalog
if err := json.Unmarshal(data, &raw); err != nil {
return nil, fmt.Errorf("parsing models.dev catalog JSON: %w", err)
}

parsed := make(pricingCache)
for providerName, provider := range raw.Providers {
normalizedProvider := NormalizeProvider(providerName)
if normalizedProvider == "" {
continue
}
if parsed[normalizedProvider] == nil {
parsed[normalizedProvider] = make(map[string]map[string]float64)
}
for modelName, model := range provider.Models {
trimmedModel := strings.TrimSpace(modelName)
if trimmedModel == "" {
continue
}
normalizedModel := strings.ToLower(trimmedModel)
pricing := parseCostMap(model.Cost)
if len(pricing) > 0 {
parsed[normalizedProvider][normalizedModel] = pricing
}
}
}
return parsed, nil
}

// parseCostMap converts a raw cost map from models.dev (per-million numbers or
// already-normalized per-token strings) into per-token float64 values.
func parseCostMap(raw map[string]json.RawMessage) map[string]float64 {
if len(raw) == 0 {
return nil
}
result := make(map[string]float64, len(raw))
for key, val := range raw {
if len(val) == 0 {
continue
}
// Attempt numeric decode — models.dev stores prices per million tokens.
var f float64
if err := json.Unmarshal(val, &f); err == nil {
result[key] = f / 1_000_000 // convert per-million → per-token
continue
}
// Fall back to string decode (pre-normalized per-token string values).
var s string
if err := json.Unmarshal(val, &s); err == nil {
if parsed, err := strconv.ParseFloat(strings.TrimSpace(s), 64); err == nil {
result[key] = parsed
}
}
}
return result
}

// NormalizeProvider maps provider aliases (e.g. "github", "copilot", "github_models")
// to their canonical form ("github-copilot") and lower-cases all other values.
func NormalizeProvider(provider string) string {
Expand Down
Loading
Loading