feat: add start/stop/status subcommands to start_proxy.py - #10
Closed
opencolin wants to merge 17 commits into
Closed
feat: add start/stop/status subcommands to start_proxy.py#10opencolin wants to merge 17 commits into
opencolin wants to merge 17 commits into
Conversation
zai-org/GLM-4.7-FP8 was retired on Nebius Token Factory. A clean install that copies .env.example boots "healthy" and passes /health, but every real request 404s with "Model not found." This bites anyone setting up the proxy without going through install.sh. Swaps the BIG/MIDDLE/SMALL_MODEL defaults (and the matching key in MODEL_PRICES_JSON, so the observability cost estimator still finds a price entry) to moonshotai/Kimi-K2.5, which is live as of writing. Adds a comment pointing users at GET /v1/models so they can verify model availability themselves. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nection
Previously, a 404 "Model not found" from the upstream provider returned
generic suggestions about checking the API key, permissions, and rate
limits — none of which were the actual problem. This was specifically
misleading when the proxy's bundled .env.example pinned a retired model:
the user's key and permissions were fine, the model just no longer
existed on their provider.
Now we pattern-match on the upstream error message and return suggestions
that match the failure mode:
- 404 / "not found" -> point at BIG/MIDDLE/SMALL/VISION_MODEL config and
link to GET <base_url>/models for verification
- 401 / 403 / unauth -> auth-key suggestions (the original default)
- 429 / "rate" -> rate-limit suggestions
- anything else -> the original generic three-line suggestion list
The error 'message' field still contains the raw upstream string, so the
existing diagnostic information is unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Boots the proxy in-process via Starlette's TestClient (no uvicorn, no port binding), invokes /test-connection, prints the JSON result, and exits 0 on success or 1 on failure. Intended for CI and install scripts (install.sh inlines the equivalent boot+probe+kill logic today and could be simplified to just call this). We deliberately do not enter TestClient as a context manager, so FastAPI lifespan handlers (which would open the observability sqlite database and start a background recorder task) don't fire. /test- connection itself records nothing, so this is sufficient for a clean smoke test with no side effects. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several Nebius-hosted models (Kimi-K2.5, DeepSeek-V3.2, GLM-5, Qwen3 thinking variants) emit hidden reasoning tokens before producing visible output. Those tokens count against max_tokens. Empirically, a 64-token request to Kimi-K2.5 comes back with empty content because the entire budget is consumed by reasoning, but no error is raised and the response looks structurally fine. Adds a "Reasoning models" subsection under Configure that lists the known reasoning-style models, explains the implication for max_tokens budgets, and points users at GET /v1/models for verification. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Off-by-default sample launchd plist for users who want the proxy to start at login and stay up automatically. Placeholder REPO_ROOT must be substituted with the user's clone path before loading. Install instructions are in the plist's leading XML comment. User-level LaunchAgent (~/Library/LaunchAgents) rather than a LaunchDaemon, so it doesn't require sudo and dies cleanly at logout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Today, running the proxy in the background means: python start_proxy.py & # in some terminal pkill -f start_proxy.py # to stop lsof -i :8083 # to check whether it's up That works but is ad-hoc and surprises users coming from systemctl- or brew-services-style daemon ergonomics. This adds three positional subcommands, parsed before --help so they don't conflict with it: python start_proxy.py start # detached background, PID file written python start_proxy.py stop # SIGTERM, escalating to SIGKILL after 5s python start_proxy.py status # liveness check via signal 0 PID lives in <repo>/.proxy.pid (gitignored), logs in <repo>/proxy.log (already covered by *.log in .gitignore). Repo-relative paths so two clones on the same machine don't collide on a shared /tmp file. The default "no-args" foreground behavior is unchanged — existing docs and the launchd plist sample (contrib/) keep working without modification. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…talls
.env.example shipped OBSERVABILITY_DB_PATH=/app/data/observability.sqlite3,
which is the path used inside the Docker container (where ./data is
bind-mounted to /app/data). On macOS/Linux non-Docker hosts, /app is
read-only or absent. A fresh `cp .env.example .env` install therefore
boots cleanly until startup_event runs, at which point
observability_recorder.start() raises:
OSError: [Errno 30] Read-only file system: '/app'
and the proxy never binds to :8083.
Removing the line lets the resolution fall through to the existing
defaults that already handle both cases:
- Non-Docker: src/core/config.py defaults to "observability.sqlite3"
in CWD (host-writable, no /app involvement).
- Docker: docker-compose.yml line 10 self-overrides to
/app/data/observability.sqlite3 via
${OBSERVABILITY_DB_PATH:-/app/data/...} alongside the
./data:/app/data bind mount. The compose `environment:`
block takes precedence over `env_file:`, so this is
unaffected by anything in .env.
A multi-line comment in place of the removed value documents the
decision so a future contributor doesn't reintroduce it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix: drop OBSERVABILITY_DB_PATH from .env.example to unbreak host installs
…test
Replaces the multi-step manual setup with a single command. Walks the
user through:
- prereq check (python3 >= 3.9, curl)
- .venv creation + pip upgrade (fresh macOS venvs ship pip 21 which
fails on pyproject editable installs)
- dependency install from requirements.txt
- secure API key prompt via read -rs so the key never lands in shell
history
- validation of BIG/MIDDLE/SMALL/VISION_MODEL against
GET /v1/models — refuses to declare success if any configured
model isn't available on the provider, since the bundled
.env.example has historically pinned models that Nebius later
retired and the failure mode is silent (proxy boots "healthy" but
every real request 404s)
- end-to-end smoke test: boots the proxy, hits /test-connection,
fails loudly if not "status: success"
Final output prints the env vars Claude Code needs to talk to the
proxy. How users wire those into their shell is left to them — bare
exports for global routing or per-invocation env vars are equally
valid choices, and the script does not write to the user's rc files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…in README The "Use with Claude Code" section previously showed a single inline command-line invocation: ANTHROPIC_BASE_URL="..." ANTHROPIC_API_KEY="..." claude That works but doesn't surface the two natural patterns: persistent shell-rc exports vs one-off command-line prefixes. install.sh explains both at the end of an install, but users who didn't run install.sh (or who want to look up the wiring later without re-running the installer) had no equivalent reference. Replaces the one-liner with: - a short paragraph naming the two env vars and what they do - a snippet for shell-rc exports (persistent across sessions) - a snippet for command-line prefix (one-off invocations) The IGNORE_CLIENT_API_KEY note stays at the bottom, unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
opencolin
pushed a commit
that referenced
this pull request
Apr 30, 2026
Adds an interactive prompt at the end of install.sh that offers to
wire Claude Code into the user's shell rc automatically:
Wire this up to your shell now?
[1] use 'claudius' to route claude through the proxy
(bare 'claude' keeps using Anthropic)
[2] set global exports so 'claude' uses the proxy
(every 'claude' invocation routes through the proxy)
[3] skip — I'll do it myself
The default is [3], so a stray Enter does nothing destructive.
Choices [1] or [2] append the chosen snippet to ~/.zshrc (or
~/.bashrc, depending on $SHELL) inside a sentinel-bracketed block:
# >>> claude-code-proxy shell wiring >>>
...
# <<< claude-code-proxy shell wiring <<<
The sentinels exist primarily for easy undo. A user who tries the
proxy and wants to revert to default Claude Code behavior can remove
the block in one command:
sed -i '/^# >>> claude-code-proxy shell wiring >>>$/,/^# <<< claude-code-proxy shell wiring <<<$/d' ~/.zshrc
Re-running install.sh detects an existing sentinel block and declines
to add a second one, telling the user where to find the block to
remove it manually if they want to switch styles.
An undocumented RC_FILE env var lets the test suite redirect writes
to a scratch file without touching the user's real rc.
Stacked on PR #10 (slim install.sh). Once #10 lands, this PR's diff
narrows to just the wiring section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat: add install.sh with prereq checks, model validation, and smoke test
fix: update .env.example to a currently-live Nebius model ID
fix: differentiate model-not-found from auth/rate errors in /test-connection
feat: add --selftest flag to start_proxy.py
…ng-models docs: document the reasoning-model gotcha in README
feat: add macOS launchd agent sample for autostart
Owner
Author
|
Closing — this was a staging branch; the change is now tracked in the corresponding PR against KiranChilledOut/claude-code-proxy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Today, running the proxy in the background means:
python start_proxy.py & # in some terminal
pkill -f start_proxy.py # to stop
lsof -i :8083 # to check whether it's up
That works but is ad-hoc and surprises users coming from systemctl-
or brew-services-style daemon ergonomics. This adds three positional
subcommands, parsed before --help so they don't conflict with it:
python start_proxy.py start # detached background, PID file written
python start_proxy.py stop # SIGTERM, escalating to SIGKILL after 5s
python start_proxy.py status # liveness check via signal 0
PID lives in /.proxy.pid (gitignored), logs in /proxy.log
(already covered by *.log in .gitignore). Repo-relative paths so two
clones on the same machine don't collide on a shared /tmp file.
The default "no-args" foreground behavior is unchanged — existing
docs and the launchd plist sample (contrib/) keep working without
modification.