Public release polish - #5
Merged
Merged
Conversation
- Add MIT LICENSE file, update pyproject.toml from Apache-2.0 to MIT - Add authors, readme, urls, and anthropic optional extra to pyproject.toml - Add docs/MODEL_GUIDE.md — model/backend recommendations by VRAM budget - Add docs/USER_GUIDE.md — usage patterns, multi-turn, context management - Slim README to quick-start example, add install steps and backend table - Add VRAM requirements table and smoke test section to BACKEND_SETUP.md - Remove broken ref_docs/Modelfile.ministral reference from BACKEND_SETUP.md - Export ToolParam in __init__.py __all__ - Move raw result tables to docs/results/raw/, update report.py and index.md Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Add CONTRIBUTING.md with setup, test, and contribution guides - Add GitHub Actions CI workflow (Python 3.12 + 3.13, unit tests on push/PR) - Add test status, Python version, and license badges to README - Link CONTRIBUTING.md from README docs section - Remove polish_plan.md (internal working doc) Co-Authored-By: Claude Opus 4.6 <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>
antoinezambelli
added a commit
that referenced
this pull request
Jun 28, 2026
* feat(clients): per-client auth credential mechanism (v0.8.0 Phase A) Add the one-credential-per-request mechanism at the client layer: forge carries exactly one credential to the backend in its native auth header, and two credentials present anywhere is a hard error (Design Principle #1, fail loud, no silent merge or precedence). - base.py: AUTH_HEADER_NAMES, has_auth_header, static_auth_present (construction-time two-source guard), resolve_request_headers (per-call one-credential rule), redact_auth_headers; extra_headers added to the send/send_stream protocol. New MultipleCredentialsError. - openai_compat/ollama/llamafile/vllm: construction api_key+extra_headers (new for the latter three) and per-call extra_headers on every outbound path (incl. ollama think-retry re-issues and llamafile's three bodies), via a _request_headers helper. Never mutates shared construction headers, so a per-request credential can't leak across the proxy's reused client. - anthropic: construction default_headers + per-call extra_headers via the SDK's per-call extra_headers=; re-cases auth headers to the SDK's case-sensitive X-Api-Key/Authorization slots; drops SDK-pinned anthropic-version/anthropic-beta; strips SDK control kwargs (extra_headers/extra_body/extra_query/timeout) a verbatim/passthrough body could smuggle past the credential gate. - Removes the old silent "extra_headers overrides api_key Authorization" behavior (ambiguous two-source config) in favor of a fail-loud guard. Tests: tests/unit/test_client_auth.py asserts real wire headers via httpx.MockTransport (and the Anthropic SDK pipeline), covering construction and per-call credentials, the two-source raises, the no-leak property across serialized requests, cross-protocol re-casing, and the smuggle guards. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * feat(inference): thread per-call credential through run_inference (v0.8.0 Phase B) run_inference and _send_streaming gain an extra_headers param, forwarded to client.send / send_stream using the same splat-only-when-set idiom as raw_openai_tools — the kwarg is passed only when set, so clients and test doubles that don't declare it keep their original signature. This is the seam the proxy uses to forward a relocated inbound credential to the backend. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * feat(proxy): inbound credential relocation + --backend-api-key (v0.8.0 Phase C) Wire the proxy to forward exactly one credential to the backend in its native auth header. An inbound auth header is relocated to the target protocol's canonical slot; a static --backend-api-key is the alternative; two credentials anywhere is a hard error (Design Principle #1). - proxy/auth.py (new): extract_inbound_credential (refuses two distinct auth headers, and — via a marker the header reader sets — the same auth header name repeated; treats a blank value as absent), relocate_credential (same-protocol verbatim; cross-protocol normalizes the token and writes the target's canonical slot: anthropic x-api-key, openai Authorization: Bearer), resolve_inbound_credential (extract → refuse inbound+static → relocate). forge forwards ONLY the one relocated credential; no other inbound header is forwarded, so httpx recomputes transport headers and no hop-by-hop stripping is needed. - server.py: thread inbound headers (per-request _QueueItem.headers) through both the direct and serialized-queue dispatch paths; HTTPServer learns backend_protocol + backend_api_key_present (never the raw secret). A credential conflict surfaces as 400 (client error), not 502. - handler.py: resolve the one credential once; thread into run_inference and the no-tools direct send. - proxy.py / __main__.py: --backend-api-key flag (FORGE_BACKEND_API_KEY env), baked into all backend clients at construction. - anthropic.py: when forge owns the credential (api_key is not None, incl. ""), suppress ambient ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN during construction and map ""→None, so no env credential and no spurious empty X-Api-Key can become a hidden second credential. api_key=None still defers to env (WR). Tests: proxy/auth full relocation matrix + hard errors; server-level threading across both serialize paths; duplicate-header and inbound+static both refused 400 with no secret in the body; Anthropic env-suppression regressions. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * docs(auth): redacted credential debug log + BACKEND_SETUP auth section (v0.8.0 Phase D) - handler.py: at the proxy's credential-resolve point, emit a DEBUG log of the forwarded auth header NAME with the value redacted wholesale (x-api-key: ***) via redact_auth_headers. Never logs a raw secret. No-ops without handlers (library use); the proxy CLI enables it under -v. - docs/BACKEND_SETUP.md: new Authentication section — the one-credential principle, WorkflowRunner (construction vs per-call) and proxy (inbound passthrough vs --backend-api-key) usage, the cross-protocol relocation table, the documented OAuth-via-OpenAI-endpoint limitation, and the ambient-env / keyless-passthrough / redacted-logging notes. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * fix(proxy): zero-credential request to Anthropic backend fails loud as 401 Live testing surfaced that a request reaching an Anthropic backend with NO credential at all (proxy pure-passthrough: api_key="" -> None, no ambient env, no inbound auth header) was refused by the Anthropic SDK with an opaque client-side "Could not resolve authentication method" error, surfaced to the caller as HTTP 502 + raw SDK text. The refusal is correct (an Anthropic backend always needs a credential) but the status/message were poor. Detect the zero-credential precondition before dispatch and fail loud with a clear forge error mapped to HTTP 401: - errors.py: new MissingCredentialError (counterpart to MultipleCredentialsError; carries no secret). - clients/anthropic.py: _ensure_credential() guards send()/send_stream() — a credential is present iff the SDK resolved a construction key (api_key/ auth_token, incl. ambient env at build time) or this call carries a per-call auth header. WR direct use (api_key=None reading ANTHROPIC_API_KEY) and static --backend-api-key are unaffected; only true zero-credential dispatch raises. - proxy/server.py: map MissingCredentialError -> 401 (Unauthorized). Verified live: pure-passthrough Anthropic proxy with no inbound auth now returns a clean 401 with the forge message, no SDK gibberish. +3 unit tests; full suite 1270 passing. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * feat(proxy): defer external-mode backend discovery to first request (v0.8.0 finding #2) External passthrough mode (no --backend-api-key) against a gated OpenAI-shape backend crashed at startup: the context-length probe (llama.cpp /props, vLLM /v1/models) and vLLM's served-name probe ran unauthenticated at boot and got a 401. Defer all external-mode startup backend probes to the first request, where they authenticate with that request's inbound credential — preserving zero-config auto-discovery for the gated-passthrough cohort instead of forcing --budget-tokens. - New LLMClient.discover_backend_metadata(extra_headers): one credentialed probe that returns the context budget and adopts any backend-owned wire identity (vLLM served-model-name) into the client. vLLM collapses its two /v1/models round-trips into one; anthropic/openai_compat/ollama are no-network stubs. - proxy._setup_external defers when passthrough (no static key) and there is a probe to run; a static --backend-api-key keeps eager startup discovery and boot fail-fast. Managed mode and the Anthropic external path are untouched. - A LazyDiscovery latch carries the deferral state proxy -> HTTPServer -> handler; the handler runs the probe once, before BOTH dispatch paths (vLLM needs its identity on every request), fails loud on a missing budget (no silent default), and latches on success only (a failed probe retries on the next request). - New BackendDiscoveryError -> 401 on a backend auth rejection (401/403), else 502. Validation: - 1302 unit tests (32 new: deferral wiring, lazy probe run/latch/failure, client discover_backend_metadata, concurrent first-requests, error->status mapping). - New self-contained gated-backend smoke test (scripts/smoke_test_proxy.py) proving deferred startup, first-request credentialed discovery, clean 401 on a missing credential, and no-latch-on-failure (retry succeeds). Also unstales harness scripts surfaced while live-validating (pre-existing drift): integration_test_proxy.py used a removed `mode=` kwarg (now backend_capability); smoke_test_proxy.py's path-1 test asserted pre-finding-#1 behaviour (now sends an inbound credential, as the one-credential rule requires) and its mocks send Connection: close for the two-hit deferred-probe flow. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * fix(proxy): address Codex review — secret hygiene, auth-error status, empty-bearer, CORS Independent review (Codex) of the v0.8.0 auth branch. This commit addresses the findings that don't change request behaviour; streaming-error status (#2) is held for discussion and the discovery race (#3) is documented, not locked. - #1 (secret hygiene): a backend error body or traceback can echo an inbound auth header. Add redact_secrets() and apply it at the proxy boundary — error responses, the error log, and the handler-error traceback — scrubbing Bearer tokens, x-api-key values, and sk- key prefixes. Best-effort net; forge still never authors a secret into a message. - #4 (empty credential): "Authorization: Bearer " (scheme, no token) was treated as present and relocated to an empty x-api-key / "Bearer ". Presence now checks the token after the scheme, matching relocate_credential — token-less auth headers are absent and fail loud. - #5 (status): a backend 401/403 during normal dispatch was mapped to 502. Map BackendError(401/403) -> 401 (caller's auth problem), keeping other faults 502. - #6 (CORS): allow X-Api-Key (+ anthropic-version/anthropic-beta) in preflight so browser clients can send the Anthropic credential slot. - #3 (documented): comment the deferred-discovery concurrency model — no lock by design (idempotent probe + await-free commit, no torn state) and the single-backend/credential-independent-metadata assumption it relies on. +12 tests (empty/scheme-only bearer, redact_secrets, backend 401/403->401 vs 500->502, no-secret-leak through the proxy response, CORS x-api-key). 1314 unit pass; gated-passthrough smoke suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * fix(proxy): resolve credential + run discovery before flushing streaming headers (Codex #2) A streaming request (stream=true) flushed the 200 OK / SSE headers before the handler ran, so a bad/duplicate credential or a failed first-request discovery came back as 200 + an SSE error event instead of a real 400/401. Run those checks in a pre-dispatch pass before the header flush: on failure return the real HTTP status; on success flush as before (success path unchanged, including the managed-mode "alive while queued" early header). - Extract run_lazy_discovery() from the handler so the pre-check and the handler share one copy (the handler call is a no-op once latched). - server._predispatch resolves the credential + runs discovery; failures map via a shared _send_exception (also dedups the existing error-status mapping). - Only the streaming-error path changes; non-streaming already carried a real status, and successful streaming is unchanged. - Update the BACKEND_SETUP SSE caveat accordingly. Scope note: this catches the proxy-level streaming errors — MultipleCredentials (two creds) and deferred-discovery 401/502. MissingCredentialError raised by the Anthropic client at send time (external passthrough, zero credentials) is a client-send decision the proxy can't pre-detect without a per-backend opinion, so that narrow streaming case still surfaces as an SSE event. +3 tests (streaming dup-auth -> 400, streaming discovery 401, streaming success unchanged). 1317 unit pass; smoke suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * fix(auth): cut backend bodies from error messages; enforce one-credential + reject blank creds at the client layer (Codex pass 2) Second Codex pass on the v0.8.0 auth branch. #3 (secret hygiene, the right way): replace the best-effort redact_secrets() denylist with safe-by-construction errors. BackendError keeps a forge-authored `detail` in its message (safe — forge never writes a secret) and takes the raw backend body via `raw_body=`, which rides exc.body and never enters the message, logs, or traceback. All raw-response sites (resp.text / streaming error bodies) route through raw_body; forge's own diagnostics ("no choices", "missing max_model_len", ...) stay in the message. redact_secrets and the proxy redaction plumbing are deleted. #2 (one credential at the client layer): static_auth_present and resolve_request_headers now COUNT auth credentials and refuse more than one in any single bag — two construction auth headers, or two per-call extra_headers — not just the static-vs-per-call collision. The proxy already enforced this on inbound; this brings the direct library path in line. #4 (no blank/garbage credentials): a blank/whitespace/scheme-only credential is not counted or forwarded. Client constructors guard the Bearer header on api_key.strip(); ProxyServer normalizes a blank --backend-api-key to None so it neither rides the wire as "Bearer " nor wrongly disables lazy discovery. #1 (streaming): kept the pre-dispatch checks before the SSE header. The residual — a backend rejecting the credential mid-dispatch on a streaming request arrives as an SSE error event because the proxy buffers — is documented in BACKEND_SETUP as a known limitation (the real fix is incremental streaming; WorkflowRunner already streams). #5 (multi-tenant race) is out of scope: it requires multiple tenants sharing one forge proxy in front of a per-key model router; the one-backend-one- model assumption is already noted in code. +10 auth tests (two-header refusal at construction & per-call, blank-credential absence, blank static key still defers); secret-not-leaked updated to the cut. 1323 unit pass; smoke suite green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu * chore(release): bump to v0.8.0 + CHANGELOG entry Version 0.7.6 -> 0.8.0 and the v0.8.0 changelog section (first-class auth; one BREAKING change scoped to auth-required backends, ungated local backends unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01NTCB4kdRSVnaw7JrGzJ6Xu --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prepare the repo for public release — documentation, packaging, and organization.
Test plan
🤖 Generated with Claude Code