Skip to content

Add REST config bootstrap helper for zero-config E2E tests#7363

Open
AntoineToussaint wants to merge 2 commits into
db-config-rest-zero-configfrom
db-config-bootstrap-helper
Open

Add REST config bootstrap helper for zero-config E2E tests#7363
AntoineToussaint wants to merge 2 commits into
db-config-rest-zero-configfrom
db-config-bootstrap-helper

Conversation

@AntoineToussaint
Copy link
Copy Markdown
Member

@AntoineToussaint AntoineToussaint commented Apr 23, 2026

Stacked on top of #7362 (which is stacked on #7361).

Motivation

After #7361 and #7362, the gateway boots from an empty Postgres and exposes /internal/config_toml* for editing. The next question is: how do tests build up a real config against a zero-config gateway?

The eventual answer (Phase 2B) is narrow per-object endpoints — POST /internal/functions/{name}, POST /internal/variants/{name}, etc. But those don't exist yet. The interim answer is the existing /internal/config_toml/apply endpoint, which takes a full TOML blob plus a CAS signature.

This PR adds a test helper that wraps the fetch-then-apply CAS dance into a single call, and writes the first end-to-end test that exercises the full loop: empty DB → install config via REST → run inference → verify.

What this PR does

1. bootstrap test helper

New file: crates/tensorzero-core/tests/e2e/zero_config/bootstrap.rs

Three public functions:

  • fetch_current_config(client) -> GetConfigTomlResponse — wraps GET /internal/config_toml with reasonable assertions. Returns the full editable view of the current config. For a just-booted empty gateway, this is the "default singletons" document with empty collections and a non-empty CAS signature.

  • bootstrap_gateway_with_toml(client, toml, path_contents) -> BootstrappedConfig — the headline helper. Performs the fetch-then-apply CAS roundtrip:

    1. Fetch current config to get its base_signature.
    2. POST /internal/config_toml/apply with the new TOML, the same path_contents, and that signature.
    3. Return the response so callers can chain.
  • apply_with_signature(client, toml, path_contents, base_signature) -> BootstrappedConfig — for tests that want to make multiple sequential edits without re-fetching between each. Caller threads the returned signature into the next call.

BootstrappedConfig mirrors ApplyConfigTomlResponse (toml, path_contents, hash, base_signature). path_contents and base_signature are flagged with #[expect(dead_code)] for now — they're exposed for chained-apply tests in follow-up work but aren't read in this PR.

2. End-to-end test that exercises the helper

New test: zero_config_bootstrap_installs_function_and_enables_inference in crates/tensorzero-core/tests/e2e/zero_config/mod.rs.

Flow:

  1. Precondition. fetch_current_config confirms the gateway starts empty (no [functions.zero_config_bootstrap_function] block).
  2. Bootstrap. Apply a minimal TOML with one dummy model ([models.dummy] + [models.dummy.providers.dummy] typed as dummy) and one chat function with a single default variant.
  3. Read-back. fetch_current_config confirms the new function is now visible in the editable TOML.
  4. Behavioural assertion. POST /inference against the newly-installed function. Expect 200. This is the critical step — it catches the case where /apply writes to the DB but fails to hot-swap the live runtime.
  5. Cleanup. reset_to_empty_config re-applies an empty TOML so the next test starts from a clean state.

The MINIMAL_BOOTSTRAP_TOML constant is intentionally trivial: no schemas, no templates, no experimentation, just enough to prove the loop.

Why this approach (and what I rejected)

  • Rejected: building narrow per-object endpoints first (Phase 2B). Out of scope for this stack. The /internal/config_toml/apply endpoint already exists, already handles CAS, and is sufficient for test setup. Phase 2C will rewrite this helper to use narrow endpoints once they exist.
  • Rejected: a per-test reset hook (Drop-style RAII). Async drops in Rust are awkward; an explicit reset_to_empty_config(&client).await at the end of each test is clearer.
  • Rejected: putting the helper in tests/e2e/common/. The bootstrap dance is specific to the zero-config flow; if Phase 2A wants to reuse it, we can promote it then.
  • Rejected: leaving path_contents / base_signature off BootstrappedConfig until needed. Including them now keeps the type aligned with ApplyConfigTomlResponse and avoids a public-API change later.

How reset_to_empty_config works

After a test, we re-apply an empty TOML using the post-test base_signature. The /apply endpoint treats "empty functions / models / etc." as an explicit clear (collections are written through, including emptiness), which restores the gateway to the same shape the next test expects to see at startup. This isn't quite byte-identical to a fresh empty-DB boot (some default singletons round-trip differently), but it's identical for the purposes of these tests.

Test plan

  • cargo check --package tensorzero-core --test e2e --features e2e_tests clean
  • cargo fmt --check clean
  • Run cargo nextest run --profile zero-config -E 'test(zero_config_bootstrap_)' against the docker-compose stack from Gate /internal/config_toml on runtime DB mode + add zero-config E2E #7362
  • Confirm MINIMAL_BOOTSTRAP_TOML parses, validates, and produces a non-empty hash
  • Confirm reset_to_empty_config truly restores empty state (next test that calls fetch_current_config sees no leaked function blocks)
  • Inference assertion catches the live-runtime hot-swap failure mode (manual: temporarily break the swap, confirm test fails)

What this unblocks

This is the skeleton Phase 2A will build on. Phase 2A will:

  1. Reuse bootstrap_gateway_with_toml to install the existing tensorzero.*.toml fixtures into a zero-config gateway.
  2. Re-run the existing inference E2E suite against that REST-bootstrapped gateway.
  3. Compare results against the file-mode run to prove parity.

Phase 2C (later) will rewrite the helper to use the Phase 2B narrow endpoints instead of the bulk /apply endpoint.

Stack context

  1. Allow gateway to boot with only a Postgres connection #7361 — gateway boots with only a Postgres connection
  2. Gate /internal/config_toml on runtime DB mode + add zero-config E2E #7362 — gate /internal/config_toml on runtime DB mode + zero-config E2E
  3. Add REST config bootstrap helper for zero-config E2E tests #7363 (this PR) — REST config bootstrap helper

🤖 Generated with Claude Code

AntoineToussaint and others added 2 commits April 24, 2026 15:32
`TryFrom<StoredRateLimitingConfig> for UninitializedRateLimitingConfig`
unwrapped `stored.rules` into `Vec::new()` on `None` and re-wrapped the
result in `Some(...)`, so a rate_limiting row stored without any rules
rehydrated as `rules: Some(vec![])` instead of `rules: None`. The
forward conversion preserves the distinction, so the two halves
disagreed.

The practical consequence: every `POST /internal/config_toml/apply` call
that went through the TOML parser left the live runtime with one
`Config::hash` and the DB-authoritative `/internal/config_toml` read
with a different one. `TryFrom<TomlUninitializedConfig>` wraps every
section in `Some(..)` regardless of whether the source TOML declared
it, so even a metric-only TOML triggers a rate_limiting row write, and
the asymmetric reverse conversion shifts the canonical `StoredConfig`
TOML used to compute the hash.

The fix preserves `None` vs `Some(vec![])` by mapping over the stored
`Option` instead of collapsing it.

Also adds a regression test in `stored_configs::load_config` that
applies a metric-only TOML through the same `toml_to_config` entry
point the apply handler uses, writes it, reloads it, and asserts both
the canonical `StoredConfig` TOML and the resulting `Config::hash`
round-trip unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Introduces `db_only_boot::bootstrap`, a test-only helper that takes an
empty gateway and installs a full config via
`POST /internal/config_toml/apply`. The helper handles the fetch-then-apply
CAS dance internally so tests can focus on what config they want, not on
threading `base_signature`.

Also adds `db_only_boot_bootstrap_installs_metric_and_hot_swaps_runtime`,
which exercises the helper end-to-end: bootstrap a single metric, verify
the write is visible via `/internal/config_toml`, and confirm the live
runtime was hot-swapped by checking that `/status`'s `config_hash`
advances to the hash `/apply` returned. The test also asserts that the
DB-read hash matches the apply hash — the rate_limiting roundtrip fix
in the preceding commit makes this invariant hold. The config is
provider-free on purpose: the db-only-boot CI job runs against the
production gateway image, which doesn't compile in the `dummy` provider.

The `db-only-boot` nextest profile is pinned to `test-threads = 1` so the
mutating bootstrap test doesn't race with the read-only boot-state tests
in the same profile.

This is the skeleton Phase 2A will build on to replay the main E2E
suite against a REST-bootstrapped gateway.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant