-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Feast Control Plane UI #6192
Description
Summary
Feast has a read-only UI that visualizes registry state. This proposal outlines evolving it into an interactive control plane where users can browse, edit, validate, and apply changes visually — and where AI agents can propose changes that humans verify on screen before applying.
This builds on the Feast UI (#2353), the REST API migration (#5411), CRUD requests (#5301, #5533, #5443), the context engine vision (#5761), and the existing MCP server integration.
Problem
The CLI + YAML workflow is powerful but exclusionary.
Not every Feast user is a data engineer comfortable writing Python definitions. Data scientists and ML engineers who consume features often learn better through visualization than reading YAML. Today, creating a feature view requires knowing table names, column names, and types ahead of time — context-switching between a database client and the feature repo. The edit → feast apply → read terminal → fix → repeat loop is slow, and feast plan (dry-run) only works on local + sqlite (_should_use_plan() in feature_store.py). In managed environments (OpenShift, air-gapped clusters), terminal-inside-pod is often the only option — and those changes don't survive pod restarts.
The existing UI shows state but cannot change it.
The React UI (ui/) has pages for data sources, entities, feature views, feature services, permissions, and lineage. But:
- Entirely read-only. Zero
POST/PUT/DELETEcalls oruseMutationhooks in the React app.useLoadRegistryfetches a single protobuf blob from/registryand derives everything from that snapshot. - REST registry API is GET-only. All routes in
api/registry/rest/are@router.get. The gRPC registry server hasApplyEntity,ApplyFeatureView,ApplyPermission, etc. — but these are not exposed via REST. - Lineage is visual but static. The React Flow graph renders relationships but cannot be edited.
- Permissions page says "manage" but only displays. Fixing a group-based access mistake requires: edit Python → commit →
feast apply→ verify — 5-6 terminal steps for what should be one click.
No verification surface for agentic workflows.
Feast already has an MCP server (fastapi_mcp at /mcp) and OpenLineage integration. But without a visual layer, agents are black boxes — users can't see what an agent proposes, verify its choices, or catch mistakes before they reach the registry. Trust requires transparency, and transparency requires a screen.
Proposed Solution
An interactive Feast Control Plane UI — not replacing CLI/YAML, but complementing it. Phased approach:
Phase 1 — Live Registry + Basic Mutations (MVP)
- Migrate UI from protobuf dump to REST API. The REST GET endpoints already exist with pagination, sorting, filtering, and relationship inclusion — the UI just doesn't consume them yet (Migrate Feast UI to Use REST API Registry Server #5411).
- Add REST write endpoints wrapping the existing gRPC
Apply*/Delete*RPCs. - Add
/planendpoint callingFeatureStore.plan()for structured diff output (currently gated to local+sqlite — registry diffs don't depend on provider and could be generalized). - Create/edit forms for entities, data sources, feature views with inline validation.
- Preview → Apply workflow with diff view and real-time status.
Phase 2 — Schema Discovery + Interactive Lineage + Permissions
- Schema discovery: browse data source tables/columns, create feature views by selecting columns interactively (eliminates the database-client context-switch).
- Interactive lineage: click nodes to edit definitions inline; build on existing React Flow graph + permission overlays (
permissionUtils.ts). - Permissions editor: transform the display-only page into a real editor — select resource types, actions, policy type, save. Run
feast permissions checklogic to surface coverage gaps.
Phase 3 — Agentic Workflows + Observability
- Agent proposals as pending diffs: an AI agent (via MCP) proposes a feature view → it appears in the UI as a visual diff → user reviews, modifies, or rejects → approved changes go through the standard validated apply path.
- Agent-assisted debugging: agent analyzes materialization failures or schema drift, surfaces findings in UI with suggested fixes.
- Observability: feature freshness, materialization history, data source health, usage metrics. Builds on Improve Feature Server Observability #5920 and existing OpenLineage emitter in
FeatureStore.
The control plane becomes the shared workspace where human intent and agent execution meet — the agent proposes, the UI displays, the human decides.
Current State vs What Is Needed
| Capability | Today | Needed |
|---|---|---|
| Registry browsing | Protobuf dump via useLoadRegistry |
Migrate to existing REST API endpoints |
| Registry mutations | gRPC Apply*/Delete*; REST is GET-only |
REST write wrappers |
| Apply via HTTP | None — only FeatureStore.apply() Python API |
REST /apply calling FeatureStore.apply() |
| Plan / diff | FeatureStore.plan() — gated to local+sqlite |
REST /plan; lift provider gate for registry diffs |
| Schema discovery | provider.validate_data_source() for validation |
New endpoint to enumerate tables/columns |
| Lineage | React Flow graph, read-only | Editable nodes, click-to-edit definitions |
| Permissions | Display-only page | Write API + edit forms |
| MCP / agentic | fastapi_mcp at /mcp (serving endpoints only) |
Registry mutation tools + approval workflow |
Technical Notes
- Backend: REST write layer must call
FeatureStore.apply()/FeatureStore.plan()— not bypass to raw gRPC Apply (which skips validation, inference, and infra updates).plan()already acceptsdesired_repo_contents: RepoContents— the entry point for UI-driven diffs without generating Python files. - Frontend: Extend existing React UI (Elastic UI, React Query, React Flow). Add mutation hooks (currently zero).
- Safety: All mutations go through
FeatureStorevalidation. Destructive ops require confirmation. Diff preview before apply. - Auth: Reuse existing OIDC + K8s auth. UI authenticates through the same mechanism as all other Feast clients.
Alternatives Considered
- CLI/YAML only. Works for experienced engineers. Does not address discoverability, onboarding, or agentic verification.
- Internal tools instead of upstream. Fragments the ecosystem. Community demand (Enhance Feast UI to allow users to create Features #5301, [UI] Support creating FeatureView in UI #5533, [UI] Support Create/Update transformation in UI #5443) shows this is a shared need.
- Wait for agentic maturity. MCP and OpenLineage already exist. Building the control plane now provides the human-in-the-loop layer that responsible agent workflows need.
Open Questions
- Where should this live? Extend
ui/(simplest), separate repo, or plugin? - Write API: gRPC wrappers vs full
FeatureStore.apply()path? The latter is safer but requires constructingRepoContentsfrom API input. - Should
FeatureStore.plan()be generalized? Registry diffs don't depend on provider — only infra diffs do. - Code-vs-registry source of truth? If teams use Git, UI-only edits could diverge. Should the UI generate exportable definitions?
- How far should MCP go? Direct mutation tools, or approval-gated proposals only?
References
- Feast Web UI #2353 — Feast Web UI
- Migrate Feast UI to Use REST API Registry Server #5411 — Migrate UI to REST API
- Enhance Feast UI to allow users to create Features #5301, [UI] Support creating FeatureView in UI #5533, [UI] Support Create/Update transformation in UI #5443 — CRUD UI requests
- Improve Feature Server Observability #5920 — Feature Server Observability
- Proposal: Evolve Feast into a Context Engine for AI Agents (Post 1.0) #5761 — Feast as Context Engine for AI Agents
Authors
Aniket Paluskar, Cursor (Claude)