Skip to content

Feast Control Plane UI #6192

@aniketpalu

Description

@aniketpalu

Summary

Feast has a read-only UI that visualizes registry state. This proposal outlines evolving it into an interactive control plane where users can browse, edit, validate, and apply changes visually — and where AI agents can propose changes that humans verify on screen before applying.

This builds on the Feast UI (#2353), the REST API migration (#5411), CRUD requests (#5301, #5533, #5443), the context engine vision (#5761), and the existing MCP server integration.

Problem

The CLI + YAML workflow is powerful but exclusionary.

Not every Feast user is a data engineer comfortable writing Python definitions. Data scientists and ML engineers who consume features often learn better through visualization than reading YAML. Today, creating a feature view requires knowing table names, column names, and types ahead of time — context-switching between a database client and the feature repo. The edit → feast apply → read terminal → fix → repeat loop is slow, and feast plan (dry-run) only works on local + sqlite (_should_use_plan() in feature_store.py). In managed environments (OpenShift, air-gapped clusters), terminal-inside-pod is often the only option — and those changes don't survive pod restarts.

The existing UI shows state but cannot change it.

The React UI (ui/) has pages for data sources, entities, feature views, feature services, permissions, and lineage. But:

  • Entirely read-only. Zero POST/PUT/DELETE calls or useMutation hooks in the React app. useLoadRegistry fetches a single protobuf blob from /registry and derives everything from that snapshot.
  • REST registry API is GET-only. All routes in api/registry/rest/ are @router.get. The gRPC registry server has ApplyEntity, ApplyFeatureView, ApplyPermission, etc. — but these are not exposed via REST.
  • Lineage is visual but static. The React Flow graph renders relationships but cannot be edited.
  • Permissions page says "manage" but only displays. Fixing a group-based access mistake requires: edit Python → commit → feast apply → verify — 5-6 terminal steps for what should be one click.

No verification surface for agentic workflows.

Feast already has an MCP server (fastapi_mcp at /mcp) and OpenLineage integration. But without a visual layer, agents are black boxes — users can't see what an agent proposes, verify its choices, or catch mistakes before they reach the registry. Trust requires transparency, and transparency requires a screen.

Proposed Solution

An interactive Feast Control Plane UI — not replacing CLI/YAML, but complementing it. Phased approach:

Phase 1 — Live Registry + Basic Mutations (MVP)

  • Migrate UI from protobuf dump to REST API. The REST GET endpoints already exist with pagination, sorting, filtering, and relationship inclusion — the UI just doesn't consume them yet (Migrate Feast UI to Use REST API Registry Server #5411).
  • Add REST write endpoints wrapping the existing gRPC Apply*/Delete* RPCs.
  • Add /plan endpoint calling FeatureStore.plan() for structured diff output (currently gated to local+sqlite — registry diffs don't depend on provider and could be generalized).
  • Create/edit forms for entities, data sources, feature views with inline validation.
  • Preview → Apply workflow with diff view and real-time status.

Phase 2 — Schema Discovery + Interactive Lineage + Permissions

  • Schema discovery: browse data source tables/columns, create feature views by selecting columns interactively (eliminates the database-client context-switch).
  • Interactive lineage: click nodes to edit definitions inline; build on existing React Flow graph + permission overlays (permissionUtils.ts).
  • Permissions editor: transform the display-only page into a real editor — select resource types, actions, policy type, save. Run feast permissions check logic to surface coverage gaps.

Phase 3 — Agentic Workflows + Observability

  • Agent proposals as pending diffs: an AI agent (via MCP) proposes a feature view → it appears in the UI as a visual diff → user reviews, modifies, or rejects → approved changes go through the standard validated apply path.
  • Agent-assisted debugging: agent analyzes materialization failures or schema drift, surfaces findings in UI with suggested fixes.
  • Observability: feature freshness, materialization history, data source health, usage metrics. Builds on Improve Feature Server Observability #5920 and existing OpenLineage emitter in FeatureStore.

The control plane becomes the shared workspace where human intent and agent execution meet — the agent proposes, the UI displays, the human decides.

Current State vs What Is Needed

Capability Today Needed
Registry browsing Protobuf dump via useLoadRegistry Migrate to existing REST API endpoints
Registry mutations gRPC Apply*/Delete*; REST is GET-only REST write wrappers
Apply via HTTP None — only FeatureStore.apply() Python API REST /apply calling FeatureStore.apply()
Plan / diff FeatureStore.plan() — gated to local+sqlite REST /plan; lift provider gate for registry diffs
Schema discovery provider.validate_data_source() for validation New endpoint to enumerate tables/columns
Lineage React Flow graph, read-only Editable nodes, click-to-edit definitions
Permissions Display-only page Write API + edit forms
MCP / agentic fastapi_mcp at /mcp (serving endpoints only) Registry mutation tools + approval workflow

Technical Notes

  • Backend: REST write layer must call FeatureStore.apply() / FeatureStore.plan() — not bypass to raw gRPC Apply (which skips validation, inference, and infra updates). plan() already accepts desired_repo_contents: RepoContents — the entry point for UI-driven diffs without generating Python files.
  • Frontend: Extend existing React UI (Elastic UI, React Query, React Flow). Add mutation hooks (currently zero).
  • Safety: All mutations go through FeatureStore validation. Destructive ops require confirmation. Diff preview before apply.
  • Auth: Reuse existing OIDC + K8s auth. UI authenticates through the same mechanism as all other Feast clients.

Alternatives Considered

Open Questions

  1. Where should this live? Extend ui/ (simplest), separate repo, or plugin?
  2. Write API: gRPC wrappers vs full FeatureStore.apply() path? The latter is safer but requires constructing RepoContents from API input.
  3. Should FeatureStore.plan() be generalized? Registry diffs don't depend on provider — only infra diffs do.
  4. Code-vs-registry source of truth? If teams use Git, UI-only edits could diverge. Should the UI generate exportable definitions?
  5. How far should MCP go? Direct mutation tools, or approval-gated proposals only?

References

Authors

Aniket Paluskar, Cursor (Claude)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions