Skip to content

Latest commit

 

History

History
244 lines (173 loc) · 10.5 KB

File metadata and controls

244 lines (173 loc) · 10.5 KB
title status last_updated version
AllSource Event Store - Monorepo
CURRENT
2026-03-03
0.13.1

AllSource Event Store

High-performance event sourcing platform with distributed architecture and AI-native tooling.

CI Container CI Docker Build Release License: MIT

Core Control Plane Query Service Web MCP Server

Core Image Control Plane Image Query Service Image Web Image


Quick Links

Get Started Quick Start · Docker Guide · Troubleshooting
Architecture Clean Architecture · Tenant Model · Replication Design
API & Specs API Reference · Performance · Event Store Features
Operations Release Guide · Quality Gates · WebSocket Config
Services Core · Control Plane · Query Service · MCP Server · Web
Deploy Helm Chart · Kubernetes · Fly.io

Architecture

Clients --> Query Service (Elixir, :3902) --> Core (Rust, :3900)
                |                                  |
           Control Plane (Go, :3901)         WAL + Parquet + DashMap
           (auth, billing, policies)         (events, projections,
                                              snapshots, schemas)

Core IS the database. No PostgreSQL for events — just WAL (CRC32, fsync), Parquet (Snappy compression), and DashMap (concurrent in-memory reads). Zero external dependencies. Full architecture docs →

Monorepo Structure

apps/
  core/               # Rust event store          — the database
  control-plane/      # Go auth/billing/ops       — the gatekeeper
  query-service/      # Elixir API gateway        — the router
  mcp-server-elixir/  # MCP server (61 tools)     — the AI interface
  web/                # Next.js dashboard          — the UI

deploy/
  helm/               # Helm charts
  k8s/                # Kubernetes manifests
  cloudrun/           # Cloud Run configs
  prometheus/         # Monitoring config
  grafana/            # Grafana provisioning

tooling/
  data-flow-test/     # E2E data flow test
  durability-test/    # WAL/Parquet durability test

Project Status & Roadmap (v0.13.1)

What's New in v0.13.1

  • WebSocket Mint migration: Replaced WebSockex with Mint.WebSocket for native IPv6 support on Fly.io
  • Service JWT auth: Query Service generates service JWTs for authenticated Core WebSocket connections
  • Chronon CLI (apps/chronon): Event-sourced task management CLI with TUI (ratatui) and web (htmx) interfaces, powered by Embedded Core
  • Fly.io production deploy: All services deployed and healthy on Fly.io with internal networking

Previous releases

  • v0.12.0: Network sync transport, configurable conflict resolution, MCP tool event emission, WebSocket backpressure
  • v0.11.0: Embedded Core library (8 phases), full dependency upgrade (arrow 57, datafusion 52, rand 0.10, reqwest 0.13, tantivy 0.25, fastembed 5)

Full roadmap: Consolidated Roadmap · Known gaps: Roadmap P0

Rust Core (port 3900) — docs · features · perf

The database. Source of truth for all event data.

  • 469K events/sec ingestion, 11.9us query latency
  • WAL (CRC32, fsync) + Parquet (Snappy) + DashMap for durability and speed
  • v0.10.4+: persistence wiring fix — env vars now correctly configure WAL+Parquet on startup
  • Leader-follower replication via WAL shipping (design)
  • Schema registry, stream processing pipelines, multi-tenancy with RBAC
  • Vector search (fastembed + HNSW) and BM25 keyword search (tantivy)
  • Embedded API: use Core as an in-process library (1489 tests, 8 phases complete) with TOON output, network sync, and conflict strategies

Go Control Plane (port 3901) — docs

Authentication, authorization, billing, and operational management.

  • JWT auth & RBAC (4 roles, 7 permissions), policy enforcement with audit logging
  • OAuth login (GitHub, Google) with CSRF-protected state cookies
  • LemonSqueezy billing integration, HAL hypermedia API, OpenAPI spec
  • OpenTelemetry distributed tracing

Elixir Query Service (port 3902) — docs · API ref

Stateless API gateway. Routes to Core, delegates auth to Control Plane.

  • Fully stateless, no PostgreSQL dependency
  • Server-side projections with fold-on-read and continuous folding via PubSub
  • POST /api/query/projected — snapshot-aware fold endpoint
  • AUTH_DISABLED mode for local dev (fully bypasses all auth)
  • Tesla HTTP client with connection pooling, Broadway event processing
  • OpenAPI specification via open_api_spex

MCP Server (61 tools) — docs

AI-native interface for Claude Desktop or any MCP client.

  • 61 tools across 11 categories (discover, search, drill-down, context, mutate, ops, tenants, schema, analytics, dev)
  • TOON format responses (~50% fewer tokens than JSON)

Web Dashboard (port 3000) — docs

  • Next.js 16 + TypeScript + React + Tailwind + shadcn/ui
  • Events, API Keys, Billing, Pipelines, Settings pages

SDKs — Rust · Go · Python · TypeScript

  • Self-hosted SDK registry (apps/registry) serving Cargo, Go, npm, and PyPI protocols
  • All SDKs distributed via registry.all-source.xyz

What's Next

Priority Focus Details
P0 Fix existing gaps 5 QS endpoints return 501, Core fork commit stubbed, MCP analytics are basic aggregations
P1 SaaS launch Fly.io deploy, LemonSqueezy products, onboarding wizard, landing page
P2 QS Phase 3 Phoenix Channels WebSocket, Broadway Kafka/RabbitMQ, distributed mode
P3 Future Multi-node Raft, geo-replication (CRDT), GraphQL

Docker Images

All services ship at v0.13.1. Total production footprint: ~129 MB.

Service Image Size Base
Core ghcr.io/all-source-os/allsource-core:0.13.1 15.7 MB Distroless
Control Plane ghcr.io/all-source-os/allsource-control-plane:0.13.1 27.9 MB Distroless
Query Service ghcr.io/all-source-os/allsource-query-service:0.13.1 35.1 MB Alpine
Web ghcr.io/all-source-os/allsource-web:0.13.1 ~50 MB Alpine
# Quick start
docker compose up -d

# Pull specific version
docker pull ghcr.io/all-source-os/allsource-core:0.13.1

See Docker Guide · Release Guide


Development

Prerequisites

  • Rust 1.92+ · Go 1.24+ · Elixir 1.17+ (OTP 27+) · Bun 1.3+

Quick Start

git clone https://github.com/all-source-os/all-source.git
cd allsource-monorepo
docker compose up -d

# Or run individual services
cd apps/core && cargo run
cd apps/control-plane && go run .
cd apps/query-service && mix phx.server

Testing

cd apps/core && cargo test --lib          # 1489 tests
cd apps/control-plane && go test ./...
cd apps/query-service && mix test
cd apps/mcp-server-elixir && mix test

Quality Gates

make quality-rust       # fmt + clippy + test + doc
make quality-go         # vet + lint + test
make quality-elixir     # format + credo + test
make check-versions     # Verify version consistency

See Quality Gates · Quality Gates Setup

Version Locations

Service File
Core apps/core/Cargo.toml
Control Plane apps/control-plane/main.go
Query Service apps/query-service/mix.exs
MCP Server apps/mcp-server-elixir/mix.exs
K8s Manifests deploy/k8s/*.yaml

License

MIT License