Skip to content

Repository files navigation

syncables

This code is open source and was produced by Michiel de Jong, using Claude as a tool. Michiel de Jong has signed off on all the code in this repo line-by-line (except for the lockfiles, which were produced by npm and pnpm), and Michiel de Jong is the publishing author in terms of copyright. This work was funded by NLNet.

Reads an OpenAPI document and gives you:

  • a mock API server that implements it, backed by a real (in-memory) CRUD store per resource, seeded with fake data generated from the document's schemas;
  • an API client that talks to any server implementing that OpenAPI document and keeps a local copy of each resource collection in sync.

Usage

pnpm install
pnpm build
pnpm test
import { loadOpenApiDocument, createMockServer, createApiClient } from 'syncables';

const document = await loadOpenApiDocument('./petstore.yaml');

const server = createMockServer(document);
const { url } = await server.listen();

const client = createApiClient(document, { baseUrl: url });
await client.sync(); // pulls every discovered resource collection into local storage

const pets = await client.list('/pets');

A "resource" is any pair of an OpenAPI collection path and its matching item path, e.g. /pets and /pets/{petId}. Paths without that pairing (health checks, one-off actions, etc.) are served from their documented examples/schemas but aren't treated as syncable resources.

Keeping in sync

sync() is safe to call on a timer: it conditionally re-fetches using ETag/Last-Modified (or a fallback comparison against the previous sync when a server doesn't support conditional requests) and only touches local storage for items that actually changed.

const handle = client.startPolling({
  intervalMs: 30_000,
  onSync: (result) => console.log('changed:', result.changed),
  onError: (error) => console.error('sync failed:', error),
});

// later
handle.stop();

Writing

create/update/remove are local-first: they update local storage immediately and return, then apply themselves against the server in the background, retrying on failure until they succeed.

const pet = await client.create('/pets', { name: 'Milo', tag: 'cat' });
// `pet` is already in local storage — the POST to the server is still
// happening (and retrying, if needed) in the background.

client.pendingWrites('/pets'); // writes not yet confirmed by the server

NLnet milestone 1

This package is the reference implementation for milestone 1 of the project's NLnet grant, which is split into two parts:

  • Part a) Read-only versioncreateApiClient's sync()/paginate() pull a full local-first copy of a resource collection (walking pagination in full, then re-fetching conditionally on later calls) into a pluggable StorageAdapter, from nothing but the OpenAPI document — see discoverResources (src/resources/discover.ts) for how "syncable" resources are found in it.
  • Part b) Full bidirectional version — the create/update/remove methods make that copy writable, not just readable: each write lands in local storage immediately, then applies itself against the server in the background through a per-record retry queue (src/client/client.ts), so the local copy can be both pulled and pushed, as the milestone describes.

Generative AI use

syncables is developed collaboratively with Claude Code (Anthropic), an agentic coding assistant: a human directs the design and reviews, edits, and tests the changes it proposes before they're committed.

As an NLnet-funded project, this follows NLnet's Generative AI policy:

  • Commits produced with AI assistance carry a Claude-Session: <url> trailer identifying the session that produced them.
  • docs/ai-logs/ holds prompt/output disclosure logs for sessions going forward, redacted for secrets and personal information, per the policy's terms for a project that was already ongoing before the policy took effect (no retroactive backfill of every past session; known historical session links are indexed as pending in docs/ai-logs/pending-historical-sessions.md).
  • AI-drafted content is reviewed and edited by a human before being committed; it is not represented as unassisted human work.

About

Keep a local copy of a dataset that is behind an API

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages