kospex

Kospex maps the knowledge, technology and maintenance risk hiding in your git repositories.

It answers questions that are surprisingly hard to answer at scale:

• Who still knows this code?
• What are we actually built on?
• What's quietly going stale?

Kospex inspects cloned repositories on disk and aggregates everything into a single queryable database, so you can ask those questions across one repo or thousands. There are two ways to use it: a CLI for scanning, querying and automation, and a Web UI (kweb) for exploring the data interactively.

Inspired by the excellent Mergestat lite, whose database structure we use to model data from git repositories.

Requires Python 3.12+. A virtual environment is optional but strongly recommended.

# 1. Install
pip install kospex

# 2. Set up ~/kospex (config, database, logs) and ~/code (cloned repos)
kospex init --create --verbose

# 3. Clone a repo — this clones into ~/code/GIT_SERVER/ORG/REPO and syncs it
kgit clone https://github.com/mergestat/mergestat-lite

# 4. Ask some questions
kospex summary
kospex developers -days 90
kospex tech-landscape -metadata

# 5. Explore it all in the browser at http://127.0.0.1:8000
kweb

Already have repositories cloned somewhere? Sync the lot in one go:

kospex sync-directory /path/to/your/repos

For complexity metrics and much better file type detection, install the scc binary (brew install scc). It's optional, but recommended.

Full walkthrough: Getting started.

Who knows what, and whether they're still around.

• Active developers (e.g. who's committed in the last 90 days) vs. historical contributors
• Contribution depth at repo, directory and file level
• Derived code ownership — top and most-recent committers who still work here
• Key person and offboarding risk — repos and files with a single active committer

More than just languages — kospex identifies the whole toolchain from filenames, paths and content:

• Languages and file types, with complexity metrics via scc
• Infrastructure as code — Docker, Terraform and friends
• Package managers and build tools — npm/yarn/pnpm, pip/uv, Maven, Gradle, Go modules, RubyGems, Composer, Cargo, NuGet
• CI/CD pipelines — GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Bitbucket Pipelines, CircleCI, Buildkite, Travis
• Linters and config — eslint, SQLFluff and other quality tooling

The result is a technology landscape you can compare over time — what you build with now, versus twelve months ago.

• Declared dependencies extracted from manifest and lock files across supported ecosystems
• How far behind the current release each dependency is
• Security advisory counts, sourced from deps.dev

Signals that code needs attention — or an owner:

• Orphaned repos — no active committer still contributing
• Aging and unmaintained code — based on last commit activity
• Out of date libraries — dependencies well behind current, or with known advisories
• Complexity hotspots — files that change often and are hard to change

Longer write-ups are in use cases.

Most commands need data synced into the kospex DB first.

Most query commands accept -repo PATH/TO/REPO for a repo on disk, or -repo_id / -org_key / -server to query synced data. For example:

kospex developers -repo_id github.com~kospex~kospex
kospex tech-landscape -repo_id github.com~kospex~kospex
kospex developers -server github.com -days 365

Run kospex COMMAND --help for the switches on any command, or see the full command reference.

kgit pull --all                  # git pull + re-sync every known clone
kgit pull --check --all          # offline staleness report, no network
kgit pull --org github.com~myorg # or scope to one org, server or repo_id
kospex sync-metadata -repo PATH/TO/REPO

Syncing a repo does not refresh its dependencies — see Refreshing data for which command updates which table.

Kospex uses a GIT_SERVER/ORG/REPO directory layout for cloned repos:

For example:

~/code/
  github.com/
    kospex/kospex
    mergestat/mergestat-lite
  bitbucket.org/
    myorg/myrepo

This gives a deterministic way of separating orgs, and different git instances as well (e.g. an on-premise Bitbucket alongside GitHub.com). Override the defaults with the KOSPEX_HOME and KOSPEX_CODE environment variables.

Most tables have a _repo_id column in the format GIT_SERVER~OWNER~REPO, so https://github.com/kospex/kospex becomes github.com~kospex~kospex. Most queries use author_email from git to mean "a developer".

Many reports describe something as active, aging, stale or unmaintained. That's a simple calculation from a given date, using these default rules:

It's applied to the last commit in a repo, the last update of a package manager file, or the release date of a library you depend on. Something labelled "unmaintained" may well be feature complete — but where there are external dependencies, code usually needs a change a couple of times a year.

• Precompute data where possible and useful
• Flatten tables, data warehouse style, to enable easier querying and slicing by git server, owner and repo
• Be as agnostic to the git provider (GitHub, Bitbucket, GitLab) as possible for base use cases
• Be mindful that "there is no perfect", only indicators
• Separate cloning and pull updates from the analysis

• Getting started — installation, authentication and your first sync
• Commands — the full kospex, kgit, kweb and krunner reference
• Web UI guide — what each view shows you
• Use cases — the longer write-ups
• Refreshing data — which command refreshes which table
• Troubleshooting
• CHANGELOG

Bug reports and pull requests are welcome via GitHub issues.

To work on kospex, clone it and install in editable mode. The [test] extra adds pytest and httpx, which the test suite needs:

git clone https://github.com/kospex/kospex
cd kospex
pip install -e ".[test]"
pytest

Quote the extra (".[test]") — zsh treats bare brackets as a glob. If you only want to run kospex rather than develop it, pip install -e . is enough.

Frontend assets are built with npm install && npm run build.

We're aiming to [k]now your c[o]de by in[spe]cting the haruspe[x]. From Wikipedia, The Latin terms haruspex and haruspicina are from an archaic word, hīra = "entrails, intestines" — so yes, we do look at the "guts of your code" to understand your applications, technology landscape (sprawl?) and developers.

MIT © Peter Freiberg