Skip to content

Speed up project effect inference - #45

Merged
dannote merged 3 commits into
elixir-vibe:masterfrom
abstractcoder:fix-local-effect-inference-performance
Sep 13, 2026
Merged

dannote merged 3 commits into
elixir-vibe:masterfrom
abstractcoder:fix-local-effect-inference-performance

Conversation

@abstractcoder

Copy link
Copy Markdown
Contributor

Summary

  • separate effect-only classification from provenance resolution so internal inference does not resolve an unknown reason for every call
  • keep project modules in source-based fixed-point inference instead of falling back to compiled dependency BEAM analysis
  • collect unknown dependency calls during the first fixed-point pass and preload each dependency module once with bounded parallel work
  • preserve explicit provenance, direct dependency classification, typespec inference, and the dependency recursion guard

Addresses #44.

Performance

Measured on a 638-file Phoenix project with the same command: MIX_ENV=test mix reach.check --arch --smells

  • Reach 2.8.2: 91.76s
  • Reach 2.8.3: 473.98s
  • this branch: 129.81s under concurrent system indexing load

Internal profiling showed the first effect-inference pass falling from 220.38s to 2.18s. Dependency preloading took 0.15s for 169 requested MFAs.

Verification

  • mix ci
  • 1,273 main tests and properties passed
  • 23 calibration tests passed
  • Dialyzer: 0 errors
  • focused regression coverage verifies one BEAM load for multiple functions in one dependency module, no dependency fallback for project modules, and no module resolution during effect-only classification

abstractcoder and others added 2 commits September 4, 2026 15:49
Give effect caches a lazy owner independent of short-lived analysis workers. Preserve each dependency target's reachable-function budget while reading its module once, and retain unsupported-node provenance.

Compile BEAM regression fixtures under test/support and assert classification results as well as load counts. Validate with mix ci and a repeated full test run.
@dannote

dannote commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Thanks for tracking this down! I pushed d947643 to fix a cache lifetime race, preserve per-function analysis limits when batching, and keep the existing provenance behavior. I also fixed the test fixtures so they actually load compiled BEAM code.

Preload IR-referenced modules, short aliases, and plugins in a worker that receives only module atoms. OTP 29 can force GC after a successful code load; loading in the graph-owning process repeatedly traversed the large project heap.

Use sorted module atoms for plugin cache fingerprints instead of repeatedly rendering names. Add a cold-module regression asserting loading happens outside the project owner.

Validated with mix ci, repeated full tests, and three cold runs over 804 local Hex corpus files. Median inference dropped from 36.4s to 5.9s with identical classifications for 7476 functions and the same 436 finding identities.
@dannote
dannote merged commit 9a10d32 into elixir-vibe:master Sep 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants