NiriZan - Continuous Evaluation Infrastructure and Python Framework for Production AI, built with Pydantic #13689

NiriZan is an open-source Continuous Evaluation Infrastructure and Python framework for AI.

NiriZan is designed around a layered evaluation pipeline: AI systems are instrumented to produce traces, those traces are collected and evaluated by metric components, results are compared against baselines, and the resulting evidence can flow into drift detection, reporting, and CI/CD quality gates.

Because these components need to exchange structured data without becoming tightly coupled, NiriZan treats the boundaries between layers as explicit contracts rather than informal Python data structures. A core design decision in NiriZan is using Pydantic v2 as the contract layer between components. Traces, spans, metric results, regression verdicts, attribution results, and baselines are represented as typed Pydantic models rather than loosely structured dictionaries.

What makes NiriZan different is that it treats the evaluator itself as something that can drift. A change in an evaluation score does not necessarily mean the AI system changed the judge, prompts, evaluation environment, or other parts of the evaluation pipeline may have changed too. NiriZan is built to detect regressions while also monitoring the reliability of the machinery used to measure them.

The project is open source, fully documented, and available on PyPI:

🔗 GitHub: https://github.com/Red1-Rahman/NiriZan
📚 Documentation: https://nirizan.readthedocs.io/en/latest/
📦 PyPI: https://pypi.org/project/nirizan