model-auditing

AI Red Team Operations Console

A structured audit framework for ML pipelines. 14 stages, 100+ checks, and every leakage pattern that silent model failures are made of.

Steering vectors with receipts: make one, catch one, deploy a calibrated one. pip install hidden-directions

Autonomous Update Review Architecture for Federated Learning. A full-stack platform for inspecting, scoring, and auditing model updates using SHAP explainability, anomaly detection, and ledger-based tracking. Built with FastAPI, React 19, and Flower.

A sealed-room protocol to audit AI models you can't steal, with probes the developer can't see, results neither side can forge, and proof the model is the real deployed one. TEE + attestation + query governor + dual-signed transcript.

Third-party-verifiable model-authenticity audit for the javis.bot Claude/GPT relay — open-source probe, runs on GitHub Actions, code/runs/results all public.

Shift type, not magnitude, determines ML failure modes under deployment shift — a cross-domain audit protocol and benchmark

Reproducible pipeline for silent-failure auditing in ECG accept-sets (MIT-BIH) with Newton–Puiseux onset scoring

A method-neutral protocol for recording and verifying evidence that latent model states causally influenced downstream decisions, outputs, or actions, without requiring storage of raw internal activations.

Production-grade platform for auditing AI/ML models for fairness, explainability, and robustness. JWT auth, SHA-256 model verification, paginated REST API, dark web dashboard. 30/30 tests passing.

Open-source toolkit for Fourier-based audits of trained quantum neural networks.

A production-grade LLM Evaluation & Benchmarking Framework for systematic model auditing. Features parallel benchmarking, fairness/bias detection, MMLU integration, and a real-time analytics dashboard powered by React and FastAPI.

Audit a public AI text classifier for bias and explain its decisions. Counterfactual fairness probing (Equity Evaluation Corpus style) + occlusion-based token attribution, on a black-box model using only inputs and outputs. Python · Transformers · PyTorch.

studying model misalignment via spectral analysis

🔬 1- A Human-Centered AI & Data Science hub for rigorous Machine Learning model evaluation, comparison, auditing, and selection, combining performance analysis, validation, hyperparameter optimization, calibration, reproducibility, and responsible real-world applications.

SHAP-based bias detection and model interpretability auditing framework. Feature importance analysis and ML audit automation.

Subgroup-stratified, calibration-aware fairness auditing for ML models: DeLong AUC confidence intervals, per-subgroup calibration error, multiple-comparison-corrected significance, and a novel five-axis cross-platform protocol (CPFE). Grounded in peer-reviewed methods.

Workspace-lens audit cards and training-workflow scaffolds for open-weight language models.

Second Look — a clinically-grounded triage safety-net & audit system (Triagegeist hackathon): calibrated ESI + vitals-independent red-flag NLP + honest informative-missingness & fairness audit + live Gradio demo.

AI Evaluator Pro 🛡️ is an AI security auditing tool that checks Hugging Face models for supply chain risks, unsafe formats, and author trust using OSINT + LLMs. It supports direct or discovery-based audits to detect security and integrity issues before deployment.