Pinned Loading

1. TokenPowerBench TokenPowerBench Public

   TokenPowerBench: Benchmarking the power consumption of LLM inference across engines and deployment configurations

   Python 15 1

2. LLM-Inference-Engine-Benchmark LLM-Inference-Engine-Benchmark Public

   A comprehensive benchmarking tool for measuring energy consumption and power efficiency of Large Language Model (LLM) inference engines including vLLM, DeepSpeed, TensorRT-LLM, and Transformers. Fe…

   Python 6

3. llm-power-profiler llm-power-profiler Public

   Lightweight local monitor for watts, tokens, and joules per token on OpenAI-compatible LLM servers.

   Python

4. vllm-project/semantic-router vllm-project/semantic-router Public

   A programmable Mixture-of-Models router for heterogeneous LLM inference

   Go 5.1k 811