Skip to content

Releases: Anbeeld/beellama.cpp

Release list

v0.4.7 Preview

v0.4.7 Preview Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Sep 03:35
preview-v0.4.7
f792091

Warning

See More

Rolling preview build from v0.4.7 at f7920911b21d. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 13/09/2026 00:34.

Changelog
  • Added true rectangular KVarN storage and attention for 64-dimensional K/V heads, including 64x128 K records, 128x64 V records, WHT64 transforms, target and owned draft caches, full-context and iSWA routing, and F16/BF16 precision tails. Existing D128, D256, and D512 record layouts and state formats remain compatible, while unsupported backend placements continue to fail closed.
  • Optimized CUDA D64 KVarN with block-parallel quantization, batched workspace staging and commit, grouped materialization, split portable/GQA attention, and generated split-MMA decode kernels. Decode consumes compressed records directly at every KV length; prompt batches above the native rotated-query limit use transient tiled materialization. Packed K2/K4 decoding now handles odd indirect and SWA offsets without slowing aligned pairs.
  • Optimized CUDA D128 KVarN decode for low GQA with exact GQA2/GQA4 geometries, an eight-warp split-128 route with a measured shallow-context crossover, cooperative staging of complete packed records, and shared-word unpacking of adjacent 3-, 5-, and 6-bit values, raising occupancy and halving deep-context split/combine work.
  • Fixed tensor-split speculative graph reuse after Meta context growth. Resetting the rotating graph context now recreates every reserved subgraph slot, preventing stale graph pointers when speculative topology changes. Attention planning uses the query-token dimension, and the original four-argument KVarN planner symbol remains available for binary compatibility.
  • Fixed Qwen4Exp QSA serving on KVarN caches. Index-mirroring hybrid memories keep one attention row per physical cache cell, KVarN-only stage slots no longer reach the plain index cache, and state save/restore selects stage rows with the mapping the store actually uses.
  • Fixed a first-decode abort from stale scheduler hash entries after graph reserve and tensor probing: inserting into the hash set now invalidates the reset state, so the next reset always clears it.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.4.7
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.4.7
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.4.7
  • CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.4.7
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.4.7
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.4.7
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.4.7

Browse all container images

Loading

v0.4.6

v0.4.6 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 08 Sep 04:35
v0.4.6
78af832
Changelog
  • Updated the llama.cpp base from 57291f264 (b10689) to 465e49b9c (b10830) and ggml to 0.23.0. Notable inherited changes include sparse FlashAttention for DeepSeek V4 and GLM, fused DFlash encoder injection, concurrent CUDA streams for multi-GPU splits, CUDA MoE reduction and FlashAttention improvements, faster KV-cell history lookup and state restoration, new Tencent Hunyuan 4 and Spark 2.5 model support, expanded DeepSeek V4 vision and Nemotron support, broader OpenCL/Metal/SYCL/Vulkan/Hexagon backend coverage, and upstream server, conversion, RPC, multimodal, and Web UI updates through that merge point.
  • Restored property-safe CUDA graph reuse for tensor splits. Projected Meta graphs now keep stable nonzero identities for standard and KVarN execution, while snapshots of reachable tensor metadata, graph roots, use counts, and operation parameters force rebuilding when executable properties change. Consumer synchronization and generation-scoped projected storage preserve asynchronous tensor lifetimes instead of relying on the broad UID-zero capture suppression introduced with sharded precision tails.
  • Restored the default CUDA KVarN prefill materialization window to 65,536 tokens, avoiding an additional floating-point partial-softmax merge through 64K contexts. Memory-constrained concurrent workloads can select a smaller transient window with GGML_KVARN_WINDOW_CHUNK without changing persistent cache capacity.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.6
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.6
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.6
  • CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.6
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.6
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.6
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.6

Browse all container images

Assets 19

v0.4.5

v0.4.5

Choose a tag to compare

View all tags
@github-actions github-actions released this 06 Sep 19:12
v0.4.5
f45c68b
Changelog
  • Updated the llama.cpp base from 6fdd0ac89 to 57291f264 (b10689). Notable inherited changes include Qwen3.8 Flash Next (qwen4exp) target support with HyperConnections, QSA sparse attention, PLE n-gram embeddings, GGUF metadata and tensor mappings, conversion, multi-GPU placement, and graph lifecycle integration, plus --kv-unified-per-slot context limits, OpenVINO 2026.3.1, and upstream server, Web UI, conversion, model, and backend updates through that merge point. The public lazy-loading API now uses llama_lazy_mode and llama_model_params::lazy_mode instead of llama_tensor_read_lazy and tensor_read_lazy.
  • Added owned draft KVarN caches through --spec-draft-type-k/v for draft-simple, EAGLE3, audited Qwen MTP (Qwen3.5/Qwen3.6 dense and MoE), DFlash1/DFlash2, and non-MLA DSpark. All six KVarN widths are selectable with the intrinsic 128-token exact suffix and no draft-tail option, target and draft settings stay isolated, capability checks use the actual MTP cache layers, DSV4/MLA DSpark fails closed because its latent cache is not dense K/V, and unsafe draft-memory fit probes fail with guidance to use -fit off rather than silently omit draft allocations. DFlash attention semantics resolve from the draft model's metadata before context construction; non-causal blocks use materialized K/V with the validated native exact-tail merge, avoiding a context-by-ubatch F32 score matrix.
  • Extended KVarN to shared Gemma 4 MTP and multi-stream SWA target caches. Gemma 4 MTP reads the target cache's KVarN records and exact-tail rows (configure target --cache-type-k/v kvarn*; explicit draft selections fail closed), the position-addressed SWA ring encodes an independent KV stream per parallel slot with per-stream staging, records, and store/materialization routing on CPU, CUDA, and Vulkan, multi-stream SWA uses materialized attention while single-stream keeps native attention, and server checkpoint, restore, and rollback no longer treat shared-cache MTP drafts as owning draft state.
  • Added standalone Qwen3.8 MTP sidecars. The draft loader follows the configured draft path, device, and offload settings; the Qwen4Exp MTP graph consumes target hidden states and applies the output HyperConnection mixer, and the converter exports complete sidecars.
  • Extended standard quantized caches, KVarN, and F16/BF16 precision tails to Qwen3.8 QSA. Sparse selection composes with generic attention, native KVarN routing, and tail handling, with a validated mirrored index-cache slot layout. Prompt checkpoints preserve transactional hybrid attention/index state, rebuild mirrored precision-tail slots, and restore PLE recurrent rows into the selected destination.
  • Fixed Windows CUDA tensor-parallel speculative decoding for MTP, DFlash1, and DFlash2, covering meta-graph headroom, GQA head partitions, mirrored softmax inputs, CPU scheduler boundaries, Windows graph-projection stack size, DFlash1 backend-sampling fallback, and DFlash2 vocabulary-logit gathering.
  • Optimized CUDA KVarN decode and speculative verification: wider descriptor reduction with division-free single-stream scan, masked multi-query batches through eight rows with skipped empty splits and K/V reuse across query tiles, occupancy-based 64/128-token geometry, packed fragment loads, half-width quantization axes, and chunked windowed materialization. Fast tiles use padded MMA query/probability storage, record-boundary checks, synchronized shared-memory reductions, and query-count-independent split partitioning. Unsupported shapes retain descriptor-native MMA fallback, and GGML_KVARN_DEBUG_ROUTES reports the selected wide MMA path (#132).
  • Improved KVarN structured cache allocation. Single-slot decode carries its planned stage assignment into cache mutation instead of rescanning the full context per token, and general stage-liveness scans use a flat sequence/group matrix instead of per-record tree allocations. Unified-cache planning reserves complete monotonic physical segments for interleaved sequences, preventing record-tile aliasing.
  • Fixed historical KVarN SWA checkpoint restoration: partial SWA checkpoints own their ring records so later prefill cannot evict live anchors, incomplete legacy SWA partial frames and shared-stream ring replacement fail closed, and repeated historical-prefix reuse no longer reprocesses the whole prefix (issue #140).
  • Fixed DFlash image position validation and SWA capacity (issue #140). Overlapping image positions and the logical gaps after them are preserved for feature injection. Draft generation and rollback use the target's next position rather than the image-expanded token count, preventing the post-image draft-acceptance and decode-speed collapse. Owned, unshared DFlash SWA caches start compact and grow on demand while preserving state, up to the full-context limit, so image feature rows can exceed their logical position extent without forcing full-capacity startup allocation. This applies to ordinary and KVarN caches; explicit --swa-full remains available.
  • Reset per-sequence MTP hidden-state carry-over before processing a prompt from position zero, while preserving restored carry for nonzero continuations and the final prefill state needed by the first draft.
  • Corrected compact-tail and KVarN suffix-rollback capability reporting so legal speculative rollback no longer triggers unnecessary checkpoints; tail-state serialization scans once per stream, MTP draft rollback follows the draft token axis with widened-removal and full-clear recovery, KV-tail placement follows the realized cache device, and projector thread counts reach every compatible scheduler backend.
  • Declined the KV precision tail when no device advertises a native KV-tail attention entry point: the context warns and resolves the request to zero instead of paying the several-times-slower generic tail route, and LLAMA_KV_TAIL_ALLOW_GENERIC=1 keeps it anyway.
  • Fixed Metal compilation for standard Q2 cache quantization and explicit F16/BF16 conversions; embedded Metal assembly now carries a content digest so sccache cannot serve stale kernels.
  • Added native Vulkan FlashAttention decoding for standard q2_0, q2_1, q3_0, q3_1, q6_0, and q6_1 K/V caches, removing the CPU fallback that collapsed decode throughput.
  • Updated release and validation tooling: ROCm runtime images bypass registry build-cache import/export, shared CUDA KVarN kernels are deduplicated with a binary-equivalence verification script, and regression coverage now exercises draft-cache validation, Qwen3.8 MTP conversion and state reuse, DFlash image/SWA growth, and historical checkpoints.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.5
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.5
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.5
  • CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.5
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.5
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.5
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.5

Browse all container images

Loading

v0.4.4

v0.4.4

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Aug 18:46
v0.4.4
f8cd4e6
Changelog
  • Updated the llama.cpp base from 74ce15741 to 6fdd0ac89 and ggml from 0.19.0 to 0.22.0. Notable inherited changes include Qwen3.8 DFlash2 with local convolution, candidate selection, M-RoPE, converter/GGUF support, speculative p_min/n_min, and wide-row CUDA top-k; Granite Switch/SWA, Muse Glimmer, Kimi K3, MiniMax Text, BailingMoE3, Nanbeige4.2-3B, Pocket TTS, expanded MTP, DFlash, and DSpark coverage, and multi-output backend sampling; media-aware server state, Web UI tools and navigation, stronger tool isolation, automatic iGPU-safe loading, Metal kernel splitting, Vulkan TQ2_0 and Lightning Indexer support, asynchronous RPC APIs, and Apple RDMA transport. Bee keeps adaptive draft-max on DFlash1 while DFlash2 uses its fixed trained block and selector confidence.
  • Optimized native KVarN SWA attention on CPU and CUDA. Fully covered non-SWA windows now use native exact storage, and CUDA routing retains split geometry and device capabilities without model- or GPU-specific policy.
  • Hardened KV and speculative state handling with transactional target/draft restore, selective checkpoint validation, KVarN stage ownership and generation tracking, restored bulk-workspace routing, complete DSA+iSWA lifecycle forwarding, head-aligned multi-GPU placement, and correct partial sharded-buffer memset.
  • Fixed MTP multi-ubatch synchronization and extended loop detection to pathological visible output while retaining force-close behavior for hidden reasoning loops.
  • Updated release packaging to CUDA 13.3, source-built Web UI assets, non-development version metadata, fetched LLVM OpenMP on Windows, and matching HIP runtime DLLs. Release metadata now validates the source version, Windows package checks fail closed on required runtime files, preview and stable concurrency are isolated, and reruns reconcile stale assets.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.4
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.4
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.4
  • CUDA 13.3: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.4
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.4
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.4
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.4

Browse all container images

Loading

v0.4.3

v0.4.3

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Aug 21:14
v0.4.3
ba27eda
Changelog
  • Updated the llama.cpp base through upstream commit 74ce15741. Notable inherited changes include Qwen3-TTS, DeepSeek V4 and DSpark, MTP support for GLM-4.7-Flash, GLM-5.2, Qwen3-Next, and DeepSeek V3.2, router LRU scheduling, initial Docker tool isolation, working-directory and filesystem tools in the server and Web UI, speculative metrics, and broad CUDA, Metal, Vulkan, SYCL, WebGPU, multimodal, conversion, and UI updates. ggml is now 0.19.0 and the RPC protocol is 5.0.1.
  • Fixed HIP/ROCm KVarN routing. Precision tails remain on native attention instead of allocating a context-sized F16 materialization, unsupported AMD matrix shapes fall back to portable direct-record attention, and split/vector decode is selected only for supported AMD routes. Capability reporting, route telemetry, memory accounting, and Vulkan RoPE/offset checks now match the implemented kernels.
  • Made KVarN and precision-tail memory fitting exact and bounded. The CLI, fit probes, and final context share one immutable tail request; invalid group specifications fail context creation; Vulkan uses backend workspace planners during fit; and final no-allocation validation restarts fit from pristine inputs or fails deterministically. Vulkan also gains a native one-softmax route for standard quantized bodies with F16/BF16 history and current segments.
  • Reworked prompt-cache reuse across standard, recurrent, and KVarN state. Live and RAM candidates share one safe-prefix planner and transactional target/draft/speculative restore; failures leave live state unchanged. Self-contained sequence-selective checkpoints preserve precision tails and repeatable RAM restores. Durable KVarN reuse remains descriptor-aligned, while standard/recurrent batching and explicit GPU-fit behavior remain upstream-compatible. Unified KVarN slots now borrow shared capacity instead of using fixed stripes, and full state remaps live stage rows across slot counts; compact sparse reads and batched checkpoint transfers improve multi-slot --cache-ram prefill throughput.
  • Fixed CUDA KVarN attention at both ends of the hardware range. Pre-Turing portable attention now passes architecture-neutral capability checks and exports the softmax metadata required to merge exact tails correctly. Split and vector decode share a once-per-device dynamic shared-memory opt-in, preventing configuration failures and extending the vector combine path beyond the default 48 KiB limit. KVarN MMA warning fixes keep fatal-warning builds clean.
  • Restored status.args and status.preset in router /v1/models responses for upstream-compatible clients while continuing to strip sensitive options.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.3
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.3
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.3
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.3
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.3
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.3
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.3

Browse all container images

Loading

v0.4.2

v0.4.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Aug 04:09
v0.4.2
d71585e
Changelog
  • Updated the llama.cpp base through upstream commit 6ba5ef247. Notable inherited changes include MiniMax-M3 sparse attention and vision, GLM-5.2-Vision and its indexer, Laguna, Nanbeige4.2, MiMo-V2.5 audio, DeepSeek4 fused operators, BitNet conversion, gpt-oss EAGLE3-v3, server MCP stdio, "reasoning_effort": "none", multi-end-sequence reasoning budgets, draft-sidecar discovery, broad backend and Web UI updates, and the unified --load-mode (-lm) selector. GGML_OP_COUNT is now 105 and the RPC protocol version is 4.0.5.
  • Enabled bounded-memory native KVarN attention on portable-only CUDA devices, including the pre-Turing route needed by issue #112. A versioned backend capability record now distinguishes portable body-plus-F16/BF16-tail support from specialized MMA, split/vector decode, original-V support, and per-route query limits. Portable CUDA prompt batches consume rotated compressed records directly instead of re-entering classic full-score attention; Turing-and-newer route selection is unchanged. CI compiles explicit CUDA 12.4 Maxwell/Pascal/Volta targets and CUDA 13.1 current targets, while pre-Turing runtime qualification remains pending matching hardware.
  • Added experimental --split-mode tensor support for standard quantized and KVarN caches with F16/BF16 precision tails. Cache components split only at complete-head boundaries, dynamic meta graphs and auxiliary speculative contexts follow the target topology safely, invalid weights fail during model creation, and invalid cache layouts fail during context creation. Layer split is unchanged; physical multi-GPU peer transfer, NCCL, and heterogeneous rollback still require validation.
  • Added CUDA support for serialized ternary, group-64 Q2_0 model weights across dequantization, GET_ROWS, MMVQ, and MMQ, including corrected template declarations. This remains distinct from Bee's cache-facing GGML_TYPE_Q2_0S.
  • Fixed CUDA matrix-kernel selection for padded or offset right-hand-side tensors. MMVF, MMF, fused, and expert paths now reject addresses or strides that are unsafe for their float2 loads instead of asserting in the kernel.
  • Fixed KVarN correctness during unaligned eager stores, multi-turn suffix removal, compact-tail reprocessing, and deep SWA rollback. Reopened groups retain the correct F16 source, compressed-body tails avoid unnecessary reprocessing, and record-safe rollback boundaries prevent stale rows.
  • Restored context creation for cacheless, noncausal architectures after KV-tail type resolution; requesting a precision tail for a model without cache memory now fails explicitly.
  • Fixed release and test infrastructure: Windows CPU and Vulkan jobs resolve Visual Studio through vswhere, CPU packages require OpenMP, release ccache can seed from the nearest ancestor version branch, LLAMA_BUILD_TESTS=ON builds compile after the upstream reasoning-budget API change, and release regressions run without ignored local files.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.2
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.2
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.2
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.2
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.2
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.2
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.2

Browse all container images

Loading

v0.4.1

v0.4.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 26 Jul 16:19
v0.4.1
176c1a1
Changelog
  • Extended native KVarN attention beyond the optimized NVIDIA CUDA path. CPU and Vulkan consume compressed records directly for advertised shapes, while HIP/ROCm uses capability-driven routing across native generic MMA, physical-wave split/vector decode, and portable direct-record attention. HIP selects wave32 or wave64 kernels for WHT, store, and attention, includes explicit RDNA/CDNA wide-MMA configurations and a CDNA-safe softmax fallback, and retains materialization for unsupported shapes or features. CPU placement works with KV offload disabled; Vulkan native attention requires shader Int64 and buffer-device-address support. CUDA and ROCm release builds explicitly enable the shared KVarN kernels, while KVarN-off builds retain link-safe false capability stubs and do not export unavailable store telemetry. The backend-query declarations, static checks, and memory mocks were updated for fatal-warning and test builds. Adding GGML_OP_KVARN_MATERIALIZE advances the RPC protocol patch version to 4.0.4.
  • Reduced KVarN host checkpoint stalls for large exact tails by serializing component-major contiguous slot runs instead of issuing one backend transfer per payload row. A normal 4096-token tail now takes 32-64 host tensor operations instead of 131,072. KVarN state format v13 preserves all exact-tail bytes and supports transactional host/on-device state, including wrapped on-device restores, plus compatible v12 reads while exposing debug-level transfer-count diagnostics.
  • Kept small speculative verification batches on CUDA's native KVarN attention path instead of materializing the full active K/V window. CUDA now reports its rotated-query capacity to the model graph through 16 tokens. Single-token generation retains the split/vector decode kernels, while multi-token verification uses tiled descriptor-native MMA so each decoded K/V tile is reused across query rows and does not create the split kernel's context-sized per-query partial output. Rotated-record loads now distribute reconstruction across the full CTA, cache K/V axes per record, decode packed value and key pairs cooperatively, and use an adaptive 128-column tile for Q9-Q16/GQA>4 when the device's opt-in shared-memory limit and measured kernel occupancy allow it; unsupported devices retain the 64-column fallback. Larger batches retain the native prefill route. This applies uniformly to DFlash, MTP, EAGLE3, and n-gram speculation. CUDA builds with dedicated KVarN kernels disabled also retain complete false capability stubs and link cleanly.
  • Fixed portable-backend KVarN prompt processing and Vulkan long-context decode. CPU, HIP, and Vulkan now materialize large query batches into their tiled standard FlashAttention path instead of running prompt prefill through the scalar direct-record shader; one-row decode and Vulkan batches through 16 rows remain native, while CUDA retains its descriptor-native prefill route. Vulkan direct decode now reuses each K/V reconstruction across groups of up to four GQA heads, applies split-K when the query/head grid under-occupies the device, reuses shared K/V indices, and uses subgroup reductions.
  • Reworked Vulkan KVarN execution around explicit backend-native routes. Compact bodyless F16/BF16 tails use validated native attention for supported head dimensions and short query batches; mixed compressed-body plus exact-tail inputs use explicit materialization followed by standard tiled FlashAttention. Large dense prefill stores validate and quantize records in parallel using reusable cache workspace, D256/D512 transforms use full-head workgroups when the device permits them, and materialization resolves each stream's live descriptor once per dispatch instead of once per output workgroup. Route counters, bounded diagnostics, synchronized Vulkan timestamp profiling, and backend-private transient-memory accounting make native, materialized, and fail-closed outcomes observable without changing CUDA/HIP routing. llama-bench --kv-memory selects telemetry from the benchmark device, consumes the complete versioned route ABI, and takes synchronized Vulkan allocation checkpoints; Vulkan memory-budget queries saturate safely when driver-reported heap usage temporarily exceeds its budget.
  • Reduced KV precision-tail memory and made routing authoritative per layer. Persistent exact storage is N + rollback rows, or W + rollback rows with no compressed body when an exact tail fully covers its SWA window, instead of a ubatch-sized arena rounded to a FlashAttention tile. Current K/V rows and execution alignment now use graph-local workspaces. One execution descriptor drives allocation, graph construction, graph identity, backend validation, and telemetry, so the final fused attention operation must support its planned native route or fail closed instead of moving silently to CPU. llama-bench --kv-memory reports exact-history, rollback-reserve, persistent, transient, and peak bytes together with native-bodyless, native-mixed, device-fallback, and CPU layer counts.
  • Made compact-tail updates and speculative rollback transactional. Memory implementations now advertise full-clear, arbitrary-range, and bounded suffix-removal capabilities through llama_memory_get_seq_rm_capability; the common layer no longer probes support by decoding and clearing dummy tokens, and the server chooses checkpoints when a speculative rollback exceeds the advertised reserve. Graph completion commits or invalidates pending compact-tail metadata, negative indexed writes are ignored consistently across backends, and bodyless full-SWA attention preserves independent planner masks and full/SWA query order. Sequence-state precision-tail manifest v3 records the compact representation and rollback horizon, rejects incompatible or corrupt payloads before mutation, and exposes degraded invalid-payload coverage; older compatible precision-tail manifests remain readable.
  • Added ffmpeg and ffprobe to every released server runtime image, including CPU, CUDA, ROCm, Vulkan, and SYCL variants, so builds produced with MTMD_VIDEO=ON can process MP4/WebM uploads instead of failing after still-image decoding succeeds.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.1
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.1
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.1
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.1
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.1
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.1
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.1

Browse all container images

Loading

v0.4.0

v0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 19 Jul 19:04
v0.4.0
5eaba17
Changelog
  • Updated the llama.cpp base through upstream commit 32e789fdf. This includes EAGLE3 and expanded MTP support; new multimodal models and video input; LFM2/LFM2.5 tool parsing; router, API, and Web UI updates; the ET-SOC backend; and updates across the existing compute backends.
  • Added KVarN target-context KV-cache compression through --cache-type-k and --cache-type-v values kvarn2, kvarn3, kvarn4, kvarn5, kvarn6, and kvarn8. It supports independent K/V widths, Qwen3.6 and Gemma 4, unified and non-unified caches, separate SWA cache types, CPU/CUDA stores, and native CUDA FlashAttention. Draft and auxiliary contexts continue to use standard cache types; unsupported placements fail closed or use an explicit bit-width-matched fallback.
  • The standard KV-cache surface now includes q2_0, q2_1, q3_0, q3_1, and q6_1 alongside q6_0, with CPU/CUDA quantization, SET_ROWS/GET_ROWS, CUDA MMQ/vec-dot, and FlashAttention vector coverage. Cache-facing q2_0 uses GGML_TYPE_Q2_0S, keeping it distinct from upstream's serialized Q2_0 weight format.
  • New KV cache precision tail (KVCPT) controls --kv-tail-tokens and --kv-tail-type keep the newest attention-visible entries exact in F16 or BF16 for standard quantized and KVarN target caches. Partial tails use compact overlays and fully covered groups can use native exact storage. Tail routing respects attention bias and cache placement, while state save/restore, prompt-cache reuse, sequence operations, and server handoff preserve the same model. Restores are transactional, and incompatible v0.3.1 sessions fail closed.
  • Scoped --cache-ram to prompt-cache storage instead of active context checkpoint policy. Prompt-cache saves are byte-counted against the host budget before commit, KV rollback aligns to multimodal chunk boundaries, text-only slot save/restore works when an mtmd projector is loaded, and MTP prompt-cache reuse no longer requires regular checkpoints.
  • Replaced the fork DFlash implementation with upstream draft-dflash. The speculative type uses upstream's draft-dflash name, and draft GGUFs must use upstream's dflash architecture, metadata keys, tensor names, and tokenizer contract. Bee retains a default-on profit-based draft-depth controller and reasoning-loop guard while using upstream token, sampler, and checkpoint behavior. Without --spec-draft-n-max, the draft limit comes from dflash.block_size - 1; an explicit value takes precedence.
  • Removed TurboQuant/TCQ cache formats, TQ3/TQ4 weight formats, DDTree, CopySpec, the fork DFlash ring/capture/tape and reduced-verifier paths, the fringe controller, and their private arguments and environment variables. The turbo2, turbo3, and turbo4 cache names, including TCQ variants, warn and redirect by width to KVarN for target caches or a standard low-bit type for draft caches. Legacy TQ GGUF type IDs fail with a re-quantization error; copyspec, suffix, and recycle fail with migration guidance to draft-dflash or upstream n-gram modes.
  • Replaced the CUDA FlashAttention quant build policy with 50 standard vector pairs and 15 balanced KVarN fast-decode pairs by default. Standard quant pairs follow the KVarN bit-pair rules; homogeneous F16 and BF16 pairs retain native precision-tail support. GGML_CUDA_FA_ALL_QUANTS=ON expands these to 169 standard pairs and all 36 ordered KVarN pairs. The default-on GGML_CUDA_KVARN option is the single KVarN compilation gate for CUDA and HIP; disabling it omits the dedicated kernels and template instances.
  • Hardened server control and router APIs. /v1/chat/completions/control reports success only when an active reasoning sampler accepts the transition. GET /models is read-only and sanitized; authenticated refresh uses POST /models/reload. Hugging Face tokens reach child processes only through HF_TOKEN and are excluded from arguments, presets, logs, and responses.
  • Updated release packaging with CUDA 12.4/13.1 assets, upstream-DFlash and KVarN container metadata, ROCm/HIP shuffle compatibility, Windows CPU OpenMP runtime packaging, and updated Intel SYCL images. Every platform build, package, container, and publication step uses the same resolved source SHA; publication stops if the selected branch or tag moves.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.4.0
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.4.0
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.4.0
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.4.0
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.4.0
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.4.0
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.4.0

Browse all container images

Loading

v0.3.2 Preview

v0.3.2 Preview Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Jun 13:02
preview-v0.3.2
fe67745

Warning

Rolling preview build from v0.3.2 at fe67745dbbd5. Assets and moving Docker tags are replaced by newer successful branch builds. Last updated: 10/07/2026 21:29.

Changelog
  • Merged a newer upstream llama.cpp master after the current main baseline. Notable inherited changes include Granite 4 Vision, Gemma 4 MTP including E2B/E4B assistants, Gemma 4 unified conversion and audio-projector fixes, multimodal video input with ffmpeg in the released image, Qwen-VL frame merge support, Mistral-Medium-3.5 conversion, the unified LFM2/LFM2.5 tool parser and reasoning round-trip fixes, speculative vocab-compatibility checks, the placeholder-bitmap token counting and */input_tokens API, optional server prompt logging, KV-cache cell-sharing/copy-avoidance fixes, GGML_OP_COL2IM_1D, ggml 0.14.0, CUDA 13.3 release images, HIP gfx1152/gfx1153 support, and backend/UI improvements across WebGPU, Vulkan, SYCL, Metal, CPU, and the Web UI.
  • Added experimental KVarN KV-cache compression for target contexts through --cache-type-k / --cache-type-v pseudo types kvarn2, kvarn3, kvarn4, kvarn5, kvarn6, and kvarn8. K and V can be selected independently across all 2/3/4/5/6/8-bit pairs, with one-sided KVarN inputs normalized to a concrete pair and draft/auxiliary contexts kept on normal cache types.
  • Wired KVarN into Qwen3.6 and Gemma 4 target memory paths, including 128/256/512-dimensional K/V heads, unified and non-unified KV storage, supported single-stream iSWA/SWA rings, explicit --cache-type-k-swa / --cache-type-v-swa overrides for SWA-layer precision, prompt-cache state save/restore, and bit-width-matched fallback cache types for layers that cannot use KVarN records. Unsupported placements fail closed unless normal-KV fallback is explicitly enabled.
  • Added native KVarN runtime support: CPU/CUDA store ops, native CUDA FlashAttention view consumption without graph-level F16 materialization, bounded windowed prefill for Qwen/Gemma/SWA paths, compact group-range prompt-cache state, ROCm/HIP low-shared-memory store support, guarded Vulkan store support, and llama-bench KVarN cache names.
  • Added new KV/cache types: turbo4_tcq plus standard quantized q2_0, q2_1, q3_0, q3_1, and q6_1, with CPU/CUDA quantize/dequantize, MMQ/vec-dot, FlashAttention vec coverage, and use as KVarN non-KVarN-layer fallback targets where appropriate.
  • Reworked CUDA/HIP/MUSA FlashAttention cache-type coverage. The no-flag vec build now focuses on the recommended q-cache and KVarN fallback pairs, GGML_CUDA_FA_HALF_QUANTS adds the K>=V half matrix for broader Turbo/TCQ experiments, GGML_CUDA_FA_ALL_QUANTS keeps the full matrix, and runtime diagnostics report or optionally ignore uncompiled pairs instead of failing opaquely. Turbo/TCQ route planning now validates effective K/V types, fixes the issue #41 CUDA paths, and keeps Gemma-sized D256/D512 mixed-Turbo routes away from unsupported vec paths.
  • Improved DFlash serving relative to v0.3.1: tensor-split Meta target placement keeps auto-detected drafters on compatible placement, reduced verification is gated by backend capability, DFlash mixed with other speculative types is detected order-insensitively for rollback planning, and MTP draft policy is isolated from DFlash-only adaptive logic.
  • Reduced DFlash/recurrent and prompt-cache memory pressure. Flat DFlash on recurrent targets now uses recurrent-only rollback state instead of dead attention/KV backup streams; prompt-cache saves are prepared and byte-counted before commit; cached prompt entries copy only the newest fitting context checkpoints; and --cache-ram remains scoped to prompt-cache storage rather than active context-checkpoint policy.
  • Improved the DFlash profit adaptive draft-max controller so cold starts hold the maximum useful depth while lower depths are characterized through gated probes, then demote only when measurements show a lower depth is actually faster.
  • Hardened shipped builds and release artifacts: ROCm/HIP shuffle compatibility, HIP/MUSA KVarN build fixes, Windows CPU OpenMP runtime packaging, Intel SYCL Docker images updated from compute runtime 25.40 / IGC v2.20.5 to compute runtime 26.18 / IGC v2.34.4, and release downloads/notes filtered to final packages and runtime artifacts.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-preview-v0.3.2
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-preview-v0.3.2
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-preview-v0.3.2
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-preview-v0.3.2
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-preview-v0.3.2
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-preview-v0.3.2
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-preview-v0.3.2

Browse all container images

Loading

v0.3.1

v0.3.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Jun 21:21
v0.3.1
15d22ac
Changelog
  • Merged latest upstream llama.cpp master. This pulls in Gemma 4 12B and Gemma 4 unified multimodal support fixes, including non-causal vision, unified audio/vision projector handling, and FPE fixes; Qwen3.5 post-norm hidden-state behavior for MTP; CUDA KV-cache quantization preallocation and PDL race fixes; WebGPU FlashAttention refactoring with standardized quantization support; CPU backend improvements for RVV/SVE; lower-latency Metal command-buffer status polling; Mermaid diagram rendering and preview support in tools/ui; updated BoringSSL, SYCL documentation, save/load-state tests, Docker docs, and small CI/release maintenance.
  • Repaired CUDA fused TurboQuant FlashAttention for same-type turbo2, turbo3, and turbo4 K/V caches. The fused MMA path now loads each supported format correctly, while mixed TurboQuant and TCQ pairs stay on the established non-fused paths; TurboQuant/TCQ partial KV offload now fails early instead of falling back to an incompatible CPU cache and reaching a scheduler crash. Added GGML_TURBO_FA_DEBUG=1 path diagnostics and regression coverage for the supported dispatch matrix.
  • Updated release packaging and documentation. HIP/ROCm builds now include all quantized FlashAttention combinations, and the prebuilt binary and Docker image lists reflect the current release outputs.

macOS:

Linux:

Windows:

Docker:

  • CPU: docker pull ghcr.io/anbeeld/beellama.cpp:server-cpu-v0.3.1
  • CUDA: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda-v0.3.1
  • CUDA 12: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda12-v0.3.1
  • CUDA 13: docker pull ghcr.io/anbeeld/beellama.cpp:server-cuda13-v0.3.1
  • ROCm: docker pull ghcr.io/anbeeld/beellama.cpp:server-rocm-v0.3.1
  • Vulkan: docker pull ghcr.io/anbeeld/beellama.cpp:server-vulkan-v0.3.1
  • SYCL: docker pull ghcr.io/anbeeld/beellama.cpp:server-sycl-v0.3.1

Browse all container images

Loading