'~6 GB of VRAM' group-offloading mode has an undocumented ~85 GB system-RAM requirement (OOM-killed on standard cloud pods) #141

Summary

The README advertises low-VRAM group offloading ("requires ~6GB of VRAM"). The VRAM claim is
accurate — we measured 4.93 GB peak. But --enable_low_vram_mode hardcodes
use_stream=True, record_stream=True (infer_helios.py:298-306), which makes diffusers pin a
CPU copy of the weights in addition to the ~40 GB resident bf16 pipeline. Peak host RAM is
therefore ~80–85 GB, which is never mentioned. On a standard RunPod RTX 4090 pod the
container cgroup cap is 46 GB → the process is OOM-killed by the kernel after a
successful model load, with no Python traceback — just Killed.

Environment

• Helios @ 8f2a2fa, torch 2.10.0+cu128, diffusers 0.39.0, Helios-Distilled bf16
• Fail case: RunPod RTX 4090 pod (24 GB VRAM, cgroup memory.max = 45,999,996,928 ≈ 46 GB;
  note free misleadingly shows the 187 GB host)
• Pass case: 2×RTX 4090 pod (cgroup 123 GB): completes with 4.93 GB peak VRAM

Reproduction

# on a pod/container with a ~46 GB memory cgroup:
python infer_helios.py --base_model_path BestWishYsh/Helios-Distilled \
  --transformer_path BestWishYsh/Helios-Distilled --sample_type t2v --num_frames 33 \
  --guidance_scale 1.0 --is_enable_stage2 --pyramid_num_inference_steps_list 2 2 2 \
  --is_amplify_first_chunk --enable_low_vram_mode --group_offloading_type leaf_level

Expected

Generation with ~6 GB VRAM on modest hardware (the feature's advertised purpose).

Actual

Loading pipeline components...: 100%|██████████| 5/5
bash: line 1:  2314 Killed                  python infer_helios.py …

(kernel OOM kill during offload setup — pinned-buffer allocation exceeds the cgroup.)
On a 123 GB-RAM pod the same command succeeds: 33 frames, max VRAM 4.93 GB.

Analysis

pipe.enable_group_offload(..., use_stream=True, record_stream=True) pre-pins CPU tensors for
async H2D streaming. For a 14B bf16 pipeline that's ~40 GB pinned on top of the ~40 GB module
copy (plus fp32 VAE and working memory). diffusers offers low_cpu_mem_usage=True (lazy
per-group pinning, slower) precisely for RAM-constrained hosts, but infer_helios.py doesn't
expose it.

Suggested fix

1. README: qualify the claim — "~6 GB VRAM (requires ≈2× model size in system RAM, ~85 GB for
   the 14B pipeline, when streams are enabled)".
2. Optional: add --offload_low_cpu_mem_usage passing low_cpu_mem_usage=True to
   enable_group_offload, trading speed for a much smaller RAM peak.

Impact

The users this feature targets (small-GPU machines) are disproportionately likely to have
<85 GB RAM; the failure mode is a bare Killed with no guidance. Measured evidence available
(cgroup values, VRAM telemetry) on request.