LLM Benchmarks — Minisforum MS-S1 Max (AMD Strix Halo)

2026-10-25 · halogen flash server 0.18.15 · qwen3.8-flash-next (62 GB, 262k context, 4 slots) · benchmark tool: halogen-bench.py, 2 reps, MTP drafter, effort low

Hardware: Minisforum MS-S1 Max, AMD Strix Halo APU, 128 GB unified memory at 8000 MT/s. Fixed settings across all runs: Linux 7.0.0-38, tuned llm-power profile (powersave governor + balance_performance EPP), same model and server config.

How the benchmark works

The tool is halogen-bench.py, shipped with the halogen server. It runs a llama-bench-shaped sweep against the live HTTP API — every number below is end-to-end over HTTP (chat template + tokenizer + engine), not a raw engine measurement.

Prefill tests (pp1024 / pp8192 / pp32768): builds a filler prompt sized to hit the target token count (calibrated against the server's own tokenizer), sends it with max_tokens=1, and reports prompt_tokens ÷ wall-clock. This is how fast the model reads your prompt — the number that matters for long documents and chat histories.
Decode tests (tg128 / tg512): sends real prompts of different shapes (prose, code, etc. from the server's eval prompt set), generates 128 or 512 tokens, and reports completion_tokens ÷ wall-clock, averaged over the prompt shapes. Because the request's own prefill is included and short generations are dominated by first-token latency, these run lower and noisier than prefill — the stdevs of ±11–15 are expected, not instability.
Settings used: 2 repetitions per cell, temperature 0.0, thinking disabled, reasoning_effort low, MTP drafter (multi-token prediction — the server's speculative decoding). Mean ± stdev across reps.
Idle power: AMD package power from /sys/class/hwmon (power1_average), sampled after ≥30 s of no requests, model loaded and serving.

Each run below was executed cold against the same running server; the box was otherwise idle (verified via server slot telemetry — no concurrent inference during any measurement).

The question: does IOMMU cost LLM performance?

Community forums claim disabling IOMMU in the BIOS is worth ~7% on LLM inference. This box started with IOMMU enabled in BIOS plus the Linux kernel parameter iommu=pt (passthrough).

Head-to-head: `iommu=pt` vs disabled in BIOS

Runs C and D differ in exactly one variable (IOMMU state); everything else — BIOS power profile, kernel, drivers, model config — identical.

Test	IOMMU disabled (Run C)	IOMMU on + `iommu=pt` (Run D)	Cost of leaving it on
Prefill 1k	1600.4 ± 6.4	1569.3 ± 9.4	−1.9%
Prefill 8k	2082.2 ± 3.2	1994.6 ± 7.4	−4.2%
Prefill 32k	2077.1 ± 23.0	1987.1 ± 21.2	−4.3%
Decode 128	71.6 ± 11.1	69.5 ± 11.0	−2.9%
Decode 512	75.7 ± 14.5	73.3 ± 14.5	−3.2%

tok/s, mean ± stdev, 2 reps. Stdevs are small enough that the prefill gaps are real, not noise.

Verdict: leaving IOMMU on (in passthrough mode) costs ~4% prefill on long prompts and ~3% decode — about half the forum's claimed 7%, and the 7% figure likely comes from tests against full translation mode (IOMMU on, no pt), which this box never ran. The cost is real but small; whether it's worth paying depends on whether you need IOMMU for anything else (device passthrough to VMs, shared virtual addressing for the NPU, stronger DMA isolation).

Results — all four runs

All numbers are tokens/s, mean ± stdev over 2 reps. Δ = % vs Run A (baseline).

Run A — baseline

IOMMU enabled in BIOS + iommu=pt, BIOS power profile: Performance

Test	tok/s	± stdev
Prefill 1k	1595.8	3.5
Prefill 8k	2071.3	6.2
Prefill 32k	2097.8	2.4
Decode 128	69.7	11.1
Decode 512	73.7	14.6

Idle package power: ~14 W

Run B — IOMMU off + BIOS Balanced

IOMMU disabled in BIOS, BIOS power profile changed to Balanced

Test	tok/s	± stdev	Δ vs A
Prefill 1k	1595.0	5.8	−0.1%
Prefill 8k	2074.0	4.5	+0.1%
Prefill 32k	2066.6	24.9	−1.5%
Decode 128	71.4	11.3	+2.4%
Decode 512	75.5	14.6	+2.4%

Idle package power: 8–12 W

Note: Run B looks flat vs A because two changes cancelled out — IOMMU off (gain) and Performance→Balanced power profile (small loss). The isolated IOMMU effect is in the C vs D table above.

Run C — cleanup

Run B plus removal of unused hardware drivers and background cruft

Test	tok/s	± stdev	Δ vs A
Prefill 1k	1600.4	6.4	+0.3%
Prefill 8k	2082.2	3.2	+0.5%
Prefill 32k	2077.1	23.0	−1.0%
Decode 128	71.6	11.1	+2.7%
Decode 512	75.7	14.5	+2.7%

Idle package power: ~6 W

Run D — IOMMU back on

Same as Run C, IOMMU re-enabled in BIOS (only variable changed)

Test	tok/s	± stdev	Δ vs A	Δ vs C
Prefill 1k	1569.3	9.4	−1.7%	−1.9%
Prefill 8k	1994.6	7.4	−3.7%	−4.2%
Prefill 32k	1987.1	21.2	−5.3%	−4.3%
Decode 128	69.5	11.0	−0.3%	−2.9%
Decode 512	73.3	14.5	−0.5%	−3.2%

Idle package power: ~5 W

Summary

Metric	A → B	B → C	C → D (IOMMU back on)
Prefill (8k/32k)	~0%	~0%	−4%
Decode	+2.4%	~0%	−3%
Idle power	14 → 10 W	10 → 6 W	6 → 5 W

Findings:

IOMMU on (with iommu=pt) costs ~4% prefill / ~3% decode vs disabled in BIOS. The "7%" forum claim was directionally right but overstated for a passthrough host.
BIOS Balanced power profile is free performance — no measurable cost, lower idle draw.
Decode stdevs are wide (±11–15) because short generations are dominated by first-token latency jitter; prefill numbers are the reliable signal.
Bench hygiene verified: server slot telemetry confirms no concurrent inference overlapped any measurement.

#sb-root {
  --editor-font: "Segoe UI" !important;
  --ui-font: "Segoe UI" !important;
  --editor-width: full !important;
}