> ./infer --prompt "Explain quantum computing" --tokens 100                                                                             Sun Mar 22 14:11:31 2026
[metal] Device: Apple M5 Pro
[metal] Shader compile: 2 ms
[metal] GPU attention buffers: 15 KV caches (16.8 MB each), scores buf 134.2 MB
[metal] Delta-net GPU buffers: 45 layers (195.4 MB state + 0.2 MB scratch)
[metal] Inference pipelines ready (multi-expert[8] + shared buffers allocated)
=== Qwen3.5-397B-A17B Metal Inference Engine ===
Model:    /Users/redacted/.cache/huggingface/hub/models--mlx-community--Qwen3.5-397B-A17B-4bit/snapshots/39159bd8aa74f5c8446d2b2dc584f62bb51cb0d3
Weights:  model_weights.bin
Manifest: model_weights.json
Vocab:    vocab.bin
K:        4 experts/layer
Quant:    4-bit experts (7077888 bytes each)
Linear:   fused GPU delta-net
Tokens:   100
Cache:    0 entries (disabled)
[manifest] Loaded 2092 tensors from model_weights.json
[weights] mmap'd 5.52 GB from model_weights.bin
[metal] Weight file wrapped as Metal buffer (5.52 GB)
[vocab] Loaded 248077 tokens
bpe_load: 248044 vocab, 247587 merges, 33 added tokens
Tokens (4): [814, 20139, 29144, 23470]
[prompt] 4 tokens: 814 20139 29144 23470
[experts] 60/60 packed layer files available (mmap'd)
[tiered-io] Cold fds (F_NOCACHE) + warm fds (page cached) active
[warmup] Page cache hint: 5.2 ms
[init] Setup: 69.8 ms

--- Generating 100 tokens ---
  [prefill] batch embed 4 tokens: 1.2 ms
[cache] Pre-computed weight pointers for 60 layers
  [prefill] 3/4 tokens: 1707 ms (first: 1220 ms, rest avg: 244 ms)
[debug] Top 5 logits (next_token=303):
  token 1040 ("Ġlike") logit=17.6052
  token 303 ("Ġin") logit=20.5380
  token 248046 ("<|im_end|>") logit=17.1903
  token 310 ("Ġto") logit=19.7185
  token 430 ("Ġas") logit=17.5620
[debug] hidden rms after final_norm=1.7007, logits rms=2.2827
[ttft] 2010 ms (prefill 4 tokens + lm_head 3 ms)

--- Output ---
ĠinĠsimple  [gen 1/100] token_id=4145 (214 ms, 4.67 tok/s)
Ġterms  [gen 2/100] token_id=3665 (184 ms, 5.44 tok/s)
.  [gen 3/100] token_id=13 (184 ms, 5.42 tok/s)
ĊĊ  [gen 4/100] token_id=271 (180 ms, 5.55 tok/s)
<think>  [gen 5/100] token_id=248068 (173 ms, 5.77 tok/s)
ĊĊ  [gen 6/100] token_id=271 (164 ms, 6.11 tok/s)
</think>  [gen 7/100] token_id=248069 (150 ms, 6.66 tok/s)
ĊĊ  [gen 8/100] token_id=271 (152 ms, 6.59 tok/s)
Here  [gen 9/100] token_id=8160 (141 ms, 7.10 tok/s)
Ġis  [gen 10/100] token_id=369 (168 ms, 5.95 tok/s)
Ġan  [gen 11/100] token_id=449 (186 ms, 5.39 tok/s)
Ġexplanation  [gen 12/100] token_id=15673 (185 ms, 5.40 tok/s)
Ġof  [gen 13/100] token_id=314 (182 ms, 5.50 tok/s)
Ġquantum  [gen 14/100] token_id=29144 (159 ms, 6.31 tok/s)
Ġcomputing  [gen 15/100] token_id=23470 (180 ms, 5.56 tok/s)
Ġin  [gen 16/100] token_id=303 (156 ms, 6.43 tok/s)
Ġsimple  [gen 17/100] token_id=4145 (142 ms, 7.06 tok/s)
Ġterms  [gen 18/100] token_id=3665 (137 ms, 7.32 tok/s)
,  [gen 19/100] token_id=11 (135 ms, 7.42 tok/s)
Ġbreaking  [gen 20/100] token_id=14293 (182 ms, 5.50 tok/s)
Ġdown  [gen 21/100] token_id=1441 (184 ms, 5.42 tok/s)
Ġthe  [gen 22/100] token_id=279 (147 ms, 6.80 tok/s)
Ġcomplex  [gen 23/100] token_id=6150 (180 ms, 5.55 tok/s)
Ġj  [gen 24/100] token_id=492 (165 ms, 6.05 tok/s)
argon  [gen 25/100] token_id=68421 (189 ms, 5.29 tok/s)
Ġinto  [gen 26/100] token_id=1083 (135 ms, 7.39 tok/s)
Ġeasy  [gen 27/100] token_id=3999 (150 ms, 6.65 tok/s)
-to  [gen 28/100] token_id=4534 (167 ms, 6.00 tok/s)
-under  [gen 29/100] token_id=68695 (178 ms, 5.63 tok/s)
stand  [gen 30/100] token_id=2598 (145 ms, 6.88 tok/s)
Ġconcepts  [gen 31/100] token_id=18369 (136 ms, 7.34 tok/s)
.  [gen 32/100] token_id=13 (154 ms, 6.50 tok/s)
ĊĊ  [gen 33/100] token_id=271 (137 ms, 7.27 tok/s)
###  [gen 34/100] token_id=13962 (127 ms, 7.88 tok/s)
Ġ  [gen 35/100] token_id=220 (127 ms, 7.90 tok/s)
1  [gen 36/100] token_id=16 (152 ms, 6.58 tok/s)
.  [gen 37/100] token_id=13 (151 ms, 6.61 tok/s)
ĠThe  [gen 38/100] token_id=561 (124 ms, 8.07 tok/s)
ĠBasic  [gen 39/100] token_id=14204 (134 ms, 7.48 tok/s)
ĠDifference  [gen 40/100] token_id=53230 (129 ms, 7.76 tok/s)
:  [gen 41/100] token_id=25 (148 ms, 6.75 tok/s)
ĠSwitch  [gen 42/100] token_id=15129 (136 ms, 7.37 tok/s)
es  [gen 43/100] token_id=287 (163 ms, 6.15 tok/s)
Ġvs  [gen 44/100] token_id=5972 (133 ms, 7.51 tok/s)
.  [gen 45/100] token_id=13 (162 ms, 6.17 tok/s)
ĠSp  [gen 46/100] token_id=2989 (146 ms, 6.86 tok/s)
inning  [gen 47/100] token_id=19167 (166 ms, 6.01 tok/s)
ĠCoins  [gen 48/100] token_id=59692 (130 ms, 7.69 tok/s)
Ċ  [gen 49/100] token_id=198 (164 ms, 6.11 tok/s)
To  [gen 50/100] token_id=1206 (132 ms, 7.56 tok/s)
Ġunderstand  [gen 51/100] token_id=3418 (154 ms, 6.50 tok/s)
Ġquantum  [gen 52/100] token_id=29144 (150 ms, 6.68 tok/s)
Ġcomputing  [gen 53/100] token_id=23470 (173 ms, 5.78 tok/s)
,  [gen 54/100] token_id=11 (167 ms, 5.98 tok/s)
Ġfirst  [gen 55/100] token_id=1118 (172 ms, 5.82 tok/s)
Ġthink  [gen 56/100] token_id=1683 (166 ms, 6.01 tok/s)
Ġabout  [gen 57/100] token_id=883 (164 ms, 6.09 tok/s)
Ġhow  [gen 58/100] token_id=1204 (138 ms, 7.27 tok/s)
Ġyour  [gen 59/100] token_id=678 (157 ms, 6.36 tok/s)
Ġcurrent  [gen 60/100] token_id=1428 (170 ms, 5.88 tok/s)
Ġcomputer  [gen 61/100] token_id=6165 (146 ms, 6.87 tok/s)
Ġ(  [gen 62/100] token_id=318 (161 ms, 6.19 tok/s)
a  [gen 63/100] token_id=64 (168 ms, 5.95 tok/s)
Ġclassical  [gen 64/100] token_id=27882 (148 ms, 6.74 tok/s)
Ġcomputer  [gen 65/100] token_id=6165 (144 ms, 6.93 tok/s)
)  [gen 66/100] token_id=8 (154 ms, 6.50 tok/s)
Ġworks  [gen 67/100] token_id=4138 (152 ms, 6.57 tok/s)
.  [gen 68/100] token_id=13 (151 ms, 6.63 tok/s)
Ċ  [gen 69/100] token_id=198 (159 ms, 6.30 tok/s)
*  [gen 70/100] token_id=9 (188 ms, 5.33 tok/s)
ĠĠ  [gen 71/100] token_id=256 (160 ms, 6.24 tok/s)
Ġ**  [gen 72/100] token_id=2972 (128 ms, 7.79 tok/s)
Class  [gen 73/100] token_id=1890 (127 ms, 7.85 tok/s)
ical  [gen 74/100] token_id=908 (131 ms, 7.63 tok/s)
ĠComputers  [gen 75/100] token_id=66133 (130 ms, 7.72 tok/s)
:**  [gen 76/100] token_id=64700 (123 ms, 8.10 tok/s)
ĠImagine  [gen 77/100] token_id=36541 (144 ms, 6.97 tok/s)
Ġa  [gen 78/100] token_id=264 (123 ms, 8.14 tok/s)
Ġlight  [gen 79/100] token_id=3000 (136 ms, 7.34 tok/s)
Ġswitch  [gen 80/100] token_id=3289 (133 ms, 7.49 tok/s)
.  [gen 81/100] token_id=13 (151 ms, 6.62 tok/s)
ĠIt  [gen 82/100] token_id=1049 (118 ms, 8.45 tok/s)
Ġcan  [gen 83/100] token_id=628 (151 ms, 6.61 tok/s)
Ġbe  [gen 84/100] token_id=381 (158 ms, 6.33 tok/s)
Ġeither  [gen 85/100] token_id=2892 (149 ms, 6.71 tok/s)
Ġ**  [gen 86/100] token_id=2972 (153 ms, 6.54 tok/s)
ON  [gen 87/100] token_id=691 (159 ms, 6.28 tok/s)
**  [gen 88/100] token_id=332 (158 ms, 6.34 tok/s)
Ġ(  [gen 89/100] token_id=318 (152 ms, 6.59 tok/s)
1  [gen 90/100] token_id=16 (134 ms, 7.49 tok/s)
)  [gen 91/100] token_id=8 (156 ms, 6.39 tok/s)
Ġor  [gen 92/100] token_id=466 (129 ms, 7.77 tok/s)
Ġ**  [gen 93/100] token_id=2972 (164 ms, 6.12 tok/s)
OFF  [gen 94/100] token_id=26200 (143 ms, 7.00 tok/s)
**  [gen 95/100] token_id=332 (135 ms, 7.41 tok/s)
Ġ(  [gen 96/100] token_id=318 (143 ms, 6.97 tok/s)
0  [gen 97/100] token_id=15 (148 ms, 6.77 tok/s)
).  [gen 98/100] token_id=553 (138 ms, 7.24 tok/s)
ĠEvery  [gen 99/100] token_id=6983 (123 ms, 8.14 tok/s)


--- Statistics ---
Total time:     17.1 s
TTFT:           2010 ms
Tokens:         100 generated
Generation:     15.1 s (6.55 tok/s)
Config:         K=4 experts, 60 layers