6 Aug 2026, 10:56 UTC56 views1 reactionsread 8 August 2026 🔬 reasoning on/off on a 2.6B model
I benchmarked LFM2.5-2.6B on native tool-use: 30 tasks, reasoning on vs off, one RTX 5090, greedy.
first pass: 96.7% both legs, McNemar p=1.0. a perfect null. except the "off" leg still logged ~700 reasoning tokens per task, which should have been near zero.
the cause: LFM2.5's GGUF chat template hard-forces <think> on every assistant turn and reads none of enable_thinking / reas…
👍1
Signed witcheer
5 Aug 2026, 05:31 UTC76 views2 reactionsread 8 August 2026 real NVFP4 on a 5090: Blackwell's 4-bit float lands on the frontier
NVFP4 is NVIDIA's native FP4 format for Blackwell. I already measured the honest K-quant frontier of Qwen3.6-27B (the quant-tax ladder), so a real NVFP4 GGUF of the same base drops straight onto it as one more level, same harness (MMLU/GSM8K/HumanEval, greedy, think-off), one RTX 5090.
it lands on the frontier. composite 92.11, between Q5 and Q6, w…
❤2
Signed witcheer
4 Aug 2026, 07:49 UTC86 views8 reactionsread 8 August 2026 the quantization tax on Qwen3.6-27B, measured side by side
the folk rule is: run the biggest GGUF quant that fits your VRAM, lower bits means a worse model. I ran the whole K-quant ladder (Q8_0 down to Q3_K_M) on one RTX 5090, same model and same harness every rung, scoring MMLU + GSM8K + HumanEval alongside decode speed and file size.
down to Q4 the tax is inside the noise:
- Q8_0: 92.3 quality, 52.8 tok/s, 29.0 …
❤8
Signed witcheer
2 Aug 2026, 05:48 UTC105 views10 reactionsread 8 August 2026 touched grass a bit end of July, coming back tomorrow with new benchmark, and tests!
I’m currently eyeing on 1 or 2 DGX Sparks, need to figure out the investment but it seems more and more worth it since the latest Deepseek release
👍5🔥4👏1
Signed witcheer
22 Jul 2026, 18:10 UTC167 views4 reactionsread 8 August 2026 fourteen nights of a benchmark that is supposed to find nothing
since 9 july a cron job on the 5090 wakes at 06:45, clones llama.cpp master fresh, builds it for sm_120, runs a smoke benchmark, and goes back to sleep. nobody watches it.
fourteen runs so far, across thirteen distinct upstream commits. every one built. every one passed the smoke test.
mean 888.22 tok/s
range 886.76 - 890.08 (3.32 tok/s, 0.374%…
👍3🔥1
Signed witcheer
17 Jul 2026, 06:09 UTC178 viewsread 8 August 2026 Photo
Photo, posted without a caption
Signed witcheer
17 Jul 2026, 06:08 UTC164 views5 reactionsread 8 August 2026 drafter training, epoch end!
8 nights, 54,000 steps, ~55 GPU-hours, every night rc=0, one RTX 5090. epoch 1 of the first EAGLE-3 draft head for Hermes-4.3-36B (Seed-OSS) is complete, and the nightly smoke retires. the release bench takes over: cuda graphs on, 3 configs, 4 workloads, 8 fixed prompts each, baseline 66 tok/s everywhere.
• tree-3-4-8: 1.38x prose, 1.38x code, 1.57x repetitive, 1.29x chat. wins every wo…
👍4🔥1
Signed witcheer
14 Jul 2026, 09:16 UTC154 views4 reactionsread 8 August 2026 nvfp4 on consumer blackwell, round 2: Unsloth's Qwen3.6-27B on one RTX 5090
the model card says "2.5x faster" (no baseline stated) and "runs on 24GB". measured on vLLM 0.21.0, sm_120.
~~~
three walls before the first token:
1. the checkpoint quantises lm_head to FP8; vLLM builds lm_head unquantised and dies on the orphan scale. open upstream, reproduced on v0.22: github.com/vllm-project/vllm/issues/44081. fix: de…
❤3🤝1
Signed witcheer
13 Jul 2026, 07:57 UTC129 views2 reactionsread 8 August 2026 Photo
2-bit KV cache in july 2026: your 4090 can run it, your 5090 cannot.
OSCAR (arXiv 2605.17757) stores most of the KV cache at 2 bits, keeps a small bf16 sink and recent window, and hands back ~8x of your KV memory. I spent today trying to run it on the RTX 5090.
the gap is kernels, not architecture. sglang's INT2 prefill calls fa3, built for sm80/86/89/90a: A100, 3090, 4090, H100. the 5090 (sm_120) is not on the lis…
👍2
Signed witcheer
9 Jul 2026, 10:18 UTC142 views2 reactionsread 8 August 2026 Photo
Photo, posted without a caption
👍2
Signed witcheer
9 Jul 2026, 10:18 UTC134 views1 reactionsread 8 August 2026 training a speculative-decoding draft head for Hermes-4.3-36B on one 32GB card
EAGLE-3 drafts train on the teacher's hidden states: 3 aux layers + the final one, ~41KB per token at hidden 5120. the standard offline recipe caches those to disk, and 54K samples comes to ~4.5TB. I have 570GB free, so offline is dead on this box.
online mode recomputes activations per batch instead: zero storage, teacher resident in VR…
💯1
Signed witcheer
9 Jul 2026, 06:46 UTC109 views1 reactionsread 8 August 2026 the last modality landed: video with synchronized audio, on one consumer card.
LTX-2.3 is Lightricks' ~19B dual-stream DiT: 14B generates video latents, 5B generates audio latents alongside them, so the sound matches the motion. one pass, one mp4 out (h264 + AAC 48kHz). the "22B" in the name is marketing.
I ran the distilled two-stage pipeline (8 steps at half resolution, then a 2x upsample + 3 refine steps) on one…
❤1
Signed witcheer
Showing the 12 most recent of 20 posts we hold for @witcheergrimoire. View and reaction counts are the latest single reading for each post, not a live figure, and a recent post is still accumulating both. A view count marked ≈ was rounded by Telegram before we ever saw it — t.me prints views in full below 1,000 and to three significant figures above, so ≈1,200,000 means somewhere between 1,150,000 and 1,249,999. Unmarked counts are exact. Text is reproduced from the public post preview and truncated for length.