Battle of the Qwens: 27B Versus Custom Flash

The model I actually reach for on this Mac is the dependable one: Qwen 3.8 at 27 billion parameters. It finishes what I give it, and it runs on the machine in front of me rather than on someone else’s server.

Then I built a model of my own out of Qwen’s Flash-Next release — a smaller copy, with parts of it removed and the rest stored in fewer bits, so it would fit on a 64 GB machine. Flash-Next is a design built to be cheap to run for its size, and the reason I bothered is straightforward: if a cut-down version ran well here, the same everyday work would come back faster on the same hardware. I compared it with the 27B I already used.

It is quicker on the work both of them finish: across the six tasks they both completed, about a quarter faster. On the task where it did best, a policy call with information deliberately left out, it came back in twenty-five seconds against the 27B’s minute. Its finished writing held up too. The briefing and the customer-email rewrite both came back clean and scored a shade above the 27B’s, though with one reviewer and one attempt apiece I would not call that a ranking.

Twice in the objective set, on tasks that have exact, checkable answers, it produced no answer at all. It started repeating itself and kept writing until it hit the output ceiling: thirty-nine minutes on one, thirty-six on the other, nothing to show for either. It did the same thing once in the writing set, thirty-three minutes for an article that never arrived. The 27B finished everything I gave it.

Counting every objective attempt, including the two that produced nothing, my build spent eighty-three minutes where the 27B spent twelve. That is the number I would plan around, not the speed on the tasks it finished.

Everything on this page is frozen at what I measured on 17 September 2026, on one machine, one run per condition. The recommendation I keep current lives in the living guide, not here.

What I use each one for

Both of them do the same useful thing: structured work on material I supply, on the machine in front of me, with nothing sent anywhere. Notes I would rather not upload, a draft that is not public yet, a small table of figures I want turned into a memo. The 27B does that reliably. My build is faster on shared passing tasks, but its failed attempts took much longer.

Anything that has to come back with an answer goes to the 27B. Across this suite it produced the fewest surprises: every objective task answered, every writing assignment delivered, with one explainer that needed a factual correction afterwards. Swift’s 27B Q8 package is a close peer — it answered everything too, within about two and a half per cent of the same time in a single trial, which is far too small a gap to rank them — and I compare the 27B packages in their own article.

I use my Flash build for work where I supply the source material. Summarising notes I hand it, rewriting a draft I hand it: that is where it finished and scored well. For regular use, I would add a timeout and a second model as a fallback to limit time lost to repetition. One writing attempt in four cost me half an hour and returned nothing.

Neither of these runs tested the models the way a coding assistant works, where it can run the code, see the error and try again. That is a different comparison and I have not done it.

How I compared them

The two models, in plain terms

The two Qwen 3.8 builds use different architectures and quantization settings.

The 27B is a dense model: 27 billion parameters, all of them involved in every token, stored at 8 bits each. Q8 is that storage precision and has nothing to do with the parameter count. The package I run is Youssofal’s Optimized-Quality build, which I will call Youssofal Q8 MTPLX where the name matters; its sibling, the Optimized-Speed package at 4 bits, is Youssofal Q4 MTPLX and is not part of this comparison. MTPLX keeps the model loaded as a small local server, so a request does not pay to load it again.

Flash-Next is built differently. It is a mixture of experts: every layer holds a large bank of specialist sub-networks, and each token is routed to only ten of them. That is what makes it cheap to run for its size, and it is also what makes it prunable. My build keeps 288 of the 512 routed experts in each layer, following the selection sh0wie published, and stores most of the remaining weights in fewer bits, keeping more precision only where it matters most. It also carries a 29.8 GiB table of common token sequences — an n-gram table — that the runtime reads off the SSD instead of holding it all in memory. The build article is the construction story, and the exact recipe is at the bottom of this page.

That 288-of-512 selection has a name, REAP-288, and there is already a published build using it, from 30cmtonyfaker, stored at a coarser precision than mine. It is a different model and I am not comparing against it here.

Both models ran with multi-token prediction switched on: a small extra head proposes several tokens at once and the main model verifies them, which can raise the decode rate — the number of tokens produced per second of decoding.

How I ran them

Twelve prompts, the same twelve for each model: eight objective tasks with checkable answers and four writing assignments. One attempt each, one trial per condition, with a total output allowance of 64,000 tokens per attempt. Reasoning and the final answer share that allowance, which matters later, because a model that thinks for too long has nothing left to answer with. No tools, no test feedback, no chance to see an error and try again. One prompt, one answer, graded afterwards.

Eight objective prompts is a small number — a single pass or fail moves the rate by 12.5 points — and the mix leans towards structured work. Five of the eight want JSON back: reconciling invoices, scheduling under constraints, simulating a cache with expiry and eviction, pulling fields out of a revised document, and making a policy call with information deliberately left out. Two are Python functions and one is a SQL query, all executed by the evaluator afterwards. Nothing here measures agentic coding, long documents, dialogue or work in another language.

The models used different runtime interfaces. The 27B answered over a small local server that stays loaded between requests; my build answered through the runtime’s direct path. Prompt rendering and sampling settings differ between those two routes, and I have not isolated how much of the timing gap belongs to that rather than to the models, so the head-to-head is best read as a comparison of two working setups rather than of two sets of weights.

I report time on failed attempts separately from time on passing tasks. Both contribute to how long I waited for the suite.

The two tasks it could not finish

The 27B answered all eight objective tasks. Mine answered six.

The two it missed are the two I would have bet on. One, intervals, asks you to merge each user’s overlapping time ranges; the other, batches, asks you to sort a dependency graph into layers that can run together. Both have exact solutions and both are graded by running the returned function against nine cases. The 27B wrote them in a minute and a quarter and a minute and a half. My build wrote for thirty-nine minutes on one and thirty-six on the other, used every one of its 64,000 tokens, and never produced a function at all.

What the output shows is repetition: text that stops advancing and cycles until the allowance runs out. I call it a loop because that is what it looks like on screen. I watched it happen; I have not established what causes it. The pruning, the quantization, the multi-token prediction, the reasoning setting or something else entirely could be behind it, and the retry runs designed to probe it — 64,000 tokens of thinking plus a separate 8,000-token answer allowance — are prepared and have not been run.

Both prompts have fixed solutions: a wrong function would have been graded wrong, not missing. Neither attempt returned a function for the evaluator to check at all.

One writing task also ended in repetition. Asked why a city stays warmer than the countryside after sunset, my build reasoned for a while, landed on the phrase “heated infrastructure” and then repeated it: from that point to the 64,000-token ceiling the output is that phrase and nothing else, with no final text at the end.

The time each one took

Task by task

On the six tasks both passed — invoice, schedule, sql, cache, extract and policy — my build was quicker on five and slower on one: invoice took it 130.35 s against the 27B’s 93.26 s. Its best margin was policy, 25.45 s against 59.58 s, which is 2.34× faster on that single task.

The geometric mean of my build’s time as a share of the 27B’s — the multiplicative average, where a halving and a doubling cancel — is 79.64%, and that is the 1.26× paired speed on the six tasks both models finished that I quoted at the top. The arithmetic mean of the same six ratios is 85.06%. They are different statistics, and neither includes the two tasks my build did not finish, so neither one is a speed for the suite.

Time spent on failed attempts

Including failures changes the total time substantially. My build’s objective suite took 4,990.94 s, which is 83.18 minutes: 487.85 s on the six tasks it passed and 4,503.09 s on the two it did not. The 27B took 743.10 s for all eight and lost nothing. The two that went missing are where the difference sits: the 27B returned intervals in 71.97 s and batches in 96.12 s, while my build spent 2,336.46 s and 2,166.63 s on the same two prompts and produced no function for either.

The failed attempts also account for most of the generated tokens. My build’s weighted decode rate across the eight tasks was 28.42 tok/s against the 27B’s 18.67, about one and a half times the token rate, and it generated 141,551 tokens against 13,661. Of those, 128,000 went into the two failed attempts. They were produced quickly and they returned no answer.

What happened with multi-token prediction off

Everything above has multi-token prediction switched on. The same build with it off, decoding one token at a time — condition matrix-02, the AR point in the map further down — passed 7 of 8, failing only batches, in 3,581.99 s. Turning it on raised the decode rate and did not reduce the failures or shorten the suite. I would not conclude from either run that multi-token prediction helps in general.

How well each one wrote

The four writing prompts ask for different things: a briefing built from notes I supplied, a 450–600-word explainer from the model’s own knowledge, a decision memo drawn from a small table with a misleading aggregate in it, and a rewrite of a hype-filled draft as a plain customer email. One assistant scored each finished answer against a weighted rubric out of 100.

The custom build completed the summary and email without flagged factual errors. My build scored 5 of 5 on factual faithfulness and coverage for the briefing, and 5 of 5 on factual preservation and reader usefulness for the email. The model judge rated both 27B answers 88/100; with one reviewer and one answer per task, I do not read a four- or five-point gap as a ranking.

The data memo contained an unsupported causal claim. My build’s conversion rates were correct, 5 of 5 on numerical accuracy, but it scored 3 of 5 on causal caution, and the reviewer recorded one moderate error, quoting this line:

The safer operational conclusion is that the checkout change did not improve conversion on either device and may have harmed it

The task notes say the advertising mix changed between the two periods and there was no control group, so nothing about the checkout’s effect follows either way. The reviewer’s replacement:

Conversion declined within both device groups; the checkout change’s causal effect remains unknown.

The 27B’s memo, rated 96/100 by the model judge, made the same calculations and left the causal question open.

The knowledge article exhausted its output allowance: 2,009.93 s and all 64,000 tokens of the allowance, with no final text at the end. The 27B’s article finished at 556 words and received 75/100 from the model judge, with two moderate factual errors flagged that would need a correction pass.

On the three tasks my build finished, its model-judge mean is 88.33/100 and the 27B’s on the same three is 90.67/100. The 27B’s four-task mean of 86.75/100 is a different denominator; setting 88.33/100 against it would turn a missing article into a win. On the same three tasks Swift scored 91.67/100 and the hosted Flash reference, run with a 64k reasoning allowance, 92.33/100.

The writing results have two limitations. The timings were recorded under elevated memory pressure, with swap reads and memory compression visible during the run; I have not measured how much that slowed anything, so the timings may be affected by an unmeasured slowdown. And the scores are one assistant’s rubric judgments, not human blind tests: my build’s answers were reviewed with its identity known, the earlier twenty with shuffled labels but prior exposure. A difference between 93/100 and 92/100 is not statistically meaningful.

What it takes up while it runs

The footprint explains the memory-pressure caveat. These are separate measurements of different things; do not add them up.

The combined 76.6 GiB on disk is larger than the machine’s memory. The build still runs because the n-gram table is SSD-backed rather than held in memory in full, and the removed experts are absent rather than paged. Loaded for text it holds about 46 GiB of the 64, and the recorded objective-run peak is just under 50 GiB. For the writing run I rebooted first and let the runtime use the device’s recommended working set of 51.840 GiB as the ceiling on memory it could lock down, instead of the 56 GiB it had asked to wire before the reboot; active memory sat around 46.5 GiB, per-task peaks stayed under 49 GiB, and pressure was still observed. Memory pressure can occur even when the model’s allocation is below physical RAM, so I make no zero-swap promise.

The 27B is much lighter: its Q8 package is 27.91 GiB on disk in the earlier command-line cohort’s records, a disk figure rather than a peak. The field guide explains the memory budget.

The rest of the field

The wider set of recorded conditions is shown below. Timing scopes differ between groups, so their results cannot be ranked directly.

The full specification

The two conditions, exactly

The 27B is condition matrix-16: the Youssofal Optimized-Quality package of Qwen 3.8 27B at Q8, served by MTPLX as a resident HTTP server, at medium reasoning effort, with multi-token prediction at depth 3 and 128k configured context.

My build is condition matrix-03, published as khuasar/Qwen3.8-Flash-Next-REAP-288-Q4g32-Q8-MTPLX. It runs through MTPLX’s native path in a patched checkout at medium effort, with multi-token prediction at depth 3 — the head proposes up to three tokens per step — and the 29.802 GiB n-gram table read from the SSD rather than held in memory.

Swift Qwen 3.8 27B Q8 (KrossKinetic) appears as a secondary comparator, condition matrix-15: 8 of 8 in 724.15 s against the 27B’s 743.10 s, a 2.55% gap in one trial, too small to rank them.

What was pruned, and at what precision

Flash-Next routes each token to ten of the routed experts in a layer. REAP-288 keeps 288 of the 512 in every layer, following the list sh0wie published and pinned; my build follows that selection rather than choosing its own. Expert weights are stored at 4 bits with one scale per 32 weights (Q4g32) and selected sensitive matrices at 8 bits, rebuilt from Qwen’s original BF16 checkpoint with no new training. Nobody in this comparison ran those full-precision weights on the Mac; they are the source the build was converted from, not a condition in the data. The published REAP-288 build from 30cmtonyfaker uses the same expert selection at Q4g64 — 4 bits with one scale per 64 weights — and is not one of the conditions compared here.

Who made what

The checkpoint and the multi-token prediction head are Qwen’s. The retained-expert selection is sh0wie’s. Native MTP, its acceptance algorithm, the SSD n-gram facility and the mixed Q4g32 plus selected-Q8 recipe are existing MTPLX work. I ran the conversion and modified the loader. No additional training was involved. The build article describes the conversion and runtime change.

The repository’s licence metadata is other, with the Qwen Community License 1.0 named in the model card alongside a link upstream. Anyone reusing this needs to check the actual upstream licence and its redistribution obligations for themselves; nothing here is legal advice.

Cohorts, timing scopes and what they mean

matrix-15 and matrix-16 are the current medium comparison, both served over resident HTTP. matrix-02 and matrix-03 — my build without and with multi-token prediction — sit in the historical, workflow-comparison cohort and run through the native path, as does the Gemma condition matrix-11, a fresh command including model load and a much smaller output budget. matrix-17 to matrix-19 are hosted runs whose times include network and queue. The cohorts are not pooled. And because the 27B ran over an HTTP server while my build ran through the native path, prompt rendering, draft sampling and other settings differ between the two arms, which is why I call the head-to-head a workflow comparison rather than an isolated model ranking.

How the writing was scored

One assistant scored each finished answer against a weighted rubric out of 100, with dimensions set per task type: factual faithfulness and coverage for the briefing, factual accuracy and clarity for the explainer, numerical accuracy and causal caution for the memo, factual preservation and reader usefulness for the rewrite. The method is not uniform across the set. My build’s four answers were reviewed with the model’s identity known; the earlier twenty were reviewed with shuffled identity labels and prior exposure to the material. There was no independent human blind review, and an answer with no final text is unscored rather than scored zero.

The limits of this comparison

What would change my mind

I deliberately let the measured runs continue up to the full output allowance, without an earlier reasoning cutoff or a corrective prompt, to see whether the model would eventually return an answer. Next week I plan to test a harness that limits reasoning, detects repetition and tells the model it is looping before asking it to try again. That might prompt it to self-correct; it could also fail again. The test needs to count completion and all time spent on retries. These are proposed tests, not results.

The following tests could change my recommendation in the living guide:

1. Repeated timings under matched conditions and normal memory pressure, so the 1.26× paired speed on the six tasks both models finished is confirmed or dissolves into trial noise.
2. Held-out writing and structured tasks neither model has seen; repeated use of these twelve prompts is development exposure, not a clean test.
3. The same harness with a bounded repair loop for both models, so the cost of my build’s loops and any recovery are measured.
4. An ablation of the fixed total-budget guard: do separate reasoning and answer allowances change my build’s completion rate?
5. More independent prompts, so one pass is worth less than 12.5 points.

None of these has been run. The 64k-thinking-plus-8k-answer retries remain paused, and nothing pending is reported here as a result.

Public sources