How I Made Qwen Flash-Next Fit on a 64 GB MacBook Pro

I wanted Qwen’s Flash-Next resident on my own machine rather than reached over a network. A model that lives on the laptop has nothing to sign into and costs nothing per request, the prompt and the answer stay where they are, and it still answers when there is no connection. Flash-Next was the model I most wanted to run locally, but it needed changes to fit.

The files come to about 77 GiB, and the MacBook Pro has 64 GiB of unified memory, shared by the CPU, the GPU and macOS. A model that size does not simply load. The exact payload figures are in the memory table further down.

Three moves made it fit. I cut the model’s bank of routed experts — small sub-networks, a few of which a router picks for every token — from 512 to 288 in every layer, following a selection sh0wie published before this build. I mixed the precision instead of using one setting throughout, so the parts judged most sensitive to it keep more bits than the rest. And I left the largest single file, a lookup table of nearly 30 GiB, on the SSD and read it from there rather than loading it at all. With the text model up, the MLX allocation — the memory the inference framework actually holds — is 45.980 GiB, about 46 of the machine’s 64.

Of the eight reasoning prompts I use — I call them the objective tasks below, because each one has a checkable right answer — it finished six, and those six took about eight minutes between them. They are ordinary working tasks: an invoice’s totals in the right order, a job schedule across two machines, a SQL query, a cache’s state, records pulled out of a messy document, and a set of eligibility rules applied to a case. It drafted prose as well, on three of the four writing prompts.

The other two reasoning prompts, and the fourth writing prompt, ran their whole token allowance out and produced no final answer at all. Including those failed attempts, the eight reasoning attempts together took about an hour and twenty-three minutes. The hosted Flash endpoint, run by Alibaba on their own hardware, finished all eight, and it came back faster than my machine on every one of the six that both of them finished.

On the failed tasks, the process kept running. Fluent generation at full speed can look like progress, but here it continued until the allowance was gone, leaving no final answer. The worst of the eight ran for about thirty-nine minutes and produced no answer. Both of the reasoning tasks it gave up on ask for a working Python function. For unattended use, I would add a timeout.

The build runs Flash-Next natively on a 64 GB Mac with multi-token prediction (MTP): a small extra network guesses the next few tokens and the main model checks them in one pass, which is the feature the whole build exists to keep. It needs a modified copy of MTPLX — the software that runs the model on the Mac — rather than a released version, and some requests end in repetition without an answer. The dense Qwen 3.8 27B in Youssofal’s Optimized-Quality Q8 package finished every task, with Swift’s Q8 a close peer; my recommendations are in the living guide, and the task-by-task trade-off is in Battle of the Qwens.

This is not training: no weight was updated, and no new expert selection was calibrated. Qwen trained the model, sh0wie worked out which experts could go, and the runtime features I leaned on were already in MTPLX; my part was integration, and the full credit is further down. The hosted endpoint finished more of my tasks than this build did, so nothing here establishes that the build preserves the full model’s quality. Every figure here is frozen at the 17 September 2026 snapshot.

How I did it

Cutting the expert bank

Flash-Next is a mixture-of-experts model: each layer holds a bank of small feed-forward networks, the experts, and a router picks a few of them for every token. Pruning removes the experts with the lowest measured contributions on the calibration workload. REAP is the method for doing it, published by Cerebras Research, and sh0wie had already applied it to Flash-Next and published which 288 of the 512 experts to keep in each layer; I used that selection as published, pinned to one revision.

I did not try to work out my own list. Calibrating a new selection is a project of its own, and there was a published one I could check against its own statistics before trusting it.

The cut itself happened here, in the conversion: the experts that are not on the list are absent from the files this build produced, not switched off at runtime. Each token still activates the top 10 experts, and the shared experts were left alone. The counts, the scoring rule and my audit of the selection are further down.

Mixing the precision

Quantization stores each weight in fewer bits than full precision. Uniform four-bit everywhere would have been simpler and smaller. Instead I made the routed expert projections four-bit and kept the pieces judged sensitive to precision at eight bits, component by component as the recipe further down lists. I rebuilt everything from the original full-precision BF16 checkpoint Qwen published, so each quantized tensor was produced once, directly from those weights, rather than converted out of somebody else’s four-bit pack. That avoids losing more than the recipe intends. It does not make this a higher-quality model than a plainer pack, and no amount of rebuilding brings back an expert that pruning removed.

Doing that on a laptop meant rebuilding in a stream: one source shard at a time, verified against the pinned manifest before it was read, so the full-precision checkpoint was never all on the machine at once and the job could be picked up again if it stopped. The whole sequence is in the specification.

Leaving the biggest file on the SSD

Flash-Next carries an n-gram embedding table: a large lookup keyed by short token sequences that the model consults while it generates. In this build it is 29.802 GiB after quantization, and I never load it into memory. Lookups reach the SSD-backed pages through a hot cache and prefetch. Reading a table that way is MTPLX’s own facility; all my build had to do was write the table in the form it expects.

The word “offload” gets used for three different things, and only one of them is what happens here. This is not macOS swap, the operating system moving memory pages to disk under pressure; swap happened separately during the writing run, described below. It is not expert offload either, keeping experts on disk and paging them in on demand. And it does not bring back the 224 removed experts per layer; they are absent from the files. File-backed pages still cost memory and I/O when touched, so the 29.802 GiB is not free either.

The one thing I had to change in the runtime

The pruned main model — the trunk — has 288 experts per layer. The multi-token-prediction head, taken from the same pinned Qwen checkpoint, still has 512. MTPLX as released builds the head with the trunk’s expert count, so the head’s weights do not fit and the load fails. The change is one optional configuration field that gives the head its own count. The generation and SSD-table features were already in MTPLX; the specification lists the existing features this build uses.

The head was never retrained against the pruned trunk, which now feeds it different hidden states, so whether it still proposes useful tokens was a question for measurement rather than reasoning. And the build only loads on the modified copy of MTPLX documented in the repository. I have not shown that it loads on every released version, and I have never validated an installation from the patch on a clean machine, which is why this article gives no install commands.

Keeping the weights and configuration consistent

Applying the published pruning selection and precision recipe requires keeping the weights, router and configuration consistent. The same expert IDs have to go, in the same order, to the expert weights and to the matching router rows, and the build has to refuse to load rather than load something half-finished. The head needs a separate expert count, supplied by the optional configuration field.

Strict weight loading is kept, so the load either works or fails outright; there is no half-loaded state to debug. A successful load does not establish whether the model will finish a task.

What fitting it cost

The cost shows up in two places: what the machine had to hold while the model worked, and what came back finished.

Does it actually fit?

To judge whether the build leaves enough room for your work, distinguish its disk size from its loaded allocation and its peak allocation. Prefill is reading a whole prompt in before generation starts. The measurements below show which part of the footprint each number describes.

The writing run is where pressure showed. I rebooted the machine before it, and the device then reported a recommended working set of 51.840 GiB, which the run used as its wired-limit ceiling — the most memory it was allowed to keep resident — lower than the 56 GiB it had asked to wire before the reboot. Active allocation was about 46.5 GiB and recorded per-task peaks stayed under 49 GiB, and I still observed pressure: swap reads and page compression and decompression were recorded. The effect on timing was not measured, so every custom writing time in this series carries the note “potentially affected by memory pressure”: not a quantified slowdown, and not a claim that swap caused the one failure.

The 65,536-token prefill was a synthetic memory test. It did not test answer quality at that length or establish that other workloads would run without swap. These numbers are from one M5 Pro; the field guide explains why context and other open apps make any single safe-peak figure unreliable.

What it finished, next to the hosted model

The reference I compare against is the hosted Flash endpoint on OpenRouter, served by Alibaba with a requested 64k reasoning allowance. It passed 8 of 8 objective tasks in 316.58 s of request time. The custom build in native MTP mode passed 6 of 8: 487.85 s across its six passing tasks, and 4,990.94 s across all eight attempts, because intervals and batches each consumed the whole 64,000-token allowance without a final solution. In ordinary one-token-at-a-time mode the same build passed 7 of 8, failing batches, so the MTP decode gain did not mean fewer failed tasks.

Hosted request time includes network and queue, and the provider’s GPU decode rate was never visible to me; the local time is a resident request on one machine; one trial each. A separate hosted trial with a 32k reasoning allowance passed 7 of 8, failing cache, so the two hosted trials do not show that the larger allowance caused the difference.

Multi-token prediction is intended to raise the decode rate: how fast the model emits tokens during generation. Across the eight objective attempts the custom build’s weighted decode was 28.42 tokens a second, and it ran between 30.50 and 33.64 tokens a second across the four writing tasks. The two runaway attempts decoded at 27.41 and 29.56 tokens a second while producing 64,000 tokens each and no answer.

The writing tasks

Writing went the same way: hosted Flash finished 4 of 4, the custom build 3 of 4.

The model-judge mean is 91.00/100 over four tasks for hosted Flash and 88.33/100 over three for the custom build, and those two are not comparable. On the three tasks both finished, custom scored 88.33/100 and hosted 92.33/100. The custom build spent 6.55 minutes on its three finished answers and 33.50 minutes on the one it never finished.

On the data memo, the custom build got every number right (5 of 5 on numerical accuracy) and lost most of its points on its closing sentence, which the reviewer quoted: “The safer operational conclusion is that the checkout change did not improve conversion on either device and may have harmed it”. The data were a non-randomized before-and-after comparison with a changed advertising mix, so the reviewer marked the causal claim as too strong and scored causal caution 3 of 5. The hosted memo on the same data said “Recommendation: do not conclude the new checkout is better.” and scored 5 on that dimension. On the knowledge article, the custom build produced no final text; its raw output ends with the phrase “heated infrastructure” repeated, which shows repetition, not whether pruning, quantization or MTP caused it. The hosted endpoint finished the same article in 59.94 s with a model-judge rating of 87/100 and no factual error recorded.

All of these scores are one assistant’s rubric judgments. The custom answers were reviewed with the model’s identity known; the twenty earlier writing reviews in this series had shuffled labels, but the same assistant had already seen those answers; there was no human blind test; a gap like 93/100 against 92/100 is not statistically meaningful.

These results do not establish that the build preserves the full model’s quality. The hosted endpoint’s weight precision was never independently verified, and pruning, quantization, runtime and effort settings all differ between the two conditions at once. The results compare these two setups; they do not isolate the effect of removing 224 experts per layer.

The full specification

What REAP removed, and what it did not

The published manifest keeps 288 of 512 routed experts in each of the 48 main layers and removes 224. Across the model that is 10,752 removed layer-and-expert pairs and 13,824 retained ones, or 43.75% of the routed bank. That percentage is of the routed expert bank only: each token still activates the top 10 experts and the shared experts are untouched, so it is not 43.75% of the weights or of the compute per token. The main model’s logical parameter count falls from 125.744 billion to 72.868 billion, about 42%, counting neither the lookup table nor the head nor the vision weights. That is not dense-model equivalence either.

Each expert’s selection score is the accumulated router-weighted magnitude of the expert’s output, divided by the number of times the router selected it. That is a saliency heuristic: how much the expert contributed when used, not how often it was used, and it says nothing about what the expert is for. Audited against the published statistics, every retained set is exactly the 288 highest scores in its layer. In the calibration statistics the retained experts account on average for 79.42% of recorded routing selections and 87.87% of accumulated score; those describe the calibration observations before pruning, not retained accuracy.

The publisher reports calibrating on about 686,000 tokens of its team’s agentic-coding traffic. The chosen sets can be reproduced from the published statistics; the statistics themselves cannot be regenerated, because the corpus is not published. Expert IDs are local to a layer, so expert 42 in layer 0 and expert 42 in layer 47 are unrelated tensors. Nothing in the manifest labels any expert as “writing” or “coding”, and no verified map of that kind exists.

The full lists are published here: retained experts by layer and removed experts by layer. The retained manifest’s SHA-256 is 50dbb9cacc445724c64c9429940bc1b6d0ace48fac3a92734376740128b13dd6, shared by the archived sh0wie selection and the retained MLX-Serve pack, so the selection audited here is the one inside the custom build.

The precision recipe

Q4g32 means four bits per weight with one scale for every group of 32 weights; a smaller group keeps more precision at the cost of more bytes.

Smaller groups and the selected Q8 matrices hold more precision in those components than a uniform Q4 with 64-weight groups, which is why this pack costs more bytes than the published REAP-288 Q4g64 build; they do not make it a higher-quality model, and nothing here isolates their effect. And nothing in the recipe restores what pruning removed. Expanding a four-bit tensor to eight bits cannot recover discarded information, which is why the rebuild starts from BF16 rather than from an existing Q4 pack.

The configuration, field by field

The optional field added to MTPLX sits beside the two that were already there:

{ "num_experts": 288, "mtp_num_experts": 512, "num_experts_per_tok": 10 }

When mtp_num_experts is present, the head is built with copied geometry using that count; when it is absent, the default path runs as before. Strict weight loading is preserved either way. The converter applies the same selected expert IDs, in the same order, to the expert weights and the matching router rows, so the router’s 288 outputs line up with the 288 experts.

The patch adds no new speculative acceptance rule, no cache rollback and no generation kernel. Native MTP generation, the acceptance algorithm that decides which proposed tokens to keep, and the SSD n-gram facility were all existing MTPLX features, and the build does not touch them.

The public repository contains MTPLX-INTEGRATION.md, describing the change, and mtplx-reap288-build.patch, the diff against one upstream commit. Three commit identifiers appear in the records, and they are not interchangeable:

• Source revision the build’s checkout came from: de4b8e4d43b917e7706784d8bb445c9af86a3540
• Runtime the evaluation ran under: 11021aedf035d10f88e829d4576e9ac969434a53
• Upstream commit the minimal patch is documented against: 21be78b3f51820eecef020e5e4855c0715eaf9a5

What the build did, in order

A record of the sequence that ran, not a recipe.

1. Pin the source checkpoint and the expert manifest to specific revisions; record their hashes.
2. Verify every source shard against the pinned manifest before reading it.
3. Process the BF16 shards one at a time, so the whole checkpoint is never resident; the build is resumable, and each output part is reopened and hashed before it is committed.
4. Select the retained experts and the matching router rows first, then quantize.
5. Keep the full MTP head and the vision weights as they are.
6. Write the tensor index and configuration only after every indexed source tensor is accounted for, so a partial build cannot load by mistake.
7. Load the result; check generation, the cache and the context ceiling.

Source retention was partial: the original checkpoint was streamed through, not kept as a complete copy on the machine. I did not train a new head or calibrate a new mask; either one is a project of its own.

Who made what

• Qwen trained and published the original Flash-Next checkpoint, including the multi-token-prediction head: a small extra network that proposes several tokens ahead for the main model to verify in one pass.
• Cerebras Research published the REAP pruning method and its reference implementation.
• sh0wie applied REAP to Flash-Next and published the REAP-288 expert selection: the manifest of which experts to keep in each layer, and the saliency statistics behind it, which score each expert’s contribution when the router used it. I used it as published, pinned to one revision.
• MTPLX, Youssofal’s runtime, already had native MTP generation, its acceptance algorithm and the facility for reading an n-gram table from the SSD; the mixed four-bit/eight-bit (Q4g32/Q8) recipe is existing MTPLX work too, inspired by Youssofal’s optimized builds.
• My part was integration: applying the retained-expert mask inside a streaming rebuild from the original full-precision BF16 checkpoint, and adding support for a 288-expert main model that still carries the original 512-expert MTP head, in an isolated checkout of the runtime.

I did not invent MTP, REAP, quantization or SSD-backed tables, and the result is not a fine-tune. No additional training was involved.

What is published, and under what licence

The build is public at khuasar/Qwen3.8-Flash-Next-REAP-288-Q4g32-Q8-MTPLX. I verified it on 17 September 2026 through the public, unauthenticated Hugging Face metadata API: revision 05858670f3f947abd21c43d430b061832b25c22c, 121 files, public. The saved upload receipt verified all 120 planned files with no mismatches; finalizing the README is a separate check.

The repository’s licence metadata says other, and the local model card names the Qwen Community License 1.0 with a link to the upstream text. Anyone redistributing or deploying the build has to check the upstream licence and its obligations; nothing here is legal assurance, and the build is not labelled Apache 2.0 or MIT. Nor is any of this a verified clean-install recipe; that check is still open.

What I would try next

These tests have not been run, and none of what follows is evidence in this article.

• Calibrate a mask on held-out non-coding data and compare it with the published REAP-288 at the same memory budget, keeping the current mask as the control.
• Test controlled swaps of a few experts near the saliency cut line, where scores are close, before building any new mask.
• Try selective Q5 or Q6 precision on the sensitive matrices and measure what changes.
• Run the custom weights in MLX-Serve: conditions matrix-06 and matrix-07 are recorded as not run, not as zero.
• Test a reasoning limit and corrective feedback in a harness next week. I deliberately omitted an earlier cutoff in these runs to see whether Flash would eventually answer. The harness could detect repetition, tell it that it is looping and ask for another attempt; self-correction is a possibility to test, not an established result. Measure completion and total time including retries. The separately prepared retries with a 64k reasoning allowance and an 8k answer allowance remain paused.

Updated recommendations will appear in the living guide.

Glossary

MLX allocation

Memory held by MLX, Apple’s array framework for its own chips, which the runtimes here are built on. It is not the whole machine’s memory use.

Routed expert

One of the small feed-forward networks in a mixture-of-experts layer. A router picks the top 10 per token in this model; the shared experts are outside the routed bank and were not pruned.

REAP-288

A published selection that keeps the 288 highest-saliency experts of 512 in each layer. Saliency is accumulated router-weighted output magnitude divided by selection count.

Q4g32 / Q8g64

Four bits per weight with one scale per 32 weights; eight bits per weight with one scale per 64. Smaller groups keep more precision and cost more bytes.

MTP

Multi-token prediction: a small head proposes several tokens and the main model verifies them. It can raise the decode rate; that is not the same as a faster or more correct finished answer.

N-gram table

A lookup keyed by short token sequences that Flash-Next consults during generation. Here it is 29.802 GiB and read from the SSD through a cache.

GiB

Powers of two. This machine has 68,719,476,736 bytes, which is 64 GiB, shared by CPU, GPU and macOS.

Public sources