Running Local AI Models on a 64 GB MacBook Pro

I wanted to know what my own laptop could do without sending anything to a provider. It turns out to be quite a lot, as long as the work is the kind with a checkable answer: Python and SQL, reconciling an invoice, pulling records out of a messy document, applying a set of rules to an awkward case. That runs offline now, on an M5 Pro with 64 GB of memory, and those jobs no longer leave the laptop.

Running a model locally involves three choices: a model package that fits alongside macOS and everything else you have open, a runtime that can actually run that package, and context and output limits sized to the job. These determine whether the model loads, how much context it can use and how long a request can run.

I settled on Qwen 3.8 27B in Youssofal’s Optimized-Quality 8-bit package, run under MTPLX. It answered every structured task I gave it, and so did Swift’s package of the same model. Youssofal’s smaller 4-bit package answered them all in an earlier round and takes about eight and a half gigabytes less room on disk, though it never went through the longer runs or the writing prompts. The 27B build comparison covers the differences between those downloads.

Another option is the much larger Flash-Next model that I cut down, stored in fewer bits and left partly on the SSD until it fit. On the work both it and the 27B finished it was about a quarter faster, and on its best task it came back in well under half the time. The build article explains how it fits; the head-to-head comparison covers its speed and the tasks it failed to finish.

Running locally gives me offline access and no per-token bill. Prompts and answers stay on the laptop when the runtime and tools make no external calls; telemetry, web tools and synced folders still need checking. I still use a hosted model when a whole codebase has to be in view at once, or when an answer is wanted in seconds rather than a minute or two.

Everything here comes from one M5 Pro with 64 GB of unified memory, one trial per condition, in the 17 September 2026 snapshot. Nothing was run the way a coding assistant works, where the model can execute the code, read the error and try again — no repository, no tools, no repair loop — so none of it establishes how these models behave as coding agents. Updated recommendations and the full results are in the living model guide.

Choosing a model and runtime

I check available memory first, choose a compatible model package and runtime, then set the context and output limits for the job.

The memory is one pool

A 64 GB MacBook Pro has a single pool of unified memory that the processor, the graphics cores and macOS all draw from. There is no separate 64 GB of video memory: the model, its working context, the runtime’s buffers, your browser and the operating system all compete for the same space. Headlines say 64 GB; the tables further down use GiB, the power-of-two unit, because that is what the tools report.

So “will it fit?” is not a question about file size. What matters is what the machine holds while the model works: the weights, the KV cache — the runtime’s running record of the conversation, which grows as the context grows — the work buffers, and everything else you have open.

The custom Flash build shows how disk size and memory use can differ. Its files add up to about 77 GiB, more than the whole machine’s memory, and it runs anyway, because around 30 GiB of that is a lookup table the runtime pages off the SSD instead of holding in memory. Loaded for text, what it actually held was about 46 GiB. How I made Qwen Flash-Next fit on a 64 GB MacBook Pro is the whole story of that design.

A smaller file does not guarantee a smaller working set, and a larger one does not rule a model out. Leave room for the context you need and the applications you keep open: both use the headroom, so there is no universal safe peak to plan against. During the custom writing run I watched memory pressure, swap reads and memory compression even though the model’s allocation was nominally below physical memory, and I had rebooted the machine beforehand. How much that slowed the run I did not measure, which is why every custom Flash writing timing on this page is marked as potentially affected by memory pressure.

The three numbers in a model’s name

A name like “Qwen 3.8 27B Q8” identifies the model generation, parameter count and storage precision.

Parameter count is the size of the network: 27B means about 27 billion weights. Active parameters is how many of those get used for each token. In a dense model, all of them. In a mixture-of-experts model each layer holds many small “expert” sub-networks and a router picks a few per token, so the active count is far below the total. Quantization bits is how many bits each weight is stored in: Q8 is eight, Q4 is four, and a suffix such as g32 is the group size, how many weights share one scale factor. So “27B Q8” is a 27-billion-parameter model stored at eight bits per weight. It is not an 8B model.

The custom build has a pruned expert bank: a large share of its stored experts were removed, which reduces file size more than the work it does per token, because the router still picks the same number of experts every time. It is not the equivalent of a smaller dense model either. The exact counts are in the full specification.

I also tried a 27-billion-parameter model squeezed down to about 2 bits per weight, which makes for a much smaller file at the same parameter count. It is not a 2B model, and on the runtime and settings I used it did not deliver what I wanted. Its numbers are in the earlier screening runs.

The runtime counts as much as the model

A runtime is the program that loads the model file, runs the arithmetic on the graphics cores, keeps track of the context and produces tokens. The local runs here sit on MTPLX, which is built on MLX, Apple’s array framework for its own chips. Where the tables say “MTPLX HTTP” the model sat behind a resident local server; “MTPLX native” means my test harness called the library directly. A model file also has to be in a format the runtime reads, with an architecture it knows how to execute.

Runtimes differ in how they produce text. Ordinary autoregressive decoding produces one token per pass through the model. Multi-token prediction adds a small head that proposes several tokens ahead and lets the main model verify them in one pass: accepted proposals come almost free, rejected ones are thrown away. It raises the rate at which text comes out. It does not, by itself, get you a correct answer sooner, because what you wait for is the whole task, and a task can run long or never end. I ran the custom build both ways. The faster-decoding mode passed one task fewer and took about twenty-three minutes longer over the full suite.

When comparing results, keep the model file, runtime, decoding mode, effort level, output allowance and context length together. Changing any of these produces a different test condition, even if the model name stays the same.

I tried Qwen’s original upstream checkpoint early on and it was too slow to be worth keeping. I did not record timings for it, so it is not in the data below and that attempt predates the recorded suite; nobody ran those full-precision weights on this Mac inside the measured comparison. The working baseline throughout is the Youssofal Optimized-Quality Q8 package, which I call Youssofal Q8 MTPLX from here on. Its sibling, the Optimized-Speed Q4 package, is Youssofal Q4 MTPLX.

How I choose for each kind of work

This is the order I read a result in, and the order the living guide uses.

1. FitDoes the artifact load with headroom for your context and your other apps?
2. FinishDoes it return a final answer inside the output allowance, every time?
3. CorrectnessDoes the answer pass the check, or survive a factual review?
4. TimeOnly now compare seconds to a passing answer, with failed time shown beside it.

These recommendations cover the workflows tested on 17 September 2026. Updates go in the living model guide.

Coding and structured tasks: the main test. For anything with a checkable answer, Youssofal Q8 MTPLX (Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality) is my conservative default: it passed every objective task I gave it. The Swift Q8 build (KrossKinetic/Swift-Qwen3.8-27B-MTPLX-Q8) passed them all too, marginally quicker in one trial, which is too little evidence for a speed ranking; I treat them as close peers. Q4 against Q8, the Cold Fusion fine-tunes and a hosted 27B check are in Which Qwen 3.8 27B Build Should You Run?.

A larger, faster option for selected tasks. Custom Flash completed six of the eight coding-suite tasks and was faster than Youssofal Q8 on five of those six. Its 1.26x speed advantage is the geometric mean over the six tasks both models finished. It offers useful speed on those tasks, though the two Python tasks did not finish in this run. The head-to-head comparison has the per-task results.

Writing: a secondary check. The four writing prompts tested summaries, explanations, data memos and rewrites. Both 27B builds completed all four; custom Flash completed three. The writing results below report completion, time and model-judge ratings separately from the coding results.

The pruning selection also reflects the workload used to choose it. The publisher reports using about 686,000 tokens of agentic coding traffic to score the experts, retaining those with the highest measured contributions on that workload. Lower-scoring experts were removed; that does not mean they contributed nothing or were irrelevant to other tasks. Calibrating a selection on writing workloads could retain different experts and improve writing results at the same memory budget, but that has not been tested here. The build article explains the selection method.

What I do on a first run

I start by pinning down exactly which artifact I have and which runtime it was built for, because a family name is not enough: “Qwen 3.8 27B” turned out to cover several materially different packages in my tests. I load one model at a time. Before loading it I look at what memory is free and what the runtime will let itself take, and I quit whatever I do not need. Then I set the context length and the output allowance to what the job actually needs, remembering that the allowance covers the model’s reasoning and its final text in one budget — a task that runs to the ceiling has spent all of that time and has nothing to hand back.

After that I give it one short task I can check by hand, before anything long. I write down how long the whole task took and whether the answer was right, and I treat the tokens-per-second counter as secondary: the custom build’s faster-decoding mode passed fewer tasks than its slower one. While it runs I watch memory pressure rather than only swap used, because occupied swap is not the same thing as active swapping; if pressure is high I keep the good outputs but stop treating the timings as clean measurements. And I make sure I know what disk activity means for the particular model. The custom build reads its lookup table off the SSD by design, which is a model-specific strategy rather than expert offloading or swap, and the experts pruning removed are gone rather than parked on disk.

How the builds performed

The main comparison covers the eight coding-suite tasks. The cards also include secondary writing results, rated out of 100 by one model judge. Each card links to its recorded condition in the living guide.

The eight coding-suite tasks: what finished, and how fast

Youssofal Q8 MTPLX passed all eight objective tasks in 743.10 seconds. Swift Q8 also passed all eight, in 724.15 seconds, 2.55% less in a single trial, with different tasks faster on each side: Swift’s SQL task took 139.99 seconds against Youssofal’s 68.07.

The custom Flash build passed six of the eight, in 487.85 seconds across those six, and 4,990.94 seconds in total once the two failed attempts are counted; each of those ran to the 64,000-token ceiling without producing a final answer. On the six tasks both it and Youssofal Q8 finished, it used 79.64% of Youssofal’s time as a geometric mean, equivalent to a 1.26x speed advantage on that set. The arithmetic mean of those same six ratios is 85.06%. It was faster on five of the six and slower on invoice, 130.35 seconds against 93.26. Its best single task was policy: 25.45 seconds against 59.58, 2.34 times faster. Its weighted decode rate was 28.42 tokens per second against 18.67, about one and a half times the token rate, which is a different quantity from finishing the work one and a half times sooner.

I also ran the custom build through the objective suite in both decoding modes to compare token rate, completion and total time.

MTP decoded faster, 28.42 tokens per second against 22.55, and the tasks it passed took less time in total, 487.85 seconds for six against 792.02 for AR’s seven. It still lost a task, and the whole suite took about 23.5 minutes longer because the two failed attempts each ran to the 64,000-token ceiling.

What the memory actually looked like

During the writing run, after a reboot, active memory sat around 46.5 GiB with per-task peaks under 49 GiB, and memory pressure was still visible. The dense 27B packages are much simpler to plan for: the Youssofal Q8 package is 27.91 GiB on disk and the Youssofal Q4 package 19.28 GiB, both disk measurements rather than peak memory.

How the hosted endpoint compared

Cloud endpoints give an external baseline with a different latency and cost profile, and the recorded results are not simply the same model running faster. A hosted Qwen 3.8 27B condition, provider-advertised BF16 with a 64k reasoning allowance, passed 6 of 8 objective tasks, failing schedule and SQL, in 273.82 seconds across all eight attempts, or 200.62 seconds on the six that passed. It finished 4 of 4 writing tasks with a model-judge mean of 86.5/100 in 495.31 seconds. Serving stack, rendering and reasoning controls were not matched to the local runs, so this is a provider-specific, date-specific workflow comparison. It does not show that quantized local weights beat BF16, or the reverse.

This suite is also far too small to justify a hardware purchase, and I did not measure energy use or running cost.

Writing as a secondary check

The four secondary writing tasks checked source summaries, knowledge explanations, data memos and customer-email rewrites. One model judge rated the finished answers out of 100 using a task-specific rubric. These ratings are separate from the coding suite’s automated pass/fail checks. Youssofal Q8 MTPLX finished all four in 1,627.96 seconds and Swift Q8 in 1,182.95. The custom Flash build finished three in 393.01 seconds, about 6.55 minutes, and then spent 2,009.93 seconds — about 33.5 minutes — on the knowledge explainer, using its entire 64,000-token output allowance and returning no final article. Its run as a whole took 40.05 minutes.

The model judge rated the custom build’s summary 93/100 and its customer announcement 92/100; they took 181.66 and 125.38 seconds. Youssofal Q8 received 88/100 on each from the same judge, in 326.64 and 232.35 seconds. The custom build’s data memo received 80/100 and took 85.98 seconds; the judge flagged a causal claim the data did not support.

Both 27B builds finished the knowledge explainer, and the judge marked both down for factual errors: one wrote that the urban heat-island effect “happens every summer evening in virtually every city”, which is too strong, and the other that tree “roots release moisture that cools the air”, which is the wrong mechanism. A finished explainer is still a draft that needs a fact check.

The full specification

How I measured

The test has 12 tasks: eight coding-suite tasks as the main evaluation and four writing tasks as a secondary check. The coding suite has checkable answers: five want JSON, two want a Python function and one wants a SQL query, and the evaluator executes the code and the query after the model answers. The four writing tasks are rated by one model judge on weighted dimensions from 0–5, then scaled to a rating out of 100, with any factual error quoted beside the score. The harness is a Python script that sends the prompt, saves the response and runs the graders; it gives the model no tools, no repository, no test feedback and no repair round.

The objective suite is only eight prompts, so one pass moves the rate by 12.5 points: 7/8 and 8/8 are one task apart, not a tier apart. Each condition is one trial, and the tasks are different jobs rather than repeat measurements, so there are no confidence intervals anywhere on this page. The suite tilts toward structured, algorithmic work with a modest writing extension; it is not a full agent benchmark. Failed attempts keep their elapsed time and their reason, unrun conditions stay “not run” rather than becoming zero, and an empty final answer is unscored rather than scored zero. The prompts, the rubric, the JSON behind every chart and the change history are on the living model guide.

Every writing score is a rating out of 100 from one model judge using a rubric, not a human blind test. The custom build’s answers were reviewed with its identity known; the other two sets had shuffled labels but prior exposure. A difference of one or two points, such as 93/100 against 92/100, is not meaningful.

What the pruning removed, and who made what

Qwen 3.8 Flash-Next is a mixture-of-experts model with 512 routed experts per layer across 48 layers, and the router uses the top 10 for each token. The published REAP-288 selection the custom build derives from keeps 288 of those experts per layer and removes 224. That is 43.75% of the routed expert bank, but not 43.75% of the weights and not 43.75% of the compute per token, because the router still picks 10 experts and the attention and shared components are untouched. The main model’s logical parameter count drops from about 125.744B to 72.868B, a reduction of about 42%. It is not equivalent to a dense 73B model: per token it still runs a small routed subset, and the removed experts are absent rather than stored elsewhere.

The selection uses a saliency score based on each expert’s router-weighted output magnitude when selected. The publisher’s calibration data came from agentic coding use cases, so the selection reflects contributions on those examples. It does not label experts by specialty or establish that pruning caused the writing failures. A writing-calibrated selection would need a separate comparison on held-out writing tasks.

The checkpoint and the multi-token-prediction head are Qwen’s. The REAP-288 expert selection is sh0wie’s published work, used here at a pinned revision. Native multi-token prediction, its acceptance algorithm and the SSD-backed n-gram facilities were already MTPLX features, and the Q8 and Q4 27B packages are Youssofal’s. My own contribution to the custom build is integration and conversion: a modified MTPLX checkout that lets the original 512-expert MTP head attach to a 288-expert trunk, and the conversion that applies the same expert selection to the expert weights and the router rows. The build uses the existing decoding algorithm and involves no additional training.

The earlier screening runs

Historical command-line runs, the current resident-server runs and the hosted runs are separate cohorts, and this page never pools them into one table.

Ternary Bonsai 27B is the clearest example of why. It is a 27-billion-parameter model stored at about 2 bits per weight — not a 2B model. Its numbers come from an earlier screening cohort whose budgets, runtimes and stopping rules vary, so they do not line up with the 17 September runs: 2 of 8 objective tasks on the first pass at a 4,096-token output allowance, and 6 of 8 when each task’s latest attempt is counted at allowances from 4,096 to 16,384 tokens, which is not one fresh re-run. Weighted decode across that mixed set was 20.16 tokens per second. That is not a verdict on ternary models in general. This artifact, on the runtime and settings I used, did not give the expected trade-off: a 27B-class model in far less memory that still finished the suite in reasonable time.

Conditions I did not run

The custom build needs a modified MTPLX checkout to run at all, and it is not established as working on every stock release. Two planned conditions for it under a different serving runtime, MLX-Serve on the same machine, were never run; the living guide shows them as “not run”, not as zero.

Glossary

GB / GiB

GB counts in powers of ten, GiB in powers of two. This Mac has 68,719,476,736 bytes, which is 64 GiB; headlines say 64 GB and the tables say GiB.

Unified memory

One pool of memory shared by the CPU, the GPU and the operating system. A model's allocation competes with everything else running, so there is no separate VRAM figure to plan against.

Dense / MoE

A dense model uses every weight for every token. A mixture-of-experts (MoE) model holds many expert sub-networks per layer and routes each token through a few of them, so its active parameters are far fewer than its total.

Q4 / Q8 / group size

The number of bits each weight is stored in: four or eight. Weights are quantized in groups that share a scale, and a smaller group (g32 rather than g64) keeps more precision at the cost of more bytes; neither guarantees better answers.

Context / output / reasoning

Context is the total tokens the runtime can hold, prompt and output together. The output allowance covers reasoning and the final text in one budget, and "requested reasoning" is an allowance, not tokens actually used.

AR / MTP

Autoregressive decoding produces one token per pass. Multi-token prediction uses a small head to propose several tokens that the main model verifies in one pass, which raises the decode rate when proposals are accepted.

tok/s versus task time

Tokens per second is how fast text comes out while it is coming out. A weighted rate pools the eight tasks in proportion to the tokens each produced, rather than averaging the per-task rates. Task time is seconds from request to a final answer that passed, which is the number these pages rank on.

Public sources