Local models for a home PC: Qwen3.8-Flash-Next, Gemma and choosing by memory

A home model starts with memory

On 4 November, Unsloth released new GGUF builds of Qwen3.8-Flash-Next. Discussions of the model can combine claims of running on an ordinary computer with parameter counts in the hundreds of billions. Its deployment architecture explains both: some data resides on the GPU, some stays in system RAM, and large tables can be loaded from an SSD. Speed depends on how much data must travel between GPU, RAM and SSD.

For home use, two tasks are worth separating. One is getting a responsive assistant for text, images or small programming tasks on an existing GPU. The other is running a large model across CPU, RAM and GPU while choosing an inference engine and offloading arrangement. Compact Qwen and Gemma models address the first task. Qwen3.8-Flash-Next, GLM-5.3-Flash and heavier DeepSeek versions enter the second.

This selection covers hardware classes as of 8 November 2026. It draws on developer model cards, quantized-build recommendations and a published home-hardware measurement log. Wherever speed is quoted, the computer, engine and test mode are identified. The editorial team did not run these models itself.

8–16 GB: a small model leaves room to work

For a GPU with 8 GB, four-bit Qwen3.5-4B is a reasonable starting point. The Unsloth table lists around 5.5 GB of total memory for this build. Quantization stores some weights at lower numerical precision, reducing model size so it can fit on a more modest device. The remaining capacity is needed for the runtime and data from the current request.

Qwen3.5-4B accepts text and images. That makes it an initial option for questions about a picture, extracting information from a small document, rewriting text and simple code fragments. Task size matters: one error screen and a large repository impose different demands.

At 12 GB, Qwen3.5-9B becomes more attractive. Unsloth lists about 6.5 GB of total memory for the four-bit build and 13 GB for eight-bit. Owners of a 16 GB GPU can therefore keep the lighter build and more headroom or try eight-bit. These allocate memory differently; long conversations and images also consume capacity.

In the Qwen3.5 developers' comparison table, 9B scores 81.7 on GPQA Diamond against 76.2 for 4B, and 65.6 on LiveCodeBench v6 against 55.8. The first tests difficult scientific questions; the second tests programming problem-solving. These are developer evaluation results. They indicate the larger version's gains, while the quality of a particular four-bit build remains a separate assessment.

For audio tasks, Gemma 4 E4B is another option. Google describes it as accepting text, images and audio and returning text. The Unsloth recommendations for Gemma 4 put four-bit E4B at roughly 5.5–6 GB of total memory and Gemma 4 12B Unified at 7–8 GB. For a laptop used to analyze voice recordings and pictures, that is a substantive distinction from selection based only on a coding score.

24–32 GB: Qwen3.8-27B as an everyday model

For a 24 GB GPU, four-bit Qwen3.8-27B is the most straightforward choice in this selection. Build provider Unsloth gives an indicative 16–19 GB of total memory. MTP acceleration requires another 1–2 GB. MTP proposes subsequent tokens in advance, and the main model checks them; successful predictions reduce the number of separate computation steps.

Qwen3.8-27B has a dense architecture: its 27 billion parameters operate without being divided into separate experts. It accepts images and video, supports tools and allows control of its reasoning mode. It is a candidate for an everyday local coding and document assistant on tasks smaller models handle less well.

Qwen's model card reports 89.2 on GPQA Diamond and 61.7 on SWE-bench Pro for 27B. The latter result uses an agent within the Claude Code harness and the conditions described beside the table. Such a test combines the language model, tools and sequence of actions. Those scores belong with the particular model version and execution setup.

A 32 GB GPU provides more options within the same 27B model. Unsloth lists 23–26 GB for a six-bit build, plus additional capacity if MTP is used. Another option is Gemma 4 31B, for which the build provider gives 17–20 GB at four bits. That permits comparison between two families on the intended tasks while retaining enough capacity for execution.

What Qwen3.8-Flash-Next contains

Qwen3.8-Flash-Next is the downloadable-weight model corresponding to “3.8 Flash.” Its official model card lists 125 billion parameters in the main model, six billion active per token, another 51 billion in n-gram tables and four billion for MTP. A smaller active portion reduces computation in a step; the other data still needs storage and timely delivery.

The model uses a mixture of experts. Only some blocks are selected for each token, so the CPU or GPU performs the associated portion of the computation. Large n-gram tables provide a separate lookup layer for token combinations. Home deployment becomes a placement problem: which blocks stay near the GPU, which come from RAM and which tables are memory-mapped from the SSD.

The current Unsloth instructions make configuration differences particularly clear. The table without MTP gives roughly 64 GB of total memory for the four-bit option, while the MTP table gives 97–115 GB. The headline about running from 32 GB concerns smaller builds and offloading methods. Before download, the complete set of files matters, including the selected table and additional prediction module.

For a computer with 64 GB of RAM, starting with 27B and establishing how much computation the GPU handles is simpler. Flash-Next becomes a separate experiment in offloading and precise build selection. At 128 GB of RAM with a 24 GB GPU, there is substantially more room for that work: the large pool of ordinary memory makes it possible to try an architecture exceeding one card's capacity.

Flash-Next has a separate Qwen Community License, while Qwen3.8-27B and the small Qwen3.5 models use Apache 2.0. A separate commercial-use agreement is required when a company or its affiliates operate a business providing models as a service or standalone AI assistants for coding or office work. Internal use is exempt from this requirement if third parties cannot access the model, its outputs or its capabilities.

The same hardware runs at different speeds with different engines

The local-ai-bench log publishes an October experiment on a Ryzen 9 5950X, 128 GB of DDR4 and a machine equipped with three RTX 3090s. The single-card runs cited here use one RTX 3090 over PCIe 4.0 x8. Flash-Next produced the following estimated minimum output rates:

The project's metric is the slowest one-minute interval in a ten-minute agent run, excluding the initial minute; its counter uses an approximation through cl100k_base. Each quantization received one such run. These are the specific conditions under which the engine and format change the outcome despite an identical model name.

On its own 50 fresh Python tasks, the project obtained 48 solutions with IQ3_S and 39 with Qwen3.8-27B using the project's separate configuration labelled 208k. The second comparison was supplementary; the set is small and weighted toward easier tasks. November's Dynamic v3.1 builds were not part of this October experiment.

GLM and DeepSeek: the larger Flash models

GLM-5.3-Flash has 320 billion parameters in total and 18 billion active. It is available for local deployment, but belongs in a home-hardware selection only where large memory capacity is available. Unsloth lists around 100 GB for the smallest one-bit version and 128–150 GB for three-bit. On a 128 GB machine, choosing the exact build and data placement becomes consequential, especially with memory also needed by the operating system.

A specific task on which GLM performs better in a personal comparison may motivate that choice. Model size alone provides a poor basis for a general home recommendation. Offloaded-expert speed also depends on the CPU, RAM channels, SSD and GPU connection. For an everyday assistant on an ordinary card, 27B offers a simpler route to a working setup.

DeepSeek-V4.1-Flash-1017, released on 17 October, is another large system. The Unsloth instructions list 128–145 GB of combined RAM and VRAM for a one-bit build with Engram tables on an SSD. The UD-IQ1_S file itself is 233 GB; UD-Q4_K_XL is about 501 GB. Such a home project begins with storage capacity and the data-reading arrangement, before the choice of chat interface.

Choosing for the computer you already own

This table gives editorial starting points based on published memory requirements. The quoted build sizes refer to total memory or a device's unified memory; GPU memory and system RAM differ in access speed. A moderate context is useful for an initial run, while long documents and visual inputs increase demand.

GGUF weights can run in applications supporting the chosen architecture: build providers publish recipes for llama.cpp and graphical applications based on it. Similarly named files from different providers can differ in quantization, chat template and additional modules. Keeping the filename, application version and context settings together records a particular working setup that can be revisited after an update.

The main change in home AI becomes visible on a 24 GB card. A dense 27B can fit entirely in GPU memory, while the much larger Flash-Next draws on the rest of the computer's memory. In the latter case, model selection also means selecting how its data moves, and results begin to differ within the same weight family.