DeepSeek V4.1 Flash: reading long prompts with less memory

Understanding a long document with AI often takes more than one question. You need to find a particular clause, compare several sections, then ask for clarification. For you, this is one conversation. For the model, it means working with a large amount of text that it must refer to repeatedly while composing its answer. The longer the material, the harder it becomes to find the relevant information quickly.

A model typically saves intermediate results from text processing so it can use them later. It therefore need not recalculate the entire document for each new answer token. Saving results also requires resources. They occupy memory, and searching and moving data takes time. With a large document or accumulated request history, handling these data can become a major burden: the processor must obtain the results of earlier computations as well as perform new ones.

DeepSeek V4.1 Flash changes how those records are handled. In its new technical report, DeepSeek describes separating reading from answer-writing, reducing the number of separate copies of data, and rebuilding some records when needed. The savings begin before the answer appears, with how the model reads the request and stores what it has read.

The model’s work is divided into two phases. During preliminary reading, or prefill, it processes the request, attached documents, and other supplied information to prepare the context for its answer. Text is divided into small pieces called tokens: words, parts of words, or punctuation. They pass through the model’s layers, successive stages of text processing. Each layer calculates two sets of numbers for each token, called keys and values. These are stored in the KV cache. The KV cache contains intermediate numerical data that the model then uses to calculate its answer.

Decode, the writing phase, follows: the answer appears sequentially, one word or token at a time. Selecting the next most probable piece requires taking previous text into account. Stored keys and values allow access to information already processed without recalculating the entire conversation from the beginning. The model spends computation on reading once, then uses the results while writing its answer.

The cache holds the results of computations over text. When the model continues its answer, it uses the data already calculated for previous tokens and performs a new calculation for the next element. This reduces repeated processing. A short request produces relatively little cached data, which fit in the available fast memory. Their storage and transfer costs therefore do not usually become the main constraint.

The situation changes when systems are expected to work independently for long periods. Agents, systems that carry out sequences of actions themselves, are expected to handle tasks for hours or days, keep track of many documents or a large codebase, and continue working with accumulated information. Intermediate data grow with the processed text. Keeping results still avoids repeated computation, but the cache itself becomes large. Each new token requires access to suitable data within that volume. Two related demands arise: storing the results and using them while continuing the answer.

The graphics processor, or GPU, performs the model’s calculations and obtains the necessary data from memory. HBM, fast high-bandwidth memory, is located beside it. It can supply data with little delay, but its capacity is limited and it is expensive. Intermediate records of processed text are not the only data that need space there, so a growing KV cache increases the system’s resource requirements. The more data a calculation needs, the more significant access speed becomes.

When the KV cache does not fit in the available fast memory, some data can be stored outside the GPU, for example on solid-state drives, or SSDs. These offer more space, but retrieving data takes longer. The system gains storage capacity while spending additional time loading saved results. A large drive therefore does not by itself solve the problem of continuing an answer quickly: the data must still reach the component that performs the calculations.

If the cache is stored on SSD, generating the next token requires retrieving data from the drive, transferring them through the motherboard into fast memory, and supplying them to the processor. The computing unit may sit idle while a large cache moves. The problem therefore involves more than the number of mathematical operations. Both the volume of data accessed and the cost of delivering them need to fall. Restructuring layers reduces repeated processing, sharing caches reduces separate data sets, and avoiding some saves reduces drive access. Each change removes particular work from the system: calculation, storage, or transfer of results.

A conventional transformer language model processes text through many successive layers. Input passes through them, the next probable answer piece emerges, it is appended to the text, and the cycle repeats. Each layer also creates its own KV cache, which needs storage.

The new architecture divides the layers into two parts: a reading block, called a causal encoder, and an answer-writing block, called a decoder. During preliminary reading, the first part performs the main processing of the source material. Global keys and values for the second part are obtained by transforming the output of the reading block’s last layer. The second part therefore need not repeat full processing of the entire request to prepare those data. Both parts then work during answer generation. This division reduces reading computation.

There is a risk: understanding could deteriorate if half the layers skip reading the original material independently. The answer-writing block therefore still processes some text independently.

Global context covers the entire request, all supplied information, and attachments. Local context concerns the sentence currently being written and text directly relevant to the next token. The decoder obtains its global data from the reading block’s results and processes this immediate neighborhood independently. It uses sliding window attention, examining the most recent tokens especially closely. The result combines an already prepared overall representation with direct processing of recent text. The second half must retain enough information for a good answer even though it performs fewer calculations and creates fewer records of its own.

In this account, roughly half the model no longer needs to create its own enormous set of global records. The technical explanation estimates the effect as approximately halving reading computation. Memory remains an issue, however: global and local records can still be large after reducing the layers working on reading. The volume of the records must also be reduced.

The comparison of global KV-cache size per token gives almost 390,000 bytes for DeepSeek V1, about 3,500 for V4 Flash, and 890 for V4.1 Flash. The last figure is described as roughly a 437-fold reduction against V1 and almost fourfold against the preceding Flash model. How can useful information survive such a reduction? One mechanism is Compressed Sparse Attention 2, or CSA2: layers share saved records and the means of searching them. These figures concern intermediate global-cache data, rather than all the memory required to run the model.

In a conventional scheme, each layer calculates and stores its own keys and values for processed tokens. Even when all layers work on the same original text, their computational results are stored separately. With a large context, the combined volume of these sets becomes substantial. CSA2 changes how caches are created and used: some layers prepare data, and others use the prepared sets. There are three modes, Full, Reindex, and Reuse. They differ in whether a layer creates a new cache and performs its own selection of data for attention. In all three modes, the layer retains its own local records: sharing concerns the global cache.

In Full mode, a layer performs all preparation: it creates a new global KV cache and builds an index to select suitable records. The indexer determines which positions in the stored data are needed for the current calculation. This directs attention to selected parts of the context rather than processing the entire available volume equally. Later layers can use the prepared cache and indexing results. The Full layer creates the original set that is then available for sharing.

Reindex mode creates no new set of global keys and values. The layer uses a Full layer’s cache but performs indexing independently, selecting suitable positions for its calculation. Sharing stored data therefore does not require an identical selection at every processing stage. One set of keys and values can serve several layers while the selection of relevant positions is performed separately for them.

The distinction between a cache and an index matters here. The cache contains intermediate token data; the index is used to select the records attention accesses. Reindex changes that selection while the stored keys and values remain the same. The model reduces repeated creation and storage of data while retaining the ability to determine again which records a particular layer needs. The savings concern the shared set of records; independent indexing still requires computation.

In Reuse mode, a layer uses both a Full layer’s prepared cache and indexing already performed by either a Full or Reindex layer. It creates no new set of global keys and values and does not repeat record selection. The additional storage for its own cache and index therefore almost disappears. The layer continues processing but accesses data prepared earlier. Having no new cache concerns intermediate storage, rather than an absence of computation altogether.

Together, the three modes abandon the assumption that every layer must have its own complete collection of records and pointers. Some layers do everything, some remain independent only in organizing search, and some use both prepared results. Total volume falls by eliminating repeated collections, since subsequent layers access information prepared earlier. This is how the volume of records falls: useful material and the means of finding it can serve several parts of the model.

The next mechanism selects suitable records in advance for subsequent processing. It is called the hierarchical sparse indexer. The decoder’s first layer examines global records and builds a candidate pool, a shortlist of relevant information. In the example given, around 16,000 tokens are selected from one million. Later indexers search for global records within this pool. Local attention continues to process recent tokens. The search space therefore narrows sharply at the start of writing.

The success condition is demanding: if an important detail misses the shortlist, the system could overlook it or invent an answer. The technical explanation claims that the model was trained to build these lists accurately enough that later layers scarcely notice the missing information. The software reduces the work required from the hardware, while the risk of losing relevant information remains.

Optimization also creates another problem: sliding window attention retains intermediate data only for recent tokens. According to the developers’ description, saving this local context for every turn of a multi-turn conversation on SSD became a serious storage burden. The user writes, the model replies, and the user continues; corresponding data are repeatedly saved so the flow of the conversation can later be restored. Caching normally means preserving data where they can be retrieved quickly, but when too much is preserved far from the processor, slow access complicates the benefit. SWA-bounded replay is proposed for this specific difficulty: replaying a bounded amount of recent text rather than continually storing local intermediate records.

Local intermediate records are not kept long-term on SSD. The conversation text and global cache remain. If the necessary local records are no longer available, the model processes the most recent 128 tokens again and rebuilds them on the GPU. This reconstruction is approximate: the data do not exactly match the results of processing the whole history. In the technical report, the developers describe the quality impact as negligible under the conditions tested.

This raises a question: the architecture previously sought to avoid repeated computation and now deliberately reintroduces it. Two risks arise: worse use of context and additional delay from the new calculation. The answer turns on the difference between repeating a small calculation and rereading the entire context. A bounded recent fragment can be reconstructed while overall context exists elsewhere in the scheme. The decision to avoid long-term storage therefore concerns sliding-window local records specifically. The saving does not remove the need for immediate context: that context is still needed, but is recalculated from recent tokens when work resumes.

Two paths are compared. The first transfers local data from the GPU through the motherboard to storage, later locates it, and delivers it back to the processor. The second uses GPU computing power to recalculate the short fragment. In this comparison, a small amount of GPU computation is faster than writing data to SSD and loading it again. The recalculation covers a short fragment of 128 tokens, rather than the entire conversation. Saving such local records therefore consumed a slow resource even though they could be rebuilt quickly. Eliminating that step should both speed work and release drive space.

By this point the scheme combines a local attention window, a division of reading and writing, preliminary selection of global material, and reuse of prepared records by some layers. Bounded recalculation complements these changes by reducing the cost of storing local state.

Single-Pass mHC reduces data movement inside the GPU. Some operations produce intermediate values needed by subsequent ones. In the conventional sequence described, results must first be written to GPU memory and then loaded again.

Long processing repeats these steps billions of times. An individual move is quick, but accumulated delay can become substantial and limit overall processing speed. Single-Pass mHC combines related operations so fewer intermediate results need to be written to memory and loaded again between them. Some actions run together instead of as separate steps in a chain. The claimed effect is less internal memory traffic and faster GPU work.

Engram is a separate memory module. During generation, its data can be prefetched from ordinary server RAM, outside expensive GPU memory. The architectural explanation assigns it the memorization of stable information, such as dates and capitals, separating some storage from active computation. These are learned numerical representations, however, rather than an ordinary fact database guaranteed to return an exact date or quotation. CSA2 reduces and reuses current-context data, while Engram adds a separate form of model memory.

The next supplementary component is called DSpark. Writing an answer sequentially, one token at a time, itself limits speed. DSpark allows several tokens to be prepared in one step, speeding up answer writing. Unlike cache changes, this mechanism addresses the pace of text generation itself: the system prepares several pieces at once instead of advancing one piece at a time. The prepared pieces form a draft that the main model verifies before accepting.

Overall efficiency is explained through the combination of changes rather than one isolated technique. Some mechanisms reduce records, others repeated processing or data movement, and another accelerates text output.

In Figure 2 of the technical report, the vertical axis shows FLOPs, floating-point operations, or the computational work required for the next answer token, and its horizontal axis shows context-window size, the amount of information the model can work with at once. As the window grows from about 4,000 to one million tokens, V4.1 Flash’s decode curve remains almost horizontal. One million tokens is approximately compared to 700,000 words or a medium-sized codebase. In this comparison, the load per new token changes relatively little when moving from a short document to thousands of pages.

Earlier generations show rising load with context. The graph shows computational work under the conditions of this comparison. FLOPs do not measure energy use, request prices, or response time.

The reported test results compare V4.1 Flash with leading models. DeepSWE 1.1 receives a score of 74.2, roughly level with GPT-6 Astra’s 74 and above other open models in this comparison. The CyberGym result is described as one of the best. AutomationBench is said to outperform even GPT-6 Astra Max.

In the LiveBench table, the model is presented as first among open models; in VALS Index, it is presented as first. The latter index is explained as measuring performance on knowledge-work tasks. These are reported results and rankings in the cited tables. Detailed testing conditions and numerical scores are not supplied for these CyberGym, AutomationBench, LiveBench, and VALS Index comparisons.

Quality is compared with the cost of completing tasks. The models in second and third place in the displayed table are described as costing over 20 times as much; a separate cost-per-task chart is also interpreted as favoring DeepSeek against GPT-6 Astra and other closed models. These are relative comparisons in the cited tables, without tariffs or a specific monetary amount per task.

When software accesses the model through its API, output speed is claimed to exceed 200 tokens per second, approximately four times GPT-6. Time to the first answer is characterized as the lowest in the industry, but no numerical latency is given. Text output speed and the wait before it begins are different measures: a model may write quickly once it starts but spend a long time preparing. Detailed measurement conditions are not supplied for these comparisons.

V4.1 Flash’s weights are openly available, and the model can be downloaded to run locally. DeepSeek publishes it alongside a technical report describing the architecture and evaluation results.

For a company, open weights make it possible to deploy a model within its own infrastructure and work with internal documents. A law firm could use such an assistant to find clauses in contracts and compare versions. A manufacturing company could search instructions and technical documentation; a support team could prepare answers using its internal knowledge base. In legal work, a specialist still needs to check the model’s findings: a long context window does not rule out missed details.

If you want to deploy AI within your own infrastructure, we can help select a model and hardware, configure it to run locally and connect your documents. You can start with one specific task, such as searching a contract archive. Tell us what materials your employees work with and which part of their work you would like to simplify.