TurboQuant 6x KV Cache Compression and HBM Demand Signals
In an experiment on finding information in long texts, Google reduced the KV cache to at most one-sixth of its original size. The impact on demand for HBM purchases depends on how much of the memory the cache accounted for and how many more requests can be handled at once after compression.
Google's figure of at least 6x compression, announced in March 2026, came from measuring the KV cache that language models use to generate answers.
If the cache is reduced to exactly one-sixth of its original size, 6 requests of the same length take up as much cache space as 1 request did before compression.
On March 26, 2026, Samsung Electronics fell 4.71% and SK hynix fell 6.23% from the previous trading day.
To see how HBM demand is changing, look at both how many requests the service handles at once and how much memory suppliers actually provide.
In experiments, Google's TurboQuant reduced the KV cache used to generate AI answers to at most one-sixth of its original size. This reduces the memory needed per request. Total demand for high-bandwidth memory (HBM) depends on how many more requests are handled at the same time.
The compression figures come from Google Research's March 24, 2026 announcement, and the stock prices are based on the March 26 close.
What is the KV cache that TurboQuant compresses?
The KV cache is temporary storage that a language model uses to refer back to earlier context. The model processes text by breaking it into units called tokens. By storing the keys and values (KV) it calculates for each token, the model can avoid repeating those calculations when it generates the next token.
The longer the conversation, the more keys and values the model needs to store. TurboQuant represents these numbers with fewer bits. This is called quantization. The weights the model learned during training still take up memory after the cache is compressed.
What did the 6x compression and 8x speed figures measure?
The two figures Google announced refer to storage space and the speed of a specific calculation, respectively. Google Research's March 24, 2026 description of the experiments explains what each figure was compared against.
Reported figure | Test conditions | What was measured
At most one-sixth of the original size | A task that involved finding information hidden in a long text | The size of the reduced KV cache while maintaining performance at finding the correct answer
Up to 8x the speed | A comparison of 4-bit TurboQuant keys with uncompressed 32-bit keys on an H100 GPU | How quickly the model calculated the scores that determine how much weight to give earlier tokens
Answer quality after compression
How well quality holds up also depends on how many bits are assigned to each number. According to the research paper, answer quality held up in evaluations even at an average of 3.5 bits per channel. At an average of 2.5 bits, quality declined slightly. Response times and answer quality in a live service need to be tested using the questions that service handles.
A worked example of concurrent requests and memory capacity
If the cache is reduced to exactly one-sixth of its original size, 6 requests of the same length use as much cache as 1 request did before compression. This calculation uses 6x, the lower bound of the compression ratio reported in the experiments. It assumes the same model and request length and that each request's cache is stored separately.
Requests handled at the same time | Total KV cache size after compression
1 request | One-sixth of the cache for 1 request before compression
6 requests | The same cache size as for 1 request before compression
More than 6 requests | A larger cache than for 1 request before compression
To calculate the savings for the whole server, you need to know how many bytes the cache used before compression. Under the conditions above, the space saved is five-sixths of the original cache size. The smaller the cache's share of total memory, the smaller the percentage saved across the server.
When HBM demand could fall or rise
Demand for HBM purchases depends on how service operators use the space they save. HBM capacity is the amount of data it can store at one time. Bandwidth is the amount of data it can move in a given time. To estimate how much HBM operators will buy based on the cache sizes measured in the experiments, you need to know whether capacity or bandwidth was the constraint driving the need for more servers.
Service operating conditions | Possible effect on HBM demand
Request volume and length stay the same, and the operator was planning to add servers because it was running out of cache space | Freeing up space on existing equipment could delay further purchases.
The operator handles more requests at the same time on the same equipment | The space saved can be used to cache new requests.
Lower costs lead to a sharp rise in the use of services that process long documents | Total memory use can rise if the added workload outweighs the savings per request.
Jevons paradox is what happens when improved efficiency leads to an even larger rise in usage, increasing total consumption of a resource. For AI services, the key question is how much actual usage rises after processing costs fall. When assessing memory makers' revenue, shipment volumes and selling prices also need to be considered separately.
Why semiconductor stocks fell on March 26
On March 26, 2026, concerns about slowing memory demand coincided with uncertainty in the Middle East. The Bank of Korea cited both factors in explaining that day's stock market decline. The won-dollar exchange rate ended daytime trading at 1,507.0 won per dollar, up 7.3 won from the previous day.
Index or stock | Change on March 26, 2026, from the previous trading day | Index close
KOSPI | -3.35% | 5,460.46 points
KOSDAQ | Down in the 2% range | 1,136.64 points
Samsung Electronics | -4.71% | Not applicable
SK hynix | -6.23% | Not applicable
Samsung Securities put net selling by foreign investors on the KOSPI market at 3.111 trillion won at the time of publication. Alteogen, which disclosed a technology export that day, rose 6.28%. Hanwha Solutions, which announced a share issue to raise capital, fell 18.22%. A single day's price moves make it hard to isolate TurboQuant's effect on orders at each company.
What to look for in semiconductor earnings reports
To assess HBM demand, it is important to distinguish the volume customers ordered from the volume suppliers actually delivered. In its April 23, 2026 earnings announcement, SK hynix said it increased sales of high-value-added products such as HBM in the first quarter. That quarter also includes the period before TurboQuant's March announcement.
· The models and workloads that operators apply compression to show how widely it has been adopted in their services.
· To compare memory savings, you need measurements of total server memory use at the same request length and with the same number of concurrent requests.
· Company earnings reports let you check separately which periods are covered by actual HBM shipments and which are covered by future supply plans.