TurboQuant 6x KV Cache Compression and HBM Demand Signals
In an experiment on finding information in long texts, Google reduced the KV cache to at most one-sixth of its original size. The impact on demand for HBM purchases depends on how much of the memory the cache accounted for and how many more requests can be handled at once after compression.
- Google's figure of at least 6x compression, announced in March 2026, came from measuring the KV cache that language models use to generate answers.
- If the cache is reduced to exactly one-sixth of its original size, 6 requests of the same length take up as much cache space as 1 request did before compression.
- On March 26, 2026, Samsung Electronics fell 4.71% and SK hynix fell 6.23% from the previous trading day.
- To see how HBM demand is changing, look at both how many requests the service handles at once and how much memory suppliers actually provide.
In experiments, Google's TurboQuant reduced the KV cache used to generate AI answers to at most one-sixth of its original size. This reduces the memory needed per request. Total demand for high-bandwidth memory (HBM) depends on how many more requests are handled at the same time.
The compression figures come from Google Research's March 24, 2026 announcement, and the stock prices are based on the March 26 close.
What is the KV cache that TurboQuant compresses?
The KV cache is temporary storage that a language model uses to refer back to earlier context. The model processes text by breaking it into units called tokens. By storing the keys and values (KV) it calculates for each token, the model can avoid repeating those calculations when it generates the next token.
The longer the conversation, the more keys and values the model needs to store. TurboQuant represents these numbers with fewer bits. This is called quantization. The weights the model learned during training still take up memory after the cache is compressed.
What did the 6x compression and 8x speed figures measure?
The two figures Google announced refer to storage space and the speed of a specific calculation, respectively. Google Research's March 24, 2026 description of the experiments explains what each figure was compared against.
| Reported figure | Test conditions | What was measured |
|---|---|---|
| At most one-sixth of the original size | A task that involved finding information hidden in a long text | The size of the reduced KV cache while maintaining performance at finding the correct answer |
| Up to 8x the speed | A comparison of 4-bit TurboQuant keys with uncompressed 32-bit keys on an H100 GPU | How quickly the model calculated the scores that determine how much weight to give earlier tokens |
Answer quality after compression
How well quality holds up also depends on how many bits are assigned to each number. According to the research paper, answer quality held up in evaluations even at an average of 3.5 bits per channel. At an average of 2.5 bits, quality declined slightly. Response times and answer quality in a live service need to be tested using the questions that service handles.
A worked example of concurrent requests and memory capacity
If the cache is reduced to exactly one-sixth of its original size, 6 requests of the same length use as much cache as 1 request did before compression. This calculation uses 6x, the lower bound of the compression ratio reported in the experiments. It assumes the same model and request length and that each request's cache is stored separately.
| Requests handled at the same time | Total KV cache size after compression |
|---|---|
| 1 request | One-sixth of the cache for 1 request before compression |
| 6 requests | The same cache size as for 1 request before compression |
| More than 6 requests | A larger cache than for 1 request before compression |
To calculate the savings for the whole server, you need to know how many bytes the cache used before compression. Under the conditions above, the space saved is five-sixths of the original cache size. The smaller the cache's share of total memory, the smaller the percentage saved across the server.
When HBM demand could fall or rise
Demand for HBM purchases depends on how service operators use the space they save. HBM capacity is the amount of data it can store at one time. Bandwidth is the amount of data it can move in a given time. To estimate how much HBM operators will buy based on the cache sizes measured in the experiments, you need to know whether capacity or bandwidth was the constraint driving the need for more servers.
| Service operating conditions | Possible effect on HBM demand |
|---|---|
| Request volume and length stay the same, and the operator was planning to add servers because it was running out of cache space | Freeing up space on existing equipment could delay further purchases. |
| The operator handles more requests at the same time on the same equipment | The space saved can be used to cache new requests. |
| Lower costs lead to a sharp rise in the use of services that process long documents | Total memory use can rise if the added workload outweighs the savings per request. |
Jevons paradox is what happens when improved efficiency leads to an even larger rise in usage, increasing total consumption of a resource. For AI services, the key question is how much actual usage rises after processing costs fall. When assessing memory makers' revenue, shipment volumes and selling prices also need to be considered separately.
Why semiconductor stocks fell on March 26
On March 26, 2026, concerns about slowing memory demand coincided with uncertainty in the Middle East. The Bank of Korea cited both factors in explaining that day's stock market decline. The won-dollar exchange rate ended daytime trading at 1,507.0 won per dollar, up 7.3 won from the previous day.
| Index or stock | Change on March 26, 2026, from the previous trading day | Index close |
|---|---|---|
| KOSPI | -3.35% | 5,460.46 points |
| KOSDAQ | Down in the 2% range | 1,136.64 points |
| Samsung Electronics | -4.71% | Not applicable |
| SK hynix | -6.23% | Not applicable |
Samsung Securities put net selling by foreign investors on the KOSPI market at 3.111 trillion won at the time of publication. Alteogen, which disclosed a technology export that day, rose 6.28%. Hanwha Solutions, which announced a share issue to raise capital, fell 18.22%. A single day's price moves make it hard to isolate TurboQuant's effect on orders at each company.
What to look for in semiconductor earnings reports
To assess HBM demand, it is important to distinguish the volume customers ordered from the volume suppliers actually delivered. In its April 23, 2026 earnings announcement, SK hynix said it increased sales of high-value-added products such as HBM in the first quarter. That quarter also includes the period before TurboQuant's March announcement.
- The models and workloads that operators apply compression to show how widely it has been adopted in their services.
- To compare memory savings, you need measurements of total server memory use at the same request length and with the same number of concurrent requests.
- Company earnings reports let you check separately which periods are covered by actual HBM shipments and which are covered by future supply plans.
FAQ
Can TurboQuant cut the number of HBM chips needed from 6 to 1?
The compression ratio of at least 6x that Google measured applies to the size of the KV cache. To estimate how many HBM chips are needed, you need to know the total memory requirements, including the model weights, as well as the bandwidth requirements.
Does TurboQuant also reduce the memory needed for AI training to 1/6 of the original amount?
That figure comes from measurements of the KV cache that a trained language model uses to find information in long texts. This experiment cannot be used to calculate the percentage reduction in the total memory needed to train a model.
Does the figure of up to 8x also apply to how long it takes to get a response?
The research team compared how quickly an H100 GPU could calculate the scores that determine how much attention to pay to earlier tokens. To measure the overall response time, you need to time the process from the start of input processing to the end of output generation.
Does compressing the cache preserve response quality in long conversations?
Google's research team found that the model was still just as good at finding the correct answers on a test that involved locating information hidden in long texts. The research paper reported a slight drop in quality when the average was reduced to 2.5 bits per channel.
What can TurboQuant be used for besides compressing the KV cache in language models?
The research team also applied TurboQuant to vector search, which finds items with similar meanings. In this case, they compress the numbers that represent the features of the items being searched, then evaluate how well the search finds the desired items.
Was the TurboQuant paper first made public in March 2026?
The researchers first posted the paper on arXiv on April 28, 2025. Google Research presented the findings on March 25, 2026 (local time).
Can the March 26 stock price drop confirm that HBM orders fell?
Stock prices that day reflected both concerns about future memory demand and the situation in the Middle East. Any actual changes in orders need to be verified by looking at supply contracts with customers or HBM shipment figures for subsequent quarters.
Sources
- Google Research, TurboQuant: Redefining AI efficiency with extreme compression
- TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
- Samsung Securities, March 26, 2026 Domestic Market Closing Report
- Bank of Korea, Key Financial Market Indicators (3.27), Trading Data for March 26, 2026
- SK hynix, 2026 Q1 Earnings Release
Images

