A model says it has a 1 million token context window. You pay for 1 million tokens. You feed it 1 million tokens. It confidently produces an answer.
The answer is wrong. Not because the model is dumb. Because the model never really read most of what you sent it.
This is the dirty secret of long-context AI in 2026: the gap between what a model accepts and what a model can reliably use is enormous, and almost nobody in marketing pages is being honest about it.
Llama 4 Scout advertises a 10 million token context window and scores 15.6% on long-context reasoning. Claude Opus 4.7, the current frontier, advertises 1 million tokens and scores 32.2% on MRCR v2 at that length. Gemini 3.1 Pro advertises 2 million tokens and scores 26.3%. These are the leaders. The rest of the market sits below them.
If you're building anything that depends on a model actually using a long input, this is the most important thing to understand right now. The token count on the spec sheet is a marketing number. The number that matters is the effective context length, and almost nobody publishes it.
The race nobody is winning
The context window race has been going on for three years. In 2023 the frontier was 32K. In 2024 it was 200K. In 2025 every major lab claimed a million. By early 2026, Llama 4 Scout said 10 million, Gemini 3 Pro said 10 million, and Subquadratic, a Miami startup that exited stealth on May 5, claimed 12 million.
These numbers are technically true. The models will accept that many tokens as input. The API won't error out. You'll get billed for the full prompt.
What the numbers don't tell you is whether the model actually attends to the tokens past a certain depth, integrates information from different positions in the input, or reasons over the whole input the way it reasons over a short one.
The answer, on every benchmark that tests for this, is: not really. Not at the advertised length. Often not anywhere close.
Research published throughout 2024 and 2025 keeps finding the same pattern. The effective context length, the length at which a model still performs roughly as well as it does on short inputs, is typically 50 to 70 percent of the advertised maximum. For most open-source models it's less than half. The original RULER paper showed Llama 3.1 70B, marketed as a 128K-token model, has an effective context length of just 64K.
So when you read "1 million tokens" on a model card, the honest translation is closer to "300K to 700K tokens, depending on what you're asking it to do, with a steep drop-off after that." When you read "10 million tokens," the honest translation is often "this works in a press release."
What the benchmarks actually measure
There are three benchmarks worth knowing about. They test different things, and the spread between them is the spread between marketing and reality.
Needle-in-a-Haystack (NIAH) is the easiest. It hides a single fact in a long document and asks the model to find it. Most modern frontier models score above 90% on this at any context length they advertise. This is the benchmark labs love to cite. It is also the benchmark that least resembles real work.
RULER extends NIAH with thirteen tasks across four categories: retrieval, multi-hop tracing, aggregation, and question answering. Models that ace NIAH routinely fall apart on RULER's harder tasks at the same context length. RULER is what introduced the term "effective context length" into mainstream LLM evaluation.
MRCR v2, or Multi-Round Coreference Resolution version 2, is the one that matters most for production work. Built by OpenAI, it gives the model a long synthetic conversation and asks it to retrieve specific instances of a repeated request. The model has to disambiguate. It has to track which version of "the third email about the budget" you mean. The 8-needle variant requires tracking eight things at once. This is the closest public benchmark to real long-context work, finding multiple non-adjacent pieces of evidence in a large input and integrating them into one coherent answer.
If a model is good at MRCR v2 at the length you actually plan to use, it can probably do your job. If it isn't, it can't.
The numbers nobody puts on the model card
Here's where it gets uncomfortable. Frontier scores on MRCR v2 vary dramatically by both context length and needle count, and labs almost always report the configuration where they look best.
At 128K context with 8 needles, Claude Opus 4.6 leads at 93%. GPT-5.5 sits at 74%. Gemini 3.1 Flash-Lite at 60%. So far, so respectable.
Move to 1 million tokens with 8 needles and the numbers collapse. Claude Opus 4.6 drops to 76%, still the leader, but a full 17 points lower. Claude Sonnet 4.5 was at 18.5% on the same test before Opus 4.6 quadrupled it. The leap from Sonnet to Opus is celebrated as a four-fold improvement; the more honest framing is that one of the most expensive models on the market moved from "actively bad at long context" to "merely mediocre."
The numbers Subquadratic used in its launch press are even rougher. They list Claude Opus 4.7 at 32.2% on MRCR v2 at 1 million tokens. Gemini 3.1 Pro at 26.3%. GPT-5.5 at 74%. Whether or not you trust Subquadratic's own model claims, those competitor numbers, which they have no incentive to deflate, match the broader pattern. At a real million tokens, on a real multi-needle test, frontier models are scoring in the twenties and thirties, not the high nineties they post on launch day.
Then there's Llama 4 Scout. Meta launched Scout in April 2025 with a 10 million token context window as the headline feature. On the multi-needle long-context tests since published, it scores 15.6%. A model with the largest advertised context window of any open-source frontier release scores worse at long context than models with one-fortieth its window size.
Bigger window, worse retention. The trend isn't subtle.
Why this happens
This isn't a failure of any one lab. It's a structural property of how transformers learn long context, and there are at least three things working against you.
- Position embeddings degrade. Every transformer encodes where each token sits in the sequence. Most use rotary position embeddings (RoPE) or variants. These were trained on shorter sequences and don't generalize cleanly to positions the model rarely saw during training. Research throughout 2024 documented a "left-skewed position frequency distribution" where positions deep in the sequence are simply undertrained. The model has technically seen token 700,000 of a million-token input, but it doesn't know what to do with it.
- The lost-in-the-middle effect. Models reliably attend to the start and end of an input and lose the middle. Stuff your most important fact at token 500,000 in a 1M-token prompt and the model's recall drops sharply versus the same fact at token 50,000 or token 950,000. This problem doesn't shrink as windows grow. It amplifies, because there's more middle.
- Attention dilutes. Standard attention is dense, every token compares to every other token. As input gets longer, the attention any single token receives from any other gets thinner. Without a mechanism that selects which tokens to focus on, the model spreads its attention budget across millions of comparisons it doesn't actually need to make. Sub-quadratic architectures like the one Subquadratic claims are an attempt to fix this, but most production frontier models still use the brute-force version.
These aren't bugs labs forgot to fix. They're trade-offs nobody has fully solved. Every approach so far has improved one of these properties at the cost of another.
What this means if you're building
If you ship anything that runs on long inputs, agents, code assistants, research tools, document analysis, customer support over chat history, this changes how you should design your stack.
- Don't trust the spec sheet. The advertised context window tells you the maximum you can submit, not the maximum the model can use. Treat it as a ceiling, not a working number.
- Run your own MRCR-style test. Build a benchmark from your actual workload. Hide three to five things the model needs to find in a representative document. Measure recall at 50K, 200K, 500K, 1M. The drop-off is your real working budget for that model.
- RAG is not dead. A lot of the marketing in 2025 said retrieval-augmented generation would become unnecessary as windows grew. The benchmarks say the opposite. Models reason much more reliably over a small, well-chosen retrieval result than over the same information buried in a million-token haystack. Building good retrieval is still the highest-leverage investment for most long-context products.
- Position matters. If you have to use a long context, put critical information at the start or the end of the prompt, not the middle. This is the cheapest performance improvement you'll find.
- Watch for benchmark version games. When a lab reports MRCR, check whether it's 2-needle or 8-needle, what context length, and at what depth in the haystack. The same model can look outstanding at 128K with 2 needles and unusable at 1M with 8 needles, and the press release won't tell you which one they ran.
- Be skeptical of any model that doesn't publish these numbers. If a launch announcement claims million-token capability and only shows needle-in-a-haystack scores, the lab is hiding something. The benchmarks that matter exist. Every serious lab is running them internally. If they're not on the model card, ask why.
What to actually use right now
For long-context work in May 2026, here's the honest read.
- Claude Opus 4.6 and 4.7 are the best for multi-needle retrieval at 1M. The numbers aren't perfect, but they're the highest published.
- GPT-5.4 and 5.5 are the best for high-precision retrieval of specific facts across long context. Near-perfect single-needle accuracy, weaker on multi-needle.
- Gemini 3.1 Pro has the largest reliable window for bulk processing at the lowest per-token cost, but accuracy drops past 500K.
- Open-source models with 10M-token claims should be treated as 200K-token models for serious work until proven otherwise on independent multi-needle tests.
This will keep changing. The labs are working on it. But the right mental model for the next two quarters is that advertised context is marketing and effective context is engineering, and your job as a builder is to know the difference.
The bottom line
Every major AI lab has been rewarded for one number, the size of the advertised context window, and the market has barely started measuring the number that actually matters. The result is a frontier full of models that nominally accept a million tokens and functionally use a fraction of that.
The labs are not lying, exactly. The numbers on the spec sheets are technically real. But they are not the numbers you should be designing your product around, and any vendor pitching you on "we have a longer context window" without showing you their MRCR v2 score at the length you actually plan to use is selling you a feature they haven't actually built.
The next phase of long-context AI won't be about stretching the window. It'll be about closing the gap between the window and the working memory. Whoever wins that race wins the next generation of agentic products. Whoever keeps shipping bigger and bigger empty rooms gets caught when the benchmarks finally catch up.
For now, run your own tests. Trust MRCR v2 over needle-in-a-haystack. Treat any "10 million token" claim like a magic trick until somebody shows you the dropped score.
And don't pay for tokens your model can't read.