Ajay Kandakatla
A vast endless library with shelves stretching into infinite darkness, a single glowing book on a central pedestal illuminating only a small radius of floor — the mirage of context
A vast endless library with shelves stretching into infinite darkness, a single glowing book on a central pedestal illuminating only a small radius of floor — the mirage of context

The Million Token Mirage

I just spent the last week stress-testing the new 1M context window against the standard 200k limit. The performance is undeniably impressive—the model handles massive, sprawling datasets with a level of coherence that previously felt like science fiction. But there is a hidden cost to this expansion, and it isn’t just the latency or the obvious API price increase.

When you move from 200k to 1M tokens, you aren’t just getting more room; you are losing the discipline of optimization. We are witnessing a fundamental shift in how we prompt, and it is making our workflows significantly more bloated.

The Death of Precision

With a 200k limit, you are forced to be an architect. You have to curate your context. You prune the noise, you summarize the irrelevant parts of a codebase, and you feed the model exactly what it needs to execute the task at hand. This constraint is a feature, not a bug. It forces a level of intentionality that results in high-density, high-signal prompts.

Once you open the floodgates to 1M tokens, that intentionality vanishes. I find myself dumping entire directories, massive documentation files, and sprawling logs into the context window without a second thought. The friction that used to prevent me from being sloppy has been removed.

The result is a massive increase in token consumption that feels disproportionate to the actual value being delivered. I am seeing usage spikes that are three or four times higher than what I would require if I were still working within the 200k boundary. We are trading surgical precision for brute-force ingestion, and it is making our prompting habits lazy.

The Efficiency Paradox

There is a pervasive myth in the LLM space that “more context equals better reasoning.” This is false. More context equals more data, and more data increases the statistical probability of noise interfering with the signal.

When I run a task through the 1M window, the model’s ability to find the “needle in the haystack” remains high—the retrieval is excellent. However, the sheer volume of tokens I am burning through to achieve that result is staggering. I am paying a massive premium in token count for the convenience of not having to organize my data.

If I want to debug a specific module, I no longer bother stripping away the boilerplate or the unrelated utility functions. I just throw the whole repo at it. The model succeeds, but the economics of that success are questionable. I am using roughly three times the tokens to solve the same problem I could have solved with a highly optimized 200k prompt. We are effectively subsidizing developer laziness with massive context windows.

The Optimization Gap

The most frustrating realization is that the optimization logic seems to have evaporated alongside the context constraints. In a 200k world, every token counts. You learn to structure your instructions to be lean because you know the ceiling is low.

In the 1M world, the model behaves as if it has infinite headroom, which leads to a degradation in how we structure our logic. We stop thinking about the hierarchy of information. We stop prioritizing the most critical context at the top or bottom of the prompt because we assume the model will just “find it.”

This creates a feedback loop:

  1. The context window expands.
  2. The developer stops optimizing the input.
  3. Token consumption scales linearly (or worse) with the window size.
  4. The cost-to-value ratio begins to plummet.

The “value” of the 1M window is real in terms of capability, but it is being undermined by a lack of systemic efficiency. We are building applications that are fundamentally bloated because the hardware/model capability allows us to ignore the fundamental principles of information theory.

The Cost of Convenience

We need to stop treating context windows as a measure of intelligence and start treating them as a measure of capacity. Capacity is cheap; intelligence is expensive.

If you are building a production system, the temptation to use the 1M window for everything will be overwhelming. It is easier to pipe a whole database schema into the prompt than it is to write a sophisticated RAG (Retrieval-Augmented Generation) pipeline or a precise summarization layer. But this is a trap.

Using 1M tokens to do a job that requires 50k is not “leveraging the model’s power.” It is an engineering failure. We are moving toward a world where “just throw more data at it” becomes the default solution for every complex reasoning task, and that is a recipe for unscalable, overpriced software.

Are we actually getting smarter at prompting, or are we just getting better at being inefficient?