In simple terms, it's the model's working memory. Don't think "brain." Think whiteboard. Whatever's on the board, the model sees. Whatever isn't, doesn't exist for it. Session ends, board gets wiped.

What's it actually for?

Everything the model does happens inside it. It reads your question there, reasons there, writes the answer there. It's the playing field.

And here's what most people miss: the model doesn't remember your last chat by itself. Every time you hit send, the app packs the whole conversation back into the window and sends it again. That's why long chats get slower and heavier.

Tokens, not words

The window is counted in tokens. A token is a chunk of text: sometimes a whole word, sometimes just a piece of one. For English, it's about 1.3 tokens per word, or roughly 0.75 words per token. So a 128k window fits around 96,000 words.

Why do providers start small?

Cost. Attention compares every token with every other token, so compute grows roughly with the square of the length. Double the window, and that part costs about four times more.

So nobody jumps straight to a huge number. Providers went 4,096, then 8,192, then 16,384, and kept climbing to 128k, 200k and beyond as hardware and techniques got better. Also, many models are trained on shorter sequences first and stretched to longer ones later. They don't start at full length.

Input and output share the same space

The window holds what you send (input) and what the model writes (output). Together, they have to fit. There's also a separate cap on output, which is much smaller than the input limit.

Example: 128k window, you ask for a 30k-token answer. You've got about 98k left for input. Simple math, but people forget it.

So is the limit fixed?

Yes, it's a hard wall. Go over and one of two things happens: you get an error, or the app quietly chops off the oldest content. The model doesn't slowly get dumber after crossing the line. It just can't see what didn't fit.

The real problem is inside the window. The fuller it gets, the worse things get. Models miss details buried in the middle of long inputs (people call it "lost in the middle"), responses slow down, and the bill goes up. Same task at 20k tokens often beats it at 120k. A big window is a limit, not a target.

What to do about it

• Send only what's relevant. Not everything you have.
• New topic, new chat.
• Long conversation? Summarize it and carry the summary forward.
• Big document set? Use retrieval (RAG) so only the useful pieces get in.
• Put key instructions at the start or the end. Not the middle.

Bottom line

The context window is working memory. Hard limit, shared between input and output, expensive to scale, and less reliable the fuller it gets. Bigger helps. Using it smartly helps more.

Hi, Welcome to my blog. In this blog we will be documenting my whole approach of solving a problem scratch. I am AI Engineer. So my problem statements will be related to it.The problem statement which i am trying to solve and the approach that i am following will be posted in the upcoming blogs as i solve it daily.

What is the Problem Statement?

we all know that closed source models are good at agentic coding and they are always has been on the front page of any news that we watch like youtube, Blogs etc.,. But the Problem has always been privacy, security, lock-in to one closed source AI tool like claude, chatgpt, gemini. Also, along with closed source we have opensource models like qwen, glm etc., but these opensource models require good compute power which needs a expensive system which generally people are not able to get. Now here comes the problem statement, we do have opensource 7B+ parameters models available that fit a gaming laptop easily. So, why can't we use them for agentic coding. I know that many people has done the research about them. That, they are not good at it because they are less capable to use for agentic coding and another issue is context size as gaming laptop in general has about 16gb ram. This is my whole problem statement which i want to solve to know with same approach why can't we build one that uses 7B parameter models for agentic coding. That's where my project comes which is autonomous system.

what is autonomous system?

we all know what is autonomous system but do we know that what are the general drawback that the autonmous systems have when we have opensource models added to them as a brain. That's where i am trying to implement with proper structure and proper deliverables of the use case of 7B Models for autonomous system with agents.

Workflow

My workflow involves using an orchestrator to call agents and these agents call the respective tools, mcp server accordingly. So basically when we give a query to the system the orchestrator picks it up and sends it to the respective agents and from the agents do the task.