Long conversations

A long chat is convenient, but it is not ideal as the default way of working. A language model has a limited context window, that is, a limited working memory that includes your prompt, previous messages, and part of the response currently being generated. The exact size depends on the model and is usually expressed in tokens, not words.

That is why a simple rule applies in practice: keep conversations short and topically clean. When things start to branch out, start a new chat.

Practical recommendation

use one chat for one topic or one work branch,
when the topic changes, start a new conversation,
if you want to compare two variants, open two separate conversations,
keep an old conversation only when its history genuinely matters.

Why long chats worsen the result

Each additional exchange gradually fills up the context window. As the amount of text grows, so does the risk that the model overlooks a detail, mixes up older and newer instructions, or starts answering less accurately. A larger context is therefore not automatically better. What matters is that it is relevant and clear.

A long conversation moreover often contains old detours, dead-end attempts, and intermediate steps that are no longer important for the current query. The model, however, still sees them as part of the workspace, which increases the noise.

What happens when the context fills up

Once a conversation approaches the limit, the application on top of the model has to do something with the older history. "Exceeding" the limit does not mean that the model can suddenly hold more text. In practice, therefore, history truncation, summarization, or compaction into a denser form are mainly used. This preserves the main direction of the conversation, but not necessarily every wording, minor detail, or intermediate step.

In other words: a long chat is not an archive with perfect memory. It is a workspace with limited capacity.

When to start a new chat

It is advisable to start a new conversation especially when you:

change the topic,
branch off into a different legal or business question,
need to develop an alternative variant,
want a shorter and more manageable context without old ballast.

Brief technical explanation

The context window is the maximum amount of text the model has available at a given moment. It includes the prompt, previous messages, and the text the model is currently producing.

Primary sources:

A long context is not automatically better. When too much older or no longer relevant information remains in it, the model may have a harder time navigating it.

Technical background:

OpenAI states that each model has a maximum combined token limit for input and output.
If you are approaching the limit, the recommended approach is to shorten the prompt, split a larger text into smaller parts, or summarize the inputs beforehand.

Primary sources:

When a session approaches the limit, some systems use compaction: they truncate older history, summarize it, or keep only the most important part.

Technical background:

This is a general technical explanation of how some AI systems handle long sessions.
For everyday work in CODEXIS AI, a simple rule follows from this: keep conversations short and start a new chat when the topic branches.

The size of the context window is usually expressed in tokens, not words. A token can correspond to a word, part of a word, a character, or a byte.

Primary source:

OpenAI Help - What are tokens and how to count them?

Supplementary technical sources for completeness

The following links are supplementary. They do not directly describe the CODEXIS AI product configuration on the GPT-5.6 models, but they are useful for deeper technical background on working with a long context and on history compaction.