Feed Long Documents So Nothing Gets Lost
Modern AI tools can take in very long documents: contracts, reports, research papers, whole books. That does not mean they read them the way you would. Give a model a 60-page report and ask a question, and it may answer confidently while missing the one paragraph that mattered.
This lesson shows how models handle long inputs, where they tend to slip, and simple techniques that make answers from long documents far more reliable.
What You'll Learn
- Why details in the middle of long inputs get missed
- Where to put instructions when you include a long document
- The extract-then-answer technique for accurate answers
- How to handle documents that are too long, or too many, for one chat
- How to check an answer against the source
Lost in the middle
Research and everyday experience show a consistent pattern: models pay the most attention to the beginning and end of a long context, and less to the middle. A key fact on page 2 or page 58 is likely to be used. The same fact on page 30 is easier to miss.
Nothing is "forgotten" in the usual sense. The text is all there. The model just weighs it less when forming an answer. The longer the input, the stronger the effect.
This has practical consequences:
- Do not assume that because a file uploaded successfully, every part of it was used.
- Put your most important material where attention is strongest.
- For critical questions, point the model to the right part instead of hoping it finds it.
Where to put your instructions
When you include a long document, the order matters.
- 1. Short task summaryWhat you want, in one or two lines
- 2. The documentClearly marked start and end
- 3. Full instructionsRepeat the task with all details and format
Putting the key instruction both before and after a long document helps, because both ends get strong attention. The version at the end is the one the model sees last before answering.
Also mark where the document starts and ends, for example:
=== START OF CONTRACT ===
[paste contract]
=== END OF CONTRACT ===
Clear markers stop the model from mixing your instructions with the document's own text.
Extract, then answer
For accuracy, split one big question into two steps. First, ask the model to pull out the exact passages that are relevant. Then ask it to answer using only those passages.
Prompt:
Step 1: From the document above, copy the exact sentences that are relevant to this question: [question]. Include the section heading or page number for each. Do not summarize yet.
Step 2: Using only the sentences you copied, answer the question. If they do not contain the answer, say so.
This works for three reasons. Searching for quotes makes the model scan the whole document. The quotes bring the key text into fresh focus. And you can check the quotes against the original in seconds.
Too long, or too many
When a document is longer than the tool can take, or you have many documents at once, work in pieces.
Summarize in chunks. Split the document into sections, summarize each one with the same instructions, then combine the summaries.
Prompt for each chunk:
This is part [N] of [total] of a report. Summarize it in 5 bullet points, focusing on [what you care about, for example "risks and deadlines"]. Keep numbers and names exact.
Then combine:
Here are the section summaries of the full report. Write a one-page overview focusing on [what you care about]. Note any section where the summaries disagree.
Keep the question narrow. "Summarize this report" loses detail. "List every deadline and who owns it" keeps it.
Filter first. If only part of a document matters, send only that part. Removing 40 irrelevant pages helps more than any clever prompt.
Small changes in structure make long-document answers much more reliable.
| Criteria | Weak approach | Strong approach |
|---|---|---|
| Instructions | One line before a 50-page paste | Short summary before, full instructions after |
| Question | "What does this say?" | "List every deadline and its owner" |
| Accuracy | Trust the first answer | Extract quotes first, then answer from them |
| Very long input | Paste everything at once | Filter, then work section by section |
Weak approach
- Instructions
- One line before a 50-page paste
- Question
- "What does this say?"
- Accuracy
- Trust the first answer
- Very long input
- Paste everything at once
Strong approach
- Instructions
- Short summary before, full instructions after
- Question
- "List every deadline and its owner"
- Accuracy
- Extract quotes first, then answer from them
- Very long input
- Filter, then work section by section
Always check against the source
Even with good technique, answers from long documents can be wrong. Build checking into the process:
- Ask for locations. "For each point, give the section or page." Then spot-check a few.
- Ask what is missing. "What did the document not cover that I should look for elsewhere?"
- Ask the reverse question. After "what are the risks?", ask "which sections contain no risks?" If the answers do not fit together, look closer.
- Check numbers and names yourself. These are the details most likely to be slightly wrong and most costly when they are.
For important documents, like contracts, medical information, or financial reports, AI is a reading aid, not the final word. Read the parts that matter yourself.
Key Takeaways
- Models pay most attention to the start and end of long inputs. Details in the middle are easier to miss.
- Put a short task summary before the document and full instructions after it, with clear start and end markers.
- Use extract, then answer: pull exact quotes first, then answer only from them.
- For very long or many documents, filter first, work section by section, and ask narrow questions.
- Ask for page or section locations and check key facts yourself, especially for important documents.