Long-document LLM best practices: full context, chunks or retrieval

A larger context allowance does not tell you how to organize a document task. For GPT, Claude or Gemini, first decide whether the user needs a global synthesis, a complete inventory or an answer about a small part of the source. This is an application-design comparison, not a context-window ranking or an assertion that a retrieval service is built into the platform.

Match the strategy to the question

Strategy	Useful starting situation	Main failure to test	Additional work
Full context	A bounded document needs cross-section reasoning	Important details are overlooked despite being present	Measure request size and validate completeness
Chunk and merge	Every section needs the same extraction pass	A relation crosses a chunk boundary or duplicates appear	Stable source IDs, overlap policy and merge rules
Retrieve then answer	Many questions target small parts of a larger corpus	Relevant evidence never reaches the model	Indexing, retrieval evaluation and access filtering

None is automatically cheapest or most accurate. Full context can resend large inputs; chunking adds requests and a merge pass; retrieval introduces infrastructure and can omit crucial evidence. Use actual application and model charges when comparing total cost.

Build a source manifest before chunking

Keep document ID, version, section ID and original ordering. Mark removed boilerplate explicitly if that change could affect interpretation. For tables, keep headers with rows and units with values. A broken table split can change meaning without producing a syntax error.

The following is a local application record, not an LLM API request or a platform upload endpoint:

{
  "document_id": "release-plan-demo",
  "version": "v1",
  "chunks": [
    {"id": "section-a", "order": 1, "text": "Cedar is planned for 12 November. This date is provisional."},
    {"id": "section-b", "order": 2, "text": "The final date depends on the accessibility review in section C."},
    {"id": "section-c", "order": 3, "text": "The accessibility review has not completed."}
  ]
}

For a question about the confirmed release date, section A alone is insufficient. A full-context answer must preserve the provisional status. Chunking must retain the cross-reference to section C. Retrieval must fetch the dependency, not just the passage with the matching date. This fixture demonstrates a failure mode; it is not a model result.

Reserve room for the actual response

Count or estimate the entire request in the selected model’s units: task instructions, document text, conversation history and any enabled tool definitions. Character count is not a universal token count. Check the platform’s current input/output constraints and leave room for the requested output; upstream headline limits do not establish your effective route limits.

Use the native output field: max_completion_tokens for the documented GPT Chat route, max_tokens for Claude Messages, or generationConfig.maxOutputTokens for Gemini content requests. Tokenizer and reasoning differences mean equal numeric limits are not equal visible output budgets. Review native finish state rather than assuming a syntactically valid table is complete.

Merge evidence, not just prose summaries

Ask each chunk for findings linked to source IDs and unresolved cross-references. Deduplicate only when evidence describes the same fact. Keep contradictory statements visible for a later review pass. A summary of summaries can erase exceptions; a compact evidence table is often easier to audit. See the Claude evidence workflow for row-level verification.

For retrieval, test whether the needed source passages were actually selected before judging the generator. Score retrieval misses separately from answers that ignore supplied evidence. Apply document permissions before sending retrieved content to the model; a helpful answer must not include content the user cannot access.

Compare workflows on boundary cases

Use fixtures with cross-section dependencies, repeated names, contradictory dates, tabular units and facts near chunk boundaries. Record answer acceptance, missed sources, number of calls, reviewer time and total observed cost. Keep indexing and retrieval expenses separate from model usage, and do not assume caching discounts without receipts.

Start with a small approved sample using the evaluation worksheet, then follow cost reconciliation. This guide runs no benchmark. Model identities and protocol references were checked 2026-11-03 against GPT-6.1 Sol, Claude models, Gemini models and the platform protocol guide; no fixed context capacities are promised here.