Summarization and Compression

When context grows beyond limits, summarization and compression preserve essential information while reducing size. These techniques are critical for long-running chains.

When to Summarize

Summarization is needed when:

• Context approaches window limits
• Steps don't need full previous outputs
• Long documents must be processed
• Conversation history grows too long

Summarization Techniques

Extractive Summarization

Pull key sentences directly from the text:

Abstractive Summarization

Generate new text that captures the meaning:

Structured Summarization

Convert prose to structured format:

// Before: 500 tokens of prose
const proseAnalysis = "The customer expressed frustration about the delayed shipping. They mentioned waiting 3 weeks for an item that was supposed to arrive in 5 days. The customer has been a member since 2019 and has made 47 previous purchases totaling $3,200. They requested either a refund or expedited reshipping. The tone was angry but not abusive...";

// After: 80 tokens of structure
const structuredAnalysis = {
  sentiment: "frustrated",
  issue: "shipping_delay",
  expected_delivery: "5 days",
  actual_wait: "3 weeks",
  customer_tenure: "5 years",
  purchase_history: { count: 47, value: 3200 },
  request: ["refund", "expedited_reshipping"],
  tone: "angry_not_abusive"
};

Compression Strategies

Hierarchical Compression

Compress at multiple levels:

async function hierarchicalCompress(data, targetTokens) {
  // Level 1: Remove redundancies
  let compressed = removeRedundancy(data);
  if (estimateTokens(compressed) <= targetTokens) return compressed;

  // Level 2: Summarize verbose sections
  compressed = await summarizeVerbose(compressed);
  if (estimateTokens(compressed) <= targetTokens) return compressed;

  // Level 3: Extract only key facts
  compressed = await extractKeyFacts(compressed);
  if (estimateTokens(compressed) <= targetTokens) return compressed;

  // Level 4: Ultra-compressed summary
  return await ultraCompress(compressed, targetTokens);
}

Key Fact Extraction

Semantic Compression

Preserve meaning while reducing length:

const semanticCompressionPrompt = `
Compress this text while preserving ALL factual information.
- Remove filler words and redundant phrases
- Convert sentences to concise statements
- Use abbreviations where clear (e.g., YoY for year-over-year)
- Combine related points
- Maintain numerical precision

Original: "${originalText}"
Target: ${targetTokens} tokens maximum

Compressed version:
`;

Context-Specific Summarization

For Conversation History

async function summarizeConversation(messages, maxTokens) {
  return await runPrompt(`
    Summarize this conversation history, preserving:
    - Key decisions made
    - Important facts shared
    - Open questions or concerns
    - Current topic/focus

    Conversation:
    ${messages.map(m => `${m.role}: ${m.content}`).join('\n')}

    Summary (${maxTokens} tokens max):
  `);
}

For Document Processing

For Chain State

async function summarizeChainState(state) {
  const summary = {
    // Always keep
    original_intent: state.original_intent,
    current_step: state.current_step,

    // Summarize
    completed_steps: state.steps
      .filter(s => s.status === 'completed')
      .map(s => ({ step: s.name, result: s.summary || s.result.slice(0, 100) })),

    // Extract key findings
    key_findings: await extractKeyFindings(state.accumulated_results),

    // Errors worth remembering
    errors: state.errors.filter(e => e.severity === 'high')
  };

  return summary;
}

Compression Quality Validation

Information Preservation Check

async function validateCompression(original, compressed) {
  const validation = await runPrompt(`
    Compare the original and compressed versions.
    Identify any important information that was lost.

    Original:
    ${original}

    Compressed:
    ${compressed}

    Lost information (if any):
  `);

  return {
    compressed,
    valid: validation.trim() === '' || validation.includes('none'),
    lostInfo: validation
  };
}

Compression Ratio Tracking

function trackCompression(original, compressed) {
  const originalTokens = estimateTokens(original);
  const compressedTokens = estimateTokens(compressed);

  return {
    originalTokens,
    compressedTokens,
    ratio: compressedTokens / originalTokens,
    saved: originalTokens - compressedTokens,
    percentSaved: ((originalTokens - compressedTokens) / originalTokens * 100).toFixed(1) + '%'
  };
}

Exercise: Build a Compression Pipeline

Design a compression pipeline for a research workflow:

Key Takeaways

• Extractive summarization pulls key sentences verbatim
• Abstractive summarization generates new condensed text
• Structured summarization converts prose to data structures
• Hierarchical compression applies multiple compression levels
• Different content types need different summarization approaches
• Always validate that compression preserves essential information
• Track compression ratios to optimize strategies
• Have fallbacks for critical information loss

Next, we'll explore external memory systems for chains that exceed context limits.