Types of Chain Failures
Understanding the different ways prompt chains can fail is essential for building robust systems. Each failure type requires a different handling strategy.
Failure Categories
Prompt chain failures fall into four main categories:
┌─────────────────────────────────────────────────────────────┐
│ CHAIN FAILURES │
├─────────────────┬─────────────────┬─────────────────────────┤
│ API/System │ Content │ Logic/Flow │
│ Failures │ Failures │ Failures │
├─────────────────┼─────────────────┼─────────────────────────┤
│ • Rate limits │ • Wrong format │ • Invalid transitions │
│ • Timeouts │ • Missing data │ • Infinite loops │
│ • Network errors│ • Hallucinations│ • Dead ends │
│ • Token limits │ • Off-topic │ • State corruption │
└─────────────────┴─────────────────┴─────────────────────────┘
API and System Failures
Rate Limiting
When you hit API rate limits:
// Example error
{
"error": {
"type": "rate_limit_exceeded",
"message": "Rate limit exceeded. Please retry after 60 seconds."
}
}
Symptoms: 429 status codes, specific error messages Impact: Step cannot execute Recovery: Exponential backoff, request queuing
Timeouts
When a step takes too long:
// Timeout scenario
try {
const result = await runStep(input, { timeout: 30000 });
} catch (error) {
if (error.code === 'TIMEOUT') {
// Step took longer than 30 seconds
}
}
Symptoms: No response within expected time Impact: Chain stalls, resources held Recovery: Retry with longer timeout, use cached result, skip step
Token Limit Exceeded
When input or output exceeds model limits:
Symptoms: API error about token limits Impact: Step fails completely Recovery: Chunk input, summarize first, use larger context model
Content Failures
Wrong Output Format
When the AI doesn't follow format instructions:
Expected:
{ "sentiment": "positive", "score": 0.8 }
Actual:
"The sentiment is positive with a high score."
Symptoms: JSON parse errors, missing fields Impact: Next step receives invalid input Recovery: Retry with stricter instructions, parse flexibly, use format validation
Missing or Incomplete Data
When extraction or analysis is incomplete:
Symptoms: Fewer results than expected, empty fields Impact: Downstream steps lack data Recovery: Second-pass extraction, human review queue
Hallucinations
When the AI generates false information:
Examples:
- Citing non-existent sources
- Adding details not in the input
- Making up statistics
Symptoms: Information doesn't match source, implausible claims Impact: Incorrect final output, trust issues Recovery: Fact-checking step, confidence thresholds, human review
Off-Topic or Irrelevant Output
When the AI doesn't address the actual task:
Symptoms: Output doesn't match request, tangential content Impact: Wasted processing, chain produces wrong result Recovery: Relevance validation, retry with clearer instructions
Logic and Flow Failures
Invalid State Transitions
When the chain reaches an impossible state:
// Example: Order processing chain
state = { status: "delivered" };
// Invalid: Can't cancel a delivered order
nextStep = "cancel_order"; // Should fail
// Valid transitions from "delivered"
validTransitions = ["return_initiated", "feedback_requested"];
Symptoms: Unexpected state combinations, assertion failures Impact: Chain in undefined state Recovery: State validation, rollback to known state
Infinite Loops
When a chain never terminates:
Step: Improve → Evaluate → (not good enough) → Improve → Evaluate → ...
Symptoms: Ever-increasing step count, no termination Impact: Resource exhaustion, cost explosion Recovery: Maximum iteration limits, early stopping criteria
Dead Ends
When a chain reaches a state with no valid next action:
// No handler for this case
if (analysis.category === "unknown") {
// What now? Chain has no path forward
}
Symptoms: Chain halts with no error, no output Impact: Incomplete processing Recovery: Default handlers, explicit "unknown" paths
Failure Detection Strategies
Structural Validation
Check output structure matches expectations:
function validateOutput(output, schema) {
const errors = [];
for (const [field, requirements] of Object.entries(schema)) {
if (requirements.required && !(field in output)) {
errors.push(`Missing required field: ${field}`);
}
if (field in output && typeof output[field] !== requirements.type) {
errors.push(`Wrong type for ${field}: expected ${requirements.type}`);
}
}
return { valid: errors.length === 0, errors };
}
Semantic Validation
Check output makes sense:
Consistency Checks
Verify outputs are internally consistent:
// Check for contradictions
if (output.sentiment === "positive" && output.score < 0) {
throw new Error("Inconsistent: positive sentiment with negative score");
}
// Check for impossible combinations
if (output.status === "completed" && !output.completion_date) {
throw new Error("Inconsistent: completed without completion date");
}
Exercise: Classify Failures
For each scenario, identify the failure type and suggest a recovery strategy:
Key Takeaways
- Failures fall into three main categories: API/System, Content, and Logic/Flow
- Each failure type has characteristic symptoms and recovery strategies
- API failures often need retry with backoff
- Content failures need validation and reformatting
- Logic failures need bounds checking and default paths
- Detection requires both structural and semantic validation
- Plan for failures during chain design, not after deployment
Next, we'll explore specific strategies for validating outputs between steps.
التالي · الدرس 12 من 43
Output Validation Strategies
قراءة 3 دقيقة
أنهيت 11 من 43 درسًا

اسأل Byte عن هذا الدرس
محادثة خاصة حول هذا الدرس. قرأ Byte الدرس كاملًا، ويجيب خلال ثوانٍ، ويُظهر القسم الذي تأتي منه كل إجابة. يمكنك السؤال بأي لغة.
الأسئلة والأجوبة
أسئلة عامة حول هذا الدرس. يجيب AI Tutor فورًا بلغتك، ويراجع فريق FreeAcademy الإجابات، ويمكن لكل المتعلمين قراءتها.

