Verify an AI Fix Before You Trust It
AI assistants are confident. They present a fix with a clear explanation, and it usually looks right. Often it is. But sometimes the fix only works for the one example you showed it, hides the real problem, or uses a function that does not exist.
The habit that separates people who learn to code with AI from people who get stuck with AI is simple: verify every fix before you trust it. This lesson shows you how, and ends with an exercise where an AI-style fix passes the obvious test but fails at a boundary.
What You'll Learn
- The most common ways AI fixes go wrong
- The difference between fixing a symptom and fixing a cause
- How to test a fix with normal, edge, and boundary cases
- How to spot invented functions and settings
- A short checklist to run after every AI fix
How AI fixes go wrong
Each failure has a quick test that catches it.
| Criteria | What happens | How to catch it |
|---|---|---|
| Fixes only your example | Works for the input you showed, fails for others | Test with several different inputs |
| Hides the symptom | Wraps the error in a try/except or check so it disappears, but the wrong value remains | Ask: why did the bad value appear in the first place? |
| Invents a function | Uses a method or setting that does not exist | Run it, and check the official documentation |
| Rewrites too much | Changes names and structure, adds new bugs | Ask for the smallest change only |
| Wrong boundary | Uses > where >= was needed, or the reverse | Test exact boundary values |
What happens
- Fixes only your example
- Works for the input you showed, fails for others
- Hides the symptom
- Wraps the error in a try/except or check so it disappears, but the wrong value remains
- Invents a function
- Uses a method or setting that does not exist
- Rewrites too much
- Changes names and structure, adds new bugs
- Wrong boundary
- Uses > where >= was needed, or the reverse
How to catch it
- Fixes only your example
- Test with several different inputs
- Hides the symptom
- Ask: why did the bad value appear in the first place?
- Invents a function
- Run it, and check the official documentation
- Rewrites too much
- Ask for the smallest change only
- Wrong boundary
- Test exact boundary values
Symptom vs cause
A symptom is what you see: an error, a crash, a wrong number. The cause is the reason it happened.
Say your code crashes because a value is None. A symptom fix adds a check: "if the value is None, skip it." The crash disappears. But why was the value None? Maybe a function forgot to return, as in the previous lesson. The symptom fix hides that, and the program now silently skips data it should have used.
When an AI suggests a fix, ask yourself, or ask the AI:
Does this fix the reason the bad value appeared, or does it just stop the error from showing?
Sometimes a check really is the right fix, for example when missing data is expected and should be skipped. The point is to know which one you are doing.
Test more than one case
A fix is not verified until it works on more than the example that broke. Test three kinds of input:
- Normal casesTypical inputs, like the one that failed
- Edge casesEmpty lists, zero, one item, very large values
- Boundary casesExactly at a limit: 18, 90, the last item
You can ask the AI to help:
Suggest 6 test inputs for this function, including edge cases and boundary values, with the expected output for each.
Then run them yourself. The AI can suggest good tests, but only running them proves anything.
Spot invented functions
AI models sometimes produce code that uses functions, methods, or settings that sound right but do not exist, or that belong to a different version of a library. Warning signs:
- An error like
AttributeError: ... has no attribute ...or... is not a functionright after applying the fix. - A method name you cannot find in the official documentation.
- A fix that depends on a setting or option you have never heard of.
When this happens, tell the AI the exact error and ask it to use only functions it is sure exist in your version. And check the official documentation for anything unfamiliar.
Your turn: the fix that passed one test
The rules are: a score of 90 or more is an A, 80 to 89 is a B, and below 80 is a C.
An AI suggested the function below after someone reported that 95 gave the wrong grade. It works for 95. Run it with all four test scores and fix it so every grade is correct. The correct output is:
A
A
B
B
Only the boundary tests, 90 and 80, reveal the bug. The first test would have told you the fix was fine.
Your after-every-fix checklist
- Do I understand why this works? If not, ask for the explanation before keeping it.
- Does it fix the cause, or hide the symptom?
- Did it change only what was needed?
- Did I test normal, edge, and boundary cases? By running them, not by reading the code.
- Does every function it uses really exist in my language and library version?
Five quick questions. Over time they become automatic, and you will find that you catch AI mistakes, and your own, before they cause trouble.
Key Takeaways
- AI fixes can work only for your example, hide the symptom, invent functions, rewrite too much, or get boundaries wrong.
- Ask whether a fix addresses the cause or just stops the error from showing.
- Test with normal, edge, and boundary cases, and run the tests yourself.
- Treat unfamiliar functions with suspicion, and check the official documentation.
- Run the five-question checklist after every AI fix.