The Skim Trap
You've done it: opened a 30-page paper, read the abstract, skimmed the figures, and convinced yourself you "get it." Then someone asks what the control group was and you go quiet. Skimming feels like reading. It isn't. It's pattern-matching on jargon while the actual argument slides past you.
AI fixes the speed problem without fixing the comprehension problem — unless you use it deliberately. A summary you didn't interrogate is just a faster way to misunderstand something. The goal here isn't to read less and know less. It's to spend your attention where it counts: the method, the findings, and the limitations. Everything else is packaging.
Extract the Skeleton First
Before you ask anything clever, get the bones. Paste the paper (or the methods and results sections) into your AI tool and force it into a fixed structure. Structure is what stops the model from rambling and what lets you compare ten papers in an afternoon.
Summarize this paper using exactly these headers, one or two
sentences each:
- Research question: what they were actually trying to find out
- Method: design, sample size, what they measured, what they compared
- Key findings: the 2-3 results that matter, with numbers
- Stated limitations: what the authors admit the study can't show
- My follow-up: 3 things a skeptical reviewer would push back on
Do not include background or motivation. Quote a phrase from the
paper for each finding so I can find it.
That last instruction matters more than it looks. Demanding a quoted phrase per claim turns the summary into something checkable. When the model invents a finding — and it will, occasionally — the missing or mangled quote is your tripwire. If it can't point to the sentence, treat the claim as unverified, not true.
Read this skeleton, then go back to the actual method and results sections in the paper. Not the whole thing — just those two. The AI told you where to look; now you look. This is the whole move: AI narrows your reading down to the paragraphs that decide whether the conclusion holds.
Interrogate, Don't Just Absorb
A summary tells you what the authors want you to think. Your job is to ask whether they earned it. Use the AI as a sparring partner, not a narrator.
The trick is to ask questions that have specific, locatable answers:
Acting as a critical peer reviewer, answer each from the paper only:
- Does the sample size support the claims, or is it underpowered?
- Could the result be explained by something other than the
authors' hypothesis? Name the most plausible confounder.
- Is the effect size practically meaningful, or just statistically
significant?
- Did they pre-register, or could this be p-hacked / cherry-picked?
If the paper doesn't give enough information to answer, say
"not reported" instead of guessing.
The "say not reported instead of guessing" clause is doing heavy lifting. Without it, a model fills gaps with plausible-sounding fabrication. With it, the gaps themselves become information — a paper that never mentions its sample size or its controls is telling you something, and now you'll notice.
Make it argue with itself
When a finding feels too clean, ask for the counter-case directly:
Steelman the strongest argument AGAINST this paper's main
conclusion. Then tell me which of the authors' choices that
critique depends on.
You're not trying to dunk on the authors. You're stress-testing whether the conclusion survives a smart objection. If the steelman collapses the moment you check it against the data, the paper is probably solid. If it doesn't, you've found exactly what to flag in your literature review.
You Are Still the Reviewer
Here's the part the hype crowd skips: the AI cannot judge quality. It can summarize a paper from a predatory journal and a Nature paper in the identical confident tone. It doesn't know that the sample was self-selected, that the effect vanished on replication, or that the authors have a financial stake. It pattern-matches text; it doesn't evaluate science.
So keep a few judgments yours and yours alone:
- Source credibility. Where was this published? Peer-reviewed, preprint, or a journal you've never heard of? AI won't flag a sketchy venue.
- Does the data support the claim? Read the key results table yourself. Models routinely overstate "significant" into "proves."
- Is this current? A 2014 method may be superseded. The model won't tell you the field moved on unless you ask.
- Whose money? Check the funding and conflict-of-interest statement with your own eyes.
A fast test: pick the single most important number in the paper and find it in the original yourself. If the AI's summary and the actual table disagree, you just learned not to trust that summary — and you caught it in thirty seconds.
A Workflow You'll Actually Repeat
Stack these into something fast enough to use on every paper, not just the one you're presenting:
- Triage (2 min). Skeleton prompt. Decide if the paper is even worth your full attention. Most aren't — kill them here.
- Targeted read (10 min). Read the method and results sections the AI pointed you to. Confirm the headline finding against the real numbers.
- Interrogate (5 min). Run the peer-reviewer and steelman prompts. Note what's "not reported."
- Judge (3 min). Apply your own checks: venue, funding, recency, data-to-claim fit.
- Capture. Save the skeleton plus your verdict into your notes — you'll thank yourself in chapter 6.
Twenty minutes a paper, and you come out able to defend what it does and doesn't show. That's the trade: you read fewer words and understand more of them, because you spent your reading on the parts that decide the argument.
If you want to go deeper on prompting models to behave as critics rather than cheerleaders, the AI for Academic Research course drills these moves with real papers. The habit underneath all of it — verify before you trust — is the one that keeps you out of trouble for the rest of this book.