An assistant, not an oracle: what AI does in research
AI for research and reports · Lesson 1 / 24
What a language model is actually doing
The model predicts how text continues. It doesn't consult a library, doesn't remember where a claim came from, and can't tell "I saw this in a thousand papers" apart from "I just assembled this out of similar-looking words". Both come out equally smooth.
Hence the rule this course runs on: AI produces the plausible, not the true. And plausibility is improving faster than verifiability — the text keeps getting more convincing while your checking time stays exactly the same. So verification has to be built into the process rather than bolted on at the end.
What the model does well
- Turns a topic into a map. Lists approaches, schools, vocabulary — something to push off from when you start searching.
- Reformulates. Converts a vague topic into five candidate questions.
- Compresses text you already have. A summary of an uploaded paper is far more reliable than a summary from memory.
- Objects. On request it finds the soft spots in your argument.
- Translates and cleans up language. Terminology, register, overloaded sentences.
What it does badly
- References and quotations. It synthesises plausible names, titles and years.
- Numbers. It rounds, confuses sample with population, and carries a figure over from a different study.
- Judging novelty. It doesn't know what appeared last week or what has since been overturned.
- Weighing evidence. An opinion column and a meta-analysis look identical to it.
The working division of labour
AI owns: drafts, structure, compression, objections, language
You own: the question, source selection, facts, numbers, conclusions
Seam rule: anything the model asserts about the world gets checked
against a source; anything it does to your text gets
checked by your own eye
Insight. The model's usefulness is inversely proportional to how much the answer depends on its memory. Give it a text and quality jumps: it stops recalling and starts working.
Common mistake. Asking "what is known about topic X" and recording the answer as a literature review. That isn't a review — it's a hypothesis about what a review might look like.
Pro tip. Keep two windows: one for generating, one for checking, with a clean history and the question "what objections apply to this claim". Splitting the roles stops the model from agreeing with itself.
Cheat sheet
- The model predicts text; it does not consult facts.
- Plausibility outruns verifiability.
- An uploaded text beats the model's memory.
- The question, the sources and the conclusions stay yours.
1. Why is the model's answer about facts unreliable?
2. In which mode is the model most reliable?
3. What does "plausibility outruns verifiability" mean?