Teaching a Small Model to Think

the wisdom required to cheat is the wisdom you were supposed to learn

It started with a simple observation

Think about how a person actually goes from knowing nothing to becoming an expert.

As a young child, you learn everything in an unstructured way. You absorb language, shapes, colors, social cues, basic cause and effect. Nobody gives you multiple choice tests. You just experience the world and build a foundation.

That’s pre-training. In machine learning, we feed a model enormous amounts of text so it builds a broad foundation of language, facts, and patterns. No structure, no exams. Just absorption.

But that broad foundation doesn’t make you an expert. Millions of people can read and write and do basic math. What separates a generalist from an expert is what comes next.

When you want to become a certified professional — a lawyer, a doctor, an engineer — you face a very specific kind of challenge. The bar exam. The GMAT. Board certifications. These don’t ask you to write long essays about everything you know. They give you structured reasoning tests. Scenario-based problems. “Given this situation, which of these four options is correct?” They test your ability to think across different contexts and commit to a decision.

And when you’re preparing for these exams, something interesting happens. You study the material — that’s the knowledge. But by working through practice problems in exam format, you’re simultaneously learning how to reason under constraint, how to eliminate wrong options, how to manage ambiguity, how to commit to an answer when you’re not sure. The content and the conduct develop together.

That observation kept nagging at me. I had a small model — a few billion parameters — that was like a student who’d been through general education but was performing poorly. Verbose, unfocused, getting questions wrong. The question wasn’t how to teach it from scratch. It was how to take a struggling student and turn them into a sharp thinker.

What I actually did

The typical approach: take a reinforcement learning algorithm called GRPO, generate multiple responses to a prompt, score them, push the model toward better responses and away from worse ones. You hand the student a stack of exam papers and a buzzer. Correct or incorrect. Figure it out.

I tried that. For hundreds of iterations, the model learned almost nothing. The buzzer kept saying wrong, but the model had no idea what was wrong with its thinking. It just kept producing the same verbose, confused chains with slightly different words.

So I thought about how we actually prepare people for expert-level exams. We don’t just hand them past papers and a mark scheme. We give them worked examples. A tutor walks through the reasoning: “Here’s the question. Here’s how I’d break it down. Here’s why option A is wrong and option C is right.” The student reads the worked example, then tries the next problem with that reasoning fresh in their mind.

Each of my training samples already had an expert answer trace — a full step-by-step reasoning chain showing how to arrive at the correct answer. In typical setups, this trace isn’t used during training. Only the prompt is.

I decided to use it.

Before asking the model to generate its own response, I injected the expert reasoning directly into its context window — like placing a worked example next to the unsolved problem on a student’s desk. The model reads through the expert’s thinking, and then continues from there with its own generation.

Then I did what any good tutor does — I stepped back gradually. Full worked example. Then partial. Then nothing. And I used multiple attempts per question at different levels of assistance, so the model was always being compared against itself. The gradient from that comparison is what drives the learning.

Within a handful of iterations, the plateau broke. The model started getting answers right.

The cheating student

There’s a saying in Urdu — “naqal ke liye bhi aqal chahiye” — even to cheat, you need wisdom.

Imagine a student sitting an exam. Not three different students — the same student, the same exam, multiple attempts.

First attempt: she has a full cheat sheet. The complete reasoning, step by step, written out by an expert. She reads through it, follows the logic, writes down the answer. Correct. Full marks. In her mind, she gamed the system. But what actually happened? She had to read that reasoning. Process it through her own brain. Build a mental model of the logic. The information went through her neural circuitry whether she intended to learn or not.

Second attempt: the cheat sheet is torn in half. She reads what’s there, but now she has to bridge the gap herself. She reasons through the missing part and arrives at the answer. Still correct. She thinks she’s still cheating, but she actually had to think this time.

Third attempt: no cheat sheet. Nothing. She tries from scratch. Gets confused. Wrong answer. No marks.

The message she takes away: cheating works. She wants the cheat sheet. But the cheat sheets keep getting smaller. And she keeps taking the exam. Not ten times. Not a hundred. Thousands of times.

Something happens that she doesn’t notice. Every time she “cheated,” she was routing expert reasoning through her own brain. She wasn’t trying to learn. She was trying to get marks. But the learning happened anyway — as a side effect of the cheating.

By the time the cheat sheet is gone, she doesn’t need it. She’s processed so many expert reasoning traces through her own thinking that the patterns are part of her now. She can look at a new problem she’s never seen before — in a completely different subject — and something in her recognizes the structure. She knows how to decompose it, how to eliminate wrong options, how to commit. She doesn’t remember learning any of this. She thought she was cheating the whole time.

That’s exactly what happens inside the model.

The expert reasoning enters the model’s context and flows through dozens of neural network layers. At each layer, it becomes high-dimensional representations encoding not just what the text says but structural patterns — how concepts relate, how elimination logic works. When the model generates its own response, every token it produces looks back at those representations through attention. It’s not copying words. It’s absorbing the shape of good thinking.

But here’s the crucial part: when the model’s weights get updated, only what the model produced from its own thinking counts. The cheat sheet was on the desk, not in the model’s head. The weight update is an X-ray of what genuinely changed inside the model’s own circuitry. And what that X-ray shows is that even though the model was just chasing marks, its brain was changing. Representations formed. Patterns solidified. Knowledge built as a side effect of chasing rewards through cheating.

نقل میں بھی عقل چاہیے. The wisdom you need to cheat turns out to be the wisdom you were supposed to learn.

What I saw the model learn

This is the part that surprised me.

First, the model got dramatically better at the training domain. On the specific types of questions it practiced, reasoning became sharp, focused, and accurate. That was the goal.

But second, performance jumped on completely unrelated domains the model had never seen during training. Biology, history, mathematics, law, philosophy. Topics with zero overlap to the training set. And it wasn’t just getting more answers right. The quality of reasoning changed. The verbose, circular rambling was gone. The model was producing focused, structured thinking chains — decomposing problems, eliminating options, committing to answers with clear justification.

The model hadn’t learned a domain. It had learned how to reason.

The operations it acquired are domain-agnostic: decomposing a question into parts, mapping options to categories, eliminating answers through structural mismatch, identifying which framework applies. These transfer. A lawyer who prepares for the bar exam doesn’t just learn law — they learn how to reason under constraint, how to choose between plausible options. Those skills carry far beyond the bar.

And there was another change I didn’t expect. Early in training, the model was producing massively bloated thinking chains. After the technique kicked in, output length dropped by more than half — while accuracy increased simultaneously. The model didn’t just get smarter. It got dramatically more efficient at thinking. It learned to say more with less.

This isn’t memorization. A small model can’t memorize its way to broad capability. But it can learn generalizable reasoning circuits: how to decompose, how to map, how to eliminate, how to select. Those circuits fit in a small number of parameters because they’re patterns, not facts.

Broad benchmark performance roughly doubled — a fraction of the way through training. On a laptop.

The manufactured aha moment

There’s been a lot of excitement about “aha moments” in AI — instances where a model spontaneously discovers structured reasoning during reinforcement learning. These moments are real and fascinating. They tend to emerge in very large models with significant compute, where the model has enough capacity to explore widely and eventually stumble onto productive reasoning patterns.

I wanted a more direct path for smaller models. My approach doesn’t wait for the aha moment to emerge through exploration. It manufactures the conditions for it deliberately. By showing the model what structured reasoning looks like through its own hidden states, the gradient carries rich structural information from the very first training step. The model doesn’t need to discover good reasoning by accident.

Same destination, different path. One waits for lightning to strike. The other installs a lightning rod.

The lightning rod is just the answer field in the training data.

What this means

A small model that reasons poorly isn’t lacking capacity. It’s lacking structure. The machinery is there. The weights can represent good reasoning patterns. They just haven’t been shaped properly.

This technique reshapes them — efficiently, on modest hardware, with small datasets. It doesn’t replace pre-training or existing RL. Those build the model’s raw capacity. This sharpens it — turning verbose, confused reasoning into focused, accurate reasoning — by combining the knowledge signal and the behavioral signal into one unified process. Show the model what good reasoning looks like while simultaneously rewarding it for producing good reasoning. Let the what and the how reinforce each other.

Expert reasoning traces already exist in most training data. They’re just not being used during the reinforcement learning stage. Feeding them back as scaffolded context adds a second, richer learning signal — essentially for free.

There’s a scaling axis beyond parameters, data, and compute: the quality of the training signal. Right now, that axis is largely untouched.

The punchline

The entire technique — which I call ICL-Guided Gradient Shaping:

1. Take your training data’s expert answer traces
2. Show them to the model as masked context before generation
3. Use multiple attempts with different amounts shown
4. Gradually reduce what’s shown to zero

No new architecture. No teacher model. No compute cluster. About few lines of code on top of standard GRPO, running on a laptop.

I started with a small reasoning model that could barely get anything right. Verbose, confused, confidently wrong. The same approach we’ve used for centuries to turn struggling students into sharp thinkers worked on a language model. Show the worked example, practice with structure, fade the support, let them fly on their own.

I’m currently preparing a technical paper with full methodology and results. If you’re working on small model reasoning, efficient RL training, or credit assignment — I’d genuinely like to hear from you. The details are worth a conversation.

Adeel Ahmad is an AI researcher training small models to think on consumer hardware. You can reach him on LinkedIn or X.

Teaching a Small Model to Think was originally published in Artificial Intelligence in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.