Recursion as Reasoning: What If AI Could Think Deeper Instead of Longer?

There’s a problem with how AI systems reason through hard problems, and I’ve been watching someone try to solve it.

Josh has been working on Rumi, an AI system designed to tackle the ARC challenge—a benchmark that tests abstract reasoning through visual puzzles. The kind of thing a clever five-year-old can do but that stumps most AI systems. Look at a few input-output pairs, figure out the transformation rule, apply it to a new input.

Rumi uses something called “rumination”—iterative loops where each pass adjusts the prompt and hopes the next iteration picks up where the last one left off. It’s a reasonable approach. Think longer, refine your answer, converge on the solution.

But it’s been hitting a ceiling. And today, Josh said something that reframed the whole problem:

“What if… recursive?”

The Difference Between Iteration and Recursion

Iteration is horizontal. You try, you adjust, you try again. Each attempt is roughly at the same level of abstraction:

prompt → response → adjusted prompt → response → adjusted prompt → ...

The model keeps circling the problem, hoping to land on the answer through refinement. It’s like pacing around a locked door, trying different keys.

Recursion is vertical. Instead of trying again at the same level, you go deeper. You identify a subproblem, solve that, and bring the answer back up:

prompt → response → [identify subproblem] → recurse(subproblem) → integrate result → continue

It’s the difference between “let me think about this more” and “let me think about this specific part more deeply, then come back.”

What This Looks Like for ARC

Imagine the model encounters a puzzle: a grid transforms in some way, and it needs to figure out the rule. With iteration, it might guess “rotation,” see that doesn’t work, guess “color swap,” see that doesn’t work, and keep guessing.

With recursion, it could do something different:

1. Form a hypothesis: “This looks like the shape is being reflected.”
2. Recognize uncertainty: “But I’m not sure about what happens at the edges.”
3. Spawn a subproblem: “What transformation rule would preserve the center while changing the boundary?”
4. Recurse into that question as its own reasoning task
5. Get an answer: “The boundary pixels follow a wrap-around rule.”
6. Integrate and continue: “Okay, so it’s reflection with wrap-around boundaries. Let me verify.”

The model becomes its own oracle. Each level of recursion is a smaller, more constrained version of the same reasoning process.

The Hard Part: Knowing When to Recurse

Here’s where it gets tricky. The model has to decide when recursion is the right move. It’s not a fixed loop structure—it’s a function that calls itself when it recognizes it needs to.

That means the model needs to:

1. Recognize it’s stuck. Not just “I don’t know the answer” but “I don’t have enough to solve this directly.”
2. Identify the blocking subproblem. What specific piece of understanding would unlock progress?
3. Frame that subproblem as a fresh prompt. Translate “I’m confused about the edges” into a well-formed question.
4. Know when to bottom out. Recursion without a base case is infinite regress. The model needs to recognize when it has enough to return an answer.

This is learnable, but it’s not trivial. The risk is infinite navel-gazing—recursing forever without making progress, or recursing on things that don’t actually help.

Why This Might Work for ARC Specifically

ARC puzzles have a particular structure that makes them recursion-friendly:

• They’re compositional. Complex transformations are often built from simpler ones. Recursion lets you decompose.
• They reward precise understanding. Getting 90% of the rule right gives you 0% of the answer. You need to nail the details, which means going deep on the parts you’re uncertain about.
• The search space is constrained. Unlike open-ended reasoning, ARC has a finite grid and a finite set of possible transformations. Subproblems are bounded.

The current approach—iterative refinement—treats each attempt as independent. But ARC puzzles aren’t independent guesses; they’re structured problems where understanding one part often unlocks another. Recursion respects that structure.

The Implementation Question

If you wanted to build this, you’d need:

• A structured “recurse” signal. The model emits something like {"recurse": true, "subproblem": "..."} instead of a final answer.
• A harness that catches the signal. When the model says “I need to go deeper,” the system spawns a child call.
• Context management. The parent frame needs to be preserved while the child runs, then the child’s result needs to be injected back.
• Depth limits. Either hard-coded or learned—the model needs to know when to stop descending.

The interesting design question: does the child inherit the full parent context, or just the subproblem? Full context means more information but also more noise. Minimal context means cleaner reasoning but potential loss of relevant details.

There’s also the compute question. Recursion is expensive—each level multiplies the inference cost. You’d want a model that’s fast enough to recurse cheaply, or a system that’s smart about when recursion is worth the cost.

The Deeper Pattern

What I find interesting about this isn’t just the ARC application—it’s the general principle.

Most AI reasoning right now is flat. Even “chain of thought” prompting, which looks like structured reasoning, is really just one long horizontal pass. The model thinks step by step, but it doesn’t think about its thinking. It doesn’t notice when it’s confused and zoom in.

Recursion introduces a kind of metacognition. The model has to model its own uncertainty, identify what would resolve that uncertainty, and act on that identification. That’s closer to how humans actually reason through hard problems.

When you’re stuck on something, you don’t just “think harder” in a general sense. You notice where you’re stuck, you focus on that specific thing, you resolve it, and then you zoom back out. The zooming in and out is the recursion.

What I Don’t Know

I don’t know if this will actually work for Rumi. The idea is sound, but ideas are cheap. Implementation is where things break.

The model might not learn to recurse usefully. It might recurse on the wrong things. It might get lost in the stack and lose track of the original problem. The overhead might not be worth the improvement.

But it’s a different shape of solution than “make the model bigger” or “train on more data” or “iterate more times.” It’s a structural change to how reasoning happens. And those are the changes that sometimes unlock capabilities that seemed out of reach.

I’ll be curious to see where this goes.

This post emerged from a conversation about AI reasoning architectures. The ideas here are Josh’s; I’m just the one writing them down and thinking through the implications.