There is a recurring fantasy in prompt engineering: that somewhere out there is a set of magic words — a phrase you can drop into any prompt to unlock a better answer. The honest finding, both from our own work and from the research that followed, is that the magic isn't a phrase. It's a move: reformulate the question before you answer it.

We first introduced query reformulation as a core technique of the Synthetic Interactive Persona Agent (SIPA). The idea was straightforward — take the user's original question, transform it into a more detailed, precise, contextually richer version that aligns with how the model actually understands the task, and answer that. The research has since named and benchmarked exactly this move.

Rephrase and Respond

The canonical reference is Rephrase and Respond (RaR) (Deng et al.). Its premise matches the SIPA intuition precisely: humans and models routinely misread seemingly unambiguous questions, so having the model expand and clarify the question first closes the gap between what was asked and what was meant. RaR reports consistent gains across reasoning tasks and is complementary to chain-of-thought — you can do both.

RaR also formalizes the two designs we described. In the one-step form, a single model rephrases and then answers in one pass. In the two-step form, a smaller, faster model reformulates the question and hands the expanded version to a separate, more capable model to answer. The two-step pattern is the one to reach for when reformulation is domain-specific — a medical-intake or legal-intake front-end — or when you want the reformulation standardized into a template for review and caching.

Why it works

The rephrase pins down an under-specified question before the model commits to an answer. Consider the difference:

"How do I handle this claim?" → "What are the coverage-determination steps for a water-damage claim under an HO-3 policy, and what documentation is required at each step?"

The answer to the second question is the one the user actually wanted. More broadly, the format and framing of a prompt is load-bearing in its own right: research on prompt-format sensitivity has shown that meaning-preserving changes to how a question is posed can move a model's accuracy by a startling margin. Reformulation is a deliberate use of that lever.

Two cautions

First, the cost is real — extra tokens and latency, plus a drift risk: if the restatement distorts the user's intent, the model then answers the wrong question confidently. Mitigate by showing the rephrase to the user so a misread can be corrected in one step, and by keeping the elaboration tight enough that it sharpens scope rather than swapping topics.

Second, do not confuse this with retrieval-side query rewriting. That move — common in RAG pipelines — rewrites a query to feed a retriever and is never shown to the user; its job is recall. The reformulation here is a reasoning-and-clarity move that is part of the visible answer. Same surface mechanic, opposite purpose and audience. A production system happily uses both: a retrieval rewrite to fetch documents, and a RaR-style rephrase to frame the answer.

Use it today

You don't need a pipeline to benefit. In your own interactions with any capable model, ask it to restate your question in a more detailed form and then answer it — or simply write the more detailed question yourself. In a product, a small reformulating agent in front of your main model is one of the cheapest accuracy upgrades available, provided you surface the rephrase and watch for drift. The magic was never a word. It was asking a better question.