The Future of AI Agents: Less Is More
For the last few years, the default assumption in AI has been simple: bigger model, better results. If you wanted a smarter agent, you reached for the largest frontier model you could afford. But a pair of recent research efforts out of NVIDIA make the case that this assumption breaks down the moment you move from chatbots to agents: systems that plan, call tools, and execute multi-step tasks.
For the last few years, the default assumption in AI has been simple: bigger model, better results. If you wanted a smarter agent, you reached for the largest frontier model you could afford. But a pair of recent research efforts out of NVIDIA make the case that this assumption breaks down the moment you move from chatbots to agents: systems that plan, call tools, and execute multi-step tasks.
The pitch isn’t “small models instead of big ones.” It’s something more interesting: small models orchestrating big ones, calling on frontier-scale intelligence only when it’s actually needed.
The problem with using a frontier model for everything
Most agentic systems today lean on a single large language model to do all of it: parse the user’s intent, decide which tool to call, format the output, and reason through edge cases. That works, but it’s inefficient. A large fraction of what an agent actually does step to step is narrow and repetitive: classifying intent, generating structured JSON, summarizing a tool’s output, deciding which of five APIs to call next. None of that requires a few-hundred-billion-parameter model. It’s a lot like hiring a surgeon to take your temperature.
NVIDIA researchers frame this as a mismatch between infrastructure cost and the value actually being delivered. Running a frontier-scale model on every single agentic step means paying frontier-scale compute costs for tasks that don’t need frontier-scale reasoning.
Small models as the “engine,” not the “brain”
One line of NVIDIA research, a position paper on small language models (SLMs) in agentic AI, argues that most of the actual mechanical work inside an agent (tool calls, structured outputs, narrow sub-tasks) is a better fit for small, specialized models than for a single generalist LLM. The argument isn’t just about cost. Small models fine-tuned on a narrow task can be more reliable at that task than a generalist model, precisely because they aren’t distracted by everything else they know how to do. A small model trained only to produce valid SQL or JSON has fewer ways to wander off-format.
The economics back this up: serving a small model can be roughly an order of magnitude (or more) cheaper in latency, energy, and compute than serving a large one, which matters enormously once you’re running millions of agent steps a day rather than a handful of chat messages.
Letting a small model be the orchestrator
The more novel piece of this research is ToolOrchestra, a method for training a small language model to sit at the center of an agentic system as the orchestrator: the component deciding which tool or model to call at each step, including when to call a frontier model at all.
This matters because of a subtle failure mode the researchers identify: when you ask a frontier model to manage its own routing decisions, it tends to over-rely on itself (or a small set of other strong models), even when a cheaper tool or smaller model would do the job just as well. It’s a kind of self-preference bias baked into how these models reason about their own capabilities.
ToolOrchestra sidesteps this by training a dedicated, much smaller orchestrator model (NVIDIA’s released version has 8 billion parameters) using reinforcement learning, with rewards that account for task correctness, cost, latency, and user preferences, not just “did it get the right answer.” The result is a routing policy that’s explicitly cost-aware rather than defaulting to the biggest hammer available.
The reported numbers are the interesting part: on the Humanity’s Last Exam benchmark, the trained orchestrator scored 37.1% versus 35.1% for GPT-5, while running about 2.5x more efficiently. On two other benchmarks, τ²-Bench and FRAMES, the orchestrator also came out ahead of GPT-5, this time while using only around 30% of the cost. In other words, a small model making smart decisions about *when* to call a big model beat a big model trying to do everything itself, and did it more cheaply.
The bigger shift
Put these two pieces together and you get a picture of where agentic AI architecture may be heading: not one giant model doing everything, but a small, efficient orchestrator making deliberate, cost-aware decisions about when to hand a problem off to a more powerful (and more expensive) model or tool, and doing most of the actual work itself the rest of the time.
It’s a shift from “smarter models” to “smarter systems.” The frontier model doesn’t disappear in this picture. It just stops being the thing running the whole show, and starts being one tool among many, called in only when its extra reasoning power actually earns its cost.
*Sources: NVIDIA’s “ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration” (research.nvidia.com) and “Small Language Models are the Future of Agentic AI” (arXiv:2506.02153).*