Orchestration

With the foundational components in place, you can consider orchestration patterns to enable 

your agent to execute workflows effectively.

While it’s tempting to immediately build a fully autonomous agent with complex architecture,
customers typically achieve greater success with an incremental approach.

In general, orchestration patterns fall into two categories:

A single agent can handle many tasks by incrementally adding tools, keeping complexity
manageable and simplifying evaluation and maintenance. Each new tool expands its capabilities
without prematurely forcing you to orchestrate multiple agents.

Every orchestration approach needs the concept of a ‘run’, typically implemented as a loop that
lets agents operate until an exit condition is reached. Common exit conditions include tool calls, 

a certain structured output, errors, or reaching a maximum number of turns.

For example, in the Agents SDK in ChatGPT, agents are started using the Runner.run() method, which loops over the LLM until either:

Example usage:

This concept of a while loop is central to the functioning of an agent. In multi-agent systems, as
you’ll see next, you can have a sequence of tool calls and handoffs between agents but allow the
model to run multiple steps until an exit condition is met.

An effective strategy for managing complexity without switching to a multi-agent framework is to
use prompt templates. Rather than maintaining numerous individual prompts for distinct use
cases, use a single flexible base prompt that accepts policy variables. This template approach
adapts easily to various contexts, significantly simplifying maintenance and evaluation. As new use
cases arise, you can update variables rather than rewriting entire workflows.

Example

When to consider creating multiple agents?

General recommendation is to maximize a single agent’s capabilities first. More agents can
provide intuitive separation of concepts, but can introduce additional complexity and overhead, 

so often a single agent with tools is sufficient.  

For many complex workflows, splitting up prompts and tools across multiple agents allows for
improved performance and scalability. When your agents fail to follow complicated instructions 

or consistently select incorrect tools, you may need to further divide your system and introduce
more distinct agents.

Practical guidelines for splitting agents include:

While multi-agent systems can be designed in numerous ways for specific workflows and
requirements, our experience with customers highlights two broadly applicable categories:

Multi-agent systems can be modeled as graphs, with agents represented as nodes. In the manager
pattern, edges represent tool calls whereas in the decentralized pattern, edges represent handoffs that transfer execution between agents.

Regardless of the orchestration pattern, the same principles apply: keep components flexible,
composable, and driven by clear, well-structured prompts.

The manager pattern empowers a central LLM—the “manager”—to orchestrate a network of
specialized agents seamlessly through tool calls. Instead of losing context or control, the manager
intelligently delegates tasks to the right agent at the right time, effortlessly synthesizing the results into a cohesive interaction. This ensures a smooth, unified user experience, with specialized
capabilities always available on-demand.

This pattern is ideal for workflows where you only want one agent to control workflow execution
and have access to the user.

Declarative vs non-declarative graphs

Some frameworks are declarative, requiring developers to explicitly define every branch, loop,
and conditional in the workflow upfront through graphs consisting of nodes (agents) and
edges (deterministic or dynamic handoffs). While beneficial for visual clarity, this approach
can quickly become cumbersome and challenging as workflows grow more dynamic and
complex, often necessitating the learning of specialized domain-specific languages.

In contrast, the Agents SDK adopts a more flexible, code-first approach. Developers can 

directly express workflow logic using familiar programming constructs without needing to 

pre-define the entire graph upfront, enabling more dynamic and adaptable agent orchestration.

In a decentralized pattern, agents can ‘handoff’ workflow execution to one another. Handoffs are a
one way transfer that allow an agent to delegate to another agent. In the Agents SDK, a handoff is
a type of tool, or function. If an agent calls a handoff function, we immediately start execution on
that new agent that was handed off to while also transferring the latest conversation state. 

This pattern involves using many agents on equal footing, where one agent can directly hand 

off control of the workflow to another agent. This is optimal when you don’t need a single agent
maintaining central control or synthesis—instead allowing each agent to take over execution and
interact with the user as needed.

This pattern is especially effective for scenarios like conversation triage, or whenever you prefer
specialized agents to fully take over certain tasks without the original agent needing to remain
involved. Optionally, you can equip the second agent with a handoff back to the original agent,
allowing it to transfer control again if necessary.