The System Prompt Can Be the Router

The moment I needed to add a second capability to the AI assistant, the question was whether to add an orchestration layer. LangChain exists for this. So do agent frameworks and router models.

I didn't add any of them. The overhead felt like overkill for what was still a simple chat feature, so I wrote a rule in the system prompt instead.

Four capabilities later, the route handler has barely changed; the prompt absorbed everything.

That's what I am trying to explore here: what does a prompt that can handle all of that actually look like?

What I Was Trying to Avoid

The purpose of this project was to be a learning exercise: when you're trying to understand something, reaching for a tool that abstracts the hard part defeats this purpose.

LangChain would have handled the routing. It also would have hidden what routing actually requires, and I wanted to know what the problem was before deciding how to solve it.

When the tool is one npm install away, it's easy to skip the question of whether you actually need it.

Turns out the problem isn't complicated enough to need a framework.

A Feature That Kept Growing

Most prompts in this codebase are pipeline prompts. generate-features takes idea context, returns a feature list. discover-competitors takes a problem statement, returns competitors. The input is controlled, the output is structured, and the prompt does one job. They don't route; they don't need to.

The chat prompt is different; it accepts free text from a user who might be asking a question, issuing a command, expressing frustration, or all three at once.

Here's what it looked like at the start, four tools, one routing rule:

const TOOLS = [{
  functionDeclarations: [
    { name: 'add_features',          description: 'Add one or more new features.' },
    { name: 'update_feature',        description: 'Update an existing feature by ID.' },
    { name: 'delete_feature',        description: 'Delete a feature by ID.' },
    { name: 'update_idea_component', description: 'Update a field in the idea analysis.' },
  ]
}]

system prompt

You have tools to directly modify the idea's data. When the user asks you to
add, update, or delete features, or update the idea analysis, USE THE TOOLS.

Simple. The model knew what it could do and when to do it.

Then users started asking how the app worked. The model answered from general knowledge, often wrong. A get_help tool fixed that. It reads the actual guide docs from disk. New tool, new rule:

system prompt

When the user asks how to use the app, call get_help.
Do not answer app usage questions from memory.

Then the discuss button, which sends a context-prefixed message when you click the chat icon on a feature card. Another rule, then feedback detection. New tool, new rule.

Here's the current state:

const TOOLS = [{
  functionDeclarations: [
    { name: 'add_features',          description: 'Add one or more new features.' },
    { name: 'update_feature',        description: 'Update an existing feature by ID.' },
    { name: 'delete_feature',        description: 'Delete a feature by ID.' },
    { name: 'update_idea_component', description: 'Update a field in the idea analysis.' },
    { name: 'get_help',              description: 'Look up how to use the app.' },
    { name: 'record_feedback',       description: 'Log user feedback for admin review.' },
  ]
}]

system prompt

When the user asks how to use the app, call get_help. Do not answer from memory.
When a message starts with "--- [Context:", the user has selected an item to discuss.
When the user expresses dissatisfaction with AI output or suggests a change to
how the app works, call record_feedback.
When a message contains both feedback and an instruction, handle both.

The route handler grew a new if (call.name === ...) branch for each tool. Mechanical execution. The routing intelligence stayed in the prompt.

The Model Is the Router

TOOLS tells the model what it can call. The system prompt tells it when. The model reads both and decides, not the code.

In a traditional routing layer you write code that inspects the message and dispatches to a handler. Here, the model does that work. The route handler just executes whatever comes back.

Gemini can return multiple tool calls in a single response:

const functionCalls = result.response.functionCalls()
// [ { name: 'record_feedback', args: {...} },
//   { name: 'update_idea_component', args: {...} } ]

const functionResponses = await Promise.all(
  functionCalls.map(async (call) => { /* execute */ })
)

A message containing both feedback and an instruction comes back as two calls. The handler executes them in parallel, sends both results to the model, gets one reply. One round trip.

Regardless of how many things the user is trying to accomplish in a single message, it can still be just one call.

The model already supports this natively, you just have to tell it the rules.

Handling Complexity

get_help is where the pattern's flexibility becomes obvious.

When a user asks "how do I delete an idea?", the model doesn't answer from memory, the system prompt told it not to. It calls get_help with the relevant guide section. The route handler reads the markdown file from disk and returns the content as a function response. The model reads that content and writes a reply from it.

That last step is what makes this different. The model doesn't trigger execution and step aside, it incorporates the tool result into its reasoning and synthesizes an answer. Router and synthesizer in one call.

You can put anything inside that tool. An external API. A database query. A search index. A call to a different model. None of that changes the orchestration above it. Add a tool, add a rule in the system prompt, and the model incorporates whatever comes back.

The same system prompt governs all of it, regardless of what the tools do inside them.

How to Actually Write a Prompt This Way

Now that I have a single prompt handling all of this, here's the design pattern I'd use on future projects.

Define your tool surface first.

Each capability gets a tool declaration with a name, description, and parameter schema. Write these before touching the system prompt:

{
  name: 'get_help',
  description: 'Look up how to use the app.',
  parameters: {
    type: SchemaType.OBJECT,
    properties: {
      section: { type: SchemaType.STRING, enum: ['getting-started', 'workspace', 'faq'] }
    }
  }
}

Write routing rules with a prohibition.

Each rule needs two parts. What to call, and what not to do instead. An incomplete rule leaves the model free to improvise:

system prompt

When the user asks about the app, call get_help.

A complete rule closes that door:

system prompt

When the user asks how to use the app, call get_help.
Do not answer app usage questions from memory.

Add tools and rules in the same change.

Each tool needs a matching rule. Drift between the two produces inconsistent behavior that doesn't throw an error:

// In TOOLS
{ name: 'record_feedback', description: 'Log user feedback for admin review.' }

system prompt

When the user expresses dissatisfaction with AI output, call record_feedback.
Do not call record_feedback for product strategy questions.

Declare parallel intent handling explicitly.

A message can carry more than one intent, and without this rule the model handles only the first:

system prompt

When a message contains both feedback and an instruction, handle both.

The Tradeoff

The system prompt is load-bearing now, which means changing a routing rule is a deploy. A broken routing rule doesn't throw an error; it just changes how the model behaves.

Automated tests that send real messages to the model and assert on which tools get called, that's what catches this. Without them, prompt changes are risky in a way that code changes usually aren't. That's the next thing to build.

This approach also has a ceiling. A tool surface with six entries and four routing rules is manageable in a single prompt, but at some point it isn't. That point arrives with long multi-step workflows with conditional branching, tasks where one agent needs to spawn another, situations where you want a visual graph of what's happening. Using a framework is the right approach in those cases, this pattern doesn't scale there and shouldn't try to.