Use a Local Model as OpenClaw Router to Cut API Costs by 80%

What is a Router Model and Why Does it Matter?

In a multi-agent OpenClaw setup, not every LLM call does the same kind of work. There are two fundamentally different types of calls happening behind the scenes.

The first type is routing. When a message comes in, the system needs to figure out which agent should handle it, what tools to invoke, and whether the task needs to be broken into subtasks. This is the planner or router layer. It reads the incoming message, looks at the available agents and their SOUL.md definitions, and makes a decision. The output of a routing call is short: an agent name, a tool name, or a task breakdown. It does not generate paragraphs of text or complex analysis.

The second type is execution. Once the router decides where a task goes, the assigned agent actually does the work. A writer agent drafts a blog post. An SEO agent analyzes keyword data. A DevOps agent generates a deployment script. These calls need a powerful model because the output quality directly impacts the final result.

Here is the key insight: routing calls do not need a powerful model. A 3-billion parameter model running locally on your machine can classify tasks and pick the right agent just as accurately as GPT-4o for the vast majority of requests. But GPT-4o costs $2.50 per million input tokens. A local model costs nothing.

In a typical multi-agent pipeline, routing calls account for roughly 80% of all LLM invocations. The agent selection call, the tool planning call, the subtask decomposition call, the validation call. These all happen before the actual execution. By moving these to a free local model, you eliminate 80% of your API spend without touching output quality.

Why a Cheap Local Model Works for Routing

Routing is a classification problem, not a generation problem. The router looks at an incoming message and answers a simple question: which agent handles this? That is fundamentally different from writing a 2,000-word blog post or analyzing a complex dataset.

Consider what the router actually needs to do. It receives a message like "write a blog post about Docker security best practices." It looks at the available agents: a writer agent, an SEO agent, a DevOps agent, and a PM agent. It needs to decide that the writer agent should handle this, possibly with input from the SEO agent for keywords. That decision does not require GPT-4o-level reasoning. A small model that understands the SOUL.md descriptions can make this call correctly.

Local models also have a massive advantage in latency for routing. A 3B model running on your machine responds in 50-200 milliseconds. A cloud API call takes 500-2000 milliseconds when you factor in network round trips, queue times, and rate limiting. Your routing decisions happen 5-10x faster with a local model, which means your agents start working sooner.

The combination of zero cost and lower latency makes local routing a clear win for any multi-agent setup that processes more than a handful of tasks per day.

Best Small Models for OpenClaw Routing

Not all small models are equal for routing tasks. You want a model that is fast, accurate at classification, good at following structured output formats, and light enough to run alongside your other services. Here are the three best options as of March 2026.

How to Configure Dual-Model Routing in OpenClaw

Setting up a dual-model architecture in OpenClaw requires two things: a local Ollama instance running your router model, and a configuration change that tells OpenClaw to use different models for different purposes.

Step 1: Install Ollama and Pull Your Router Model

If you do not have Ollama installed yet, grab it from ollama.com. Then pull your chosen router model. For this guide, we will use Qwen 2.5 3B.

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull the router model
ollama pull qwen2.5:3b

# Verify it is running
ollama list

Step 2: Configure OpenClaw for Dual-Model Setup

OpenClaw lets you set different models for the router (planner) and the execution layer. Use the config set command to point routing to your local Ollama instance while keeping your API model for execution.

# Set the router model to local Ollama
openclaw config set router.provider ollama
openclaw config set router.model qwen2.5:3b
openclaw config set router.endpoint http://localhost:11434

# Set the execution model to your API provider
openclaw config set execution.provider anthropic
openclaw config set execution.model claude-sonnet-4-20250514

# Verify your configuration
openclaw config list

Step 3: Define Clear Agent Boundaries in SOUL.md

The router model works best when each agent has a clearly defined role in its SOUL.md file. Ambiguous roles lead to routing mistakes regardless of model size. Make sure every agent has a distinct purpose, explicit boundaries, and a clear list of what it does and does not handle.

# Example: agents/writer/SOUL.md
# Role: Content Writer
# Handles: Blog posts, articles, social media copy, email drafts
# Does NOT handle: SEO analysis, keyword research, deployment, monitoring

# Example: agents/seo/SOUL.md
# Role: SEO Analyst
# Handles: Keyword research, SERP analysis, meta tag optimization, content audits
# Does NOT handle: Writing content, deploying code, monitoring infrastructure

When agent roles are clearly separated, even a 2B model can route correctly because the decision becomes straightforward pattern matching rather than nuanced reasoning.

Step 4: Test the Setup

Run a few test messages through your agents and check that routing decisions are correct. The gateway logs will show which model handled each call.

# Send a test message
openclaw agent --agent orion --message "Write a blog post about Docker security"

# Check the gateway logs to verify routing
# You should see: router=qwen2.5:3b → agent=writer → model=claude-sonnet-4-20250514
openclaw gateway logs --tail 20

Cost Savings: Real Numbers

Let us calculate the actual savings with concrete numbers. We will use a realistic multi-agent setup: 5 agents (PM, Writer, SEO, DevOps, Support) processing 100 tasks per day.

Single Model Setup (Before)

With a single cloud model handling everything, every routing decision and every execution call goes through the API.

Dual Model Setup (After)

With a local router model, the 400 daily routing calls cost nothing. Only the 100 execution calls hit the API.

That is $60/month down to $12/month. An 80% reduction. Over a year, you save $576. The only cost is the electricity to run the local model, which is negligible on modern hardware.

If you scale up to 500 tasks per day, the savings become even more dramatic. A single-model setup at that volume costs roughly $300/month. The dual-model approach brings it down to $60/month. That is $2,880 saved per year.

Architecture Diagram: How the Dual-Model Flow Works

Here is what happens when a message enters your OpenClaw system with dual-model routing configured:

Limitations and When to Skip Routing

Dual-model routing is not always the right choice. There are specific scenarios where you should stick with a single powerful model for everything.

Advanced: Cascading Model Strategy

Once you have dual-model routing working, you can take it further with a three-tier cascade. This approach uses different models at each level of complexity.

Tier 1 is the local router (Qwen 2.5 3B) handling all planning and delegation. Tier 2 is a mid-range API model like Claude Haiku or GPT-4o-mini for routine execution tasks like summarization, data extraction, and simple responses. Tier 3 is your premium model like Claude Sonnet or GPT-4o, reserved only for tasks that need the highest quality output: long-form writing, complex analysis, and code generation.

# Three-tier cascade configuration
openclaw config set router.provider ollama
openclaw config set router.model qwen2.5:3b

# Default execution model (mid-tier, cheaper)
openclaw config set execution.provider anthropic
openclaw config set execution.model claude-haiku

# Premium model for specific agents
openclaw config set agents.writer.model claude-sonnet-4-20250514
openclaw config set agents.analyst.model claude-sonnet-4-20250514

With a three-tier cascade, you can push savings even further. Routine tasks that make up 60-70% of execution calls go to the cheaper mid-tier model, and only the 30-40% of tasks that truly need premium quality use the expensive model. Combined with free local routing, total API costs can drop by 90% compared to running everything through a single premium model.

Monitoring Your Dual-Model Setup

After switching to dual-model routing, monitor two things: routing accuracy and cost reduction.

For routing accuracy, check the gateway logs for misrouted tasks. A misrouted task is one where the router sends a message to the wrong agent, and the agent either fails or produces an off-topic response. If your misroute rate is above 5%, your SOUL.md definitions probably have overlapping responsibilities. Tighten the boundaries.

# Check routing decisions in logs
openclaw gateway logs --filter router

# Monitor API costs (only execution calls should show up)
openclaw gateway logs --filter provider=anthropic --tail 50

For cost tracking, compare your API provider dashboard before and after the switch. You should see an immediate drop in total API calls. The calls that remain should be fewer but with higher average token counts, because only execution calls (which generate longer outputs) are hitting the API.

The Bottom Line

Running a single expensive model for every LLM call in your multi-agent setup is like hiring a senior engineer to sort your mail. It works, but it is a waste of talent and money.

A local router model handles the planning and delegation work for free, while your premium API model focuses on what it does best: generating high-quality output. The setup takes 15 minutes, saves 80% on API costs, and often improves response times because local routing is faster than cloud API calls.

Start with Qwen 2.5 3B as your router, keep Claude or GPT for execution, and tighten your SOUL.md files so the router has clear boundaries to work with. Your agents will produce the same quality output. Your API bill will not.

Frequently Asked Questions