Shuichiro Ogawa
日本語

Notes · updated 2026-07-05

Model Tiering Patterns for Fable

Claude Fable 5 is the top-tier model in the Claude family, with a high cost per token. As discussed in frontier-model-premium-history-debate, paying for a frontier model buys three things: early access, learning opportunities, and time. There is no necessity to deploy the frontier model on every task. In real workloads, 60-80% of requests can be handled adequately by Haiku or Sonnet, and restricting Fable to complex reasoning and final judgments promises a substantial cost reduction while maintaining quality.

1. Router-Based Model Selection

Model routing is a technique that automatically dispatches requests to the appropriate model according to task difficulty and type. A classifier analyzes each request and judges whether a cheaper model suffices or a higher-tier model is required.

RouteLLM implements multiple router methods trained on Chatbot Arena preference data and was accepted at ICLR 2025 (Ong et al., 2025). The matrix factorization router showed the best performance, achieving up to 3.66x cost reduction at CPT 80% (the threshold routing 80% of calls to the cheaper model). It also generalizes to model pairs not included in training (such as Claude 3 Opus and Sonnet). OpenRouter’s Auto Router adopts a NotDiamond-based classifier and can continuously adjust the weight of quality versus cost via the cost_quality_tradeoff parameter (0-10).

Within the Claude family, requests are dispatched to four tiers according to the classifier’s judgment. Simple tasks (classification, extraction, boilerplate responses) are assigned to Haiku, standard tasks (summarization, code generation, question answering) to Sonnet, complex reasoning and agent workflows to Opus, and planning and final judgments demanding the highest quality to Fable.

“Routing” alone promises a 40-70% cost reduction. RouteLLM’s up-to-3.66x cost reduction is an upper bound for combinations with large price gaps between models, and the effective value within the Claude family depends on the price ratios.

The router’s classification accuracy determines the quality ceiling of the entire system. Quality degradation on tasks misclassified to a cheaper model is directly visible to users. Training the classifier requires data mapping tasks to their optimal models, imposing an initial cold-start investment.

Academic research in June 2026 revealed the structural limits of “routing.” In a systematic evaluation using LLMRouterBench, routing achieves up to 31.7% cost reduction relative to the single best model, while accuracy improvement stays within at most 4% (Zhao et al., 2025). The commercial router OpenRouter recorded a -24.7% performance drop against the single best model. A phenomenon was confirmed in which the accuracy differences among the top 15 routers converge to 0.23pp, and it was named the routing plateau (Chari et al., 2026). Expanding the model pool yields diminishing returns. Because RouteLLM’s generalization capability also operates inside this ceiling, quality improvement from routing alone has an upper bound.

2. Cascade

A cascade (cascade / fallback) first attempts processing with a cheap model and escalates to a higher-tier model only when the output quality is insufficient. FrugalGPT (Chen et al., 2023) pioneered the formalization of this technique, followed by multiple independent implementations such as AutoMix and EcoAssistant.

What separates success from failure in a “cascade” is the accuracy of the post-hoc quality estimator. Unless it can accurately judge whether the lower model’s output is sufficient, either costs balloon from unnecessary escalations or quality-deficient outputs pass through as-is. In an evaluation on SWE-Bench, a method combining routing and cascading achieved an AUC of 54.12, consistently outperforming routing-only at 40.47 and cascading-only at 38.52.

Within the Claude family, escalation proceeds in the order Haiku -> Sonnet -> Opus -> Fable. Because the majority of requests complete processing with Haiku or Sonnet, only a small fraction of all requests reaches Fable.

The “cascade” showed results 34% better than “routing” alone on SWE-Bench (AUC 54.12 vs 40.47). However, each escalation adds latency, so delay becomes a problem in uses requiring real-time responses.

3. Orchestrator + Worker

A pattern that assigns the frontier model to planning and judgment and lower-tier models to execution and labor. It is a form of the hierarchical orchestration organized in agentic-coding-orchestration-patterns.

Anthropic itself has validated this configuration. A multi-agent system with Opus as the lead agent and Sonnet as subagents outperformed a single Opus in 90.2% of cases.

Claude Code’s subagent mechanism directly supports this pattern. The model frontmatter of a subagent definition can specify one of haiku, sonnet, opus, or fable, and model resolution follows a four-level hierarchy. Priority runs in the order (1) the CLAUDE_CODE_SUBAGENT_MODEL environment variable, (2) the per-invocation model parameter, (3) the subagent definition’s frontmatter, (4) the main conversation’s model. Fork subagents reuse the parent’s prompt cache and therefore run more cheaply than fresh subagents.

On a 14M-token build, splitting planning to the frontier model and execution to cheaper models achieved a 57% cost reduction. This project also adopts the same configuration, classifying its 27 subagents into 11 Opus (debate leads, academic critics) and 16 Sonnet (collection researchers, utilities).

A multi-agent configuration consumes roughly 15x the tokens of a single chat. The cost reduction comes from the difference in model unit prices, but inter-agent communication overhead pushes up the total token volume. Adoption is justified only when the task’s value exceeds this overhead.

4. Task Classification

A pattern that statically assigns models per task type. Whereas the “router” judges dynamically per request, the “task classification” pattern fixes assignments at design time.

Anthropic’s recommended allocation is as follows. Haiku is assigned to classification, routing, extraction, and high-volume simple queries. Sonnet is assigned to most production tasks. Opus is assigned to complex reasoning and agent workflows.

Claude Code’s built-in agents also adopt this approach. statusline-setup is assigned Sonnet, and claude-code-guide is assigned Haiku. This project’s 27 subagents are likewise classified by task nature into 11 Opus (the core of debates) and 16 Sonnet (collection and utilities).

The rough allocation is three-tiered: route 40-60% of all requests to the cheapest (Haiku), 30-40% to the middle (Sonnet), and 10-20% to the most expensive (Opus or Fable). A typical configuration has Sonnet as the workhorse handling 60-70% of request volume.

Because no dynamic judgment is performed, implementation is simple and predictable, but when a task’s actual complexity does not match the prior classification, either quality over-investment or missed cost savings result.

5. Stacking Prompt Caching and Batch Processing

A pattern that layers caching and batch processing on top of model selection (patterns 1-4) to compress costs further.

A prompt cache hit is 90% cheaper than standard input tokens. On Sonnet 4.6, a cache hit costs $0.30/MTok versus $3.00/MTok for regular input, one tenth the price. ProjectDiscovery achieved a 59% cost reduction by introducing prompt caching, raising its cache rate in stages from 7% to 74% and further to 85%. On a task with 67.5M input tokens it realized a 91.8% cache rate, producing a cost difference of roughly 60x.

Layering a semantic cache on top of “routing” can eliminate 40-60% of API calls themselves through cache hits on similar requests.

Combining the five optimization levers (model routing, context compression, prompt caching, prompt optimization, batch processing) promises a 70-85% total cost reduction. In API spend terms, that works out to compressing an outlay of $22.50 down to $2-3.50.

Cache efficiency depends on the stability of the prompt structure. Operations that frequently change the system prompt will not raise the cache hit rate. Batch processing is limited to asynchronous uses that tolerate latency and cannot be applied to real-time responses.

The Overall Picture of Cost Reduction

LeverReductionNotes
Routing alone40-70%RouteLLM reaches up to 3.66x at CPT 80%, but that is an upper bound for configurations with large model price gaps. Due to the routing plateau, accuracy improvement stays within at most 4%
Cascade+34% vs routing-onlySWE-Bench AUC: combined 54.12 vs routing-only 40.47
Orchestrator + worker57%Measured on a 14M-token build
All levers combined70-85%With the five optimizations stacked ($22.50 -> $2-3.50)

Caveats

  • It has not been verified that improvements on benchmarks (cost reduction rates, accuracy retention rates) translate directly into practically meaningful quality differences.
  • Systematic research on the reliability of quality judgment in “cascades” and on staged escalation is limited.
  • The performance claims of commercial routers (OpenRouter and others) lack independent third-party verification.
  • The effect of prompt caching depends on conditions such as multi-agent configurations and very long system prompts; it is weak for sequential processing of short prompts.

References

← All Notes · Home