The Case for Specialized Models

A customer asks, “Has order BK-907 been dispatched yet?” The bookshop assistant can write a pleasant answer about delivery times, but the customer needs something more specific — the current status of one order.

To answer, the application must look up BK-907 in the shop’s order records. If the record says it has been dispatched, the assistant can use that information to compose a reply. Without the lookup, even a convincing answer could be wrong.

There is a smaller decision inside this exchange: where should the application look? For this request, the answer is the shop’s database. For a question about an uploaded receipt, it would be the document. For a calculation with all the numbers supplied, it would be a calculator.

This book begins with a model whose entire job is to make that decision. By the end of this chapter, you will be able to describe its job, work through a few examples yourself, and explain why we might want to specialize a small model for it.

You can follow the book with basic Python — functions, dictionaries, loops, and running commands. No previous experience building a language model is required, because we introduce the model and training concepts as we need them.

The experiments ran on an Apple Silicon Mac through PyTorch’s MPS backend. The companion code also carries CUDA and CPU paths, but the recorded results did not use them, so a reader on another machine should expect to confirm the setup before trusting a number. The starting model is small enough for an ordinary laptop: its weights occupy about 2.4 GB in memory, and the whole baseline process peaked near 5 GB. Chapter 4 gives the exact versions and sizes, and every chapter’s results can be read from the companion’s saved files without running the model at all.

One request, two different jobs

A language model produces text in response to the text it receives. A general-purpose model has already been trained to handle many kinds of language tasks, such as explaining an idea, summarizing a passage, or drafting an email. That existing training gives us a starting point; we do not have to build its language abilities from nothing.

Our bookshop could ask a general model to decide which tool each request needs, and the same model might also help write the final answer once the application has gathered the information. These are two different jobs, even if one model performs both.

Consider the order question again. The application needs to:

1. Recognize that the answer requires private order records.
2. Check that the requester may access those records and look up the order.
3. Turn the retrieved status into a useful reply.

We will give the first job to a router: a component that chooses where a request should go next. Its output is one destination label:

Request: Has order BK-907 been dispatched yet?
Route:   DATABASE

That is the router’s complete answer — it does not locate the parcel or tell the customer when it will arrive. The application reads DATABASE and sends the request toward the component that can work with order records.

The permission check still belongs to the application, because choosing the right destination does not establish that the customer is allowed to see a particular order.

Separating these jobs gives us something manageable to study: we can judge whether DATABASE is the right route without connecting a live database or building the whole assistant.

Give the router a clear job

A specialist is a model assigned a narrow job that we can check. Our router specializes in choosing destinations. Later specialists will do other jobs, but each will have a clear input and an expected kind of output.

We call that agreement an input/output contract. For the router, it is simple: receive one request and return exactly one of five labels.

Destination	What the request needs	Example
DATABASE	Private operational records	“Which of our invoices are unpaid?”
DOCUMENT	A supplied document or named policy source	“Extract the vendor from this receipt.”
WEB	Current external facts or an explicit web lookup	“Is the city library open today?”
CALCULATOR	Arithmetic with all necessary values supplied	“What is 12 percent of 250?”
GENERAL_LLM	General explanation, drafting, or clarification	“Explain why invoices exist.”

The last label sends a request to a general language model. The letters LLM stand for large language model; here, GENERAL_LLM is simply the name the application expects for that destination.

Try choosing a destination for each of these before reading on:

• “What is an invoice?”
• “What amount is printed on this attached invoice?”
• “Which invoices in our records are overdue?”

All three mention invoices, but they need different sources. The first asks for a general explanation, so it goes to GENERAL_LLM. The second needs the attached document, so it goes to DOCUMENT. The third needs private records, so it goes to DATABASE.

The useful question is “What information or capability does this request need?” Matching a keyword such as “invoice” would send all three to the same place and miss that distinction. If explicit rules handle the requests your application accepts, an ordinary Python function may be enough. We investigate a language model because customers can express the same need in varied wording; whether it earns the extra complexity is something to test.

A calculation can have the same wrinkle: “Calculate our revenue yesterday” requires DATABASE, because the numbers must first be found. “Sales were 120 and 180; calculate their sum” belongs to CALCULATOR, because both numbers are already present.

These examples are also how we make the job clear to ourselves — if we cannot agree on the destination, we need to settle the routing rule before asking a model to follow it.

Two ways to teach the same job

How do we get an existing model to produce these labels? We will begin by giving it instructions. The text we send to a model is called a prompt, and it can contain the task, examples, and the new request.

Here is a small illustration of what that might look like — the idea rather than the complete instructions used in our experiments:

Choose the destination needed to handle the request.
Reply with only the destination label.

Example request: Which of our invoices are unpaid?
Example answer: DATABASE

New request: Has order BK-907 been dispatched yet?

The desired response is DATABASE. The example shows the model both a routing decision and the form of the answer. We can change the instructions or add examples, then try another request.

This is prompting: during an ordinary request, the model uses the supplied text without changing its stored parameters. Parameters, also called weights, are the internal numbers that training adjusts to shape a model’s behavior.

Fine-tuning is a separate training process that starts with an already-trained model and adjusts parameters using examples for our task. For the router, an example pairs a request with the destination we want, and training uses those pairs to adjust how the model responds to requests.

After training, we use the resulting model to route new requests — we still send it an input, and we may still include instructions. Whether it handles unfamiliar wording correctly is something we must test.

The distinction is where the examples do their work. In prompting, they are part of the text supplied when asking for a response; in fine-tuning, they are used during a training stage that changes the model. Chapter 2 explores the options; Chapter 3 works through how training changes those internal numbers.

Figure 01.1. Read each path from bottom to top. Prompting supplies examples with the request. Fine-tuning uses examples in a separate training stage; a new request enters the resulting model afterward. Instructions may still accompany that new request.

A model can therefore serve as our routing specialist before we fine-tune it — “specialist” names its job, and fine-tuning is one way we can try to improve its performance at that job.

Why try a small model?

Once the job is narrow, we can ask whether a smaller model can handle it well enough. A router only needs to choose among five destinations; it does not need to write the bookshop’s marketing copy or explain every book in the catalogue.

That limited output makes routing a useful starting experiment, although understanding varied customer wording can still be difficult — five possible answers do not guarantee an easy task.

Our main starting model is Qwen/Qwen3-0.6B: Qwen3 is the model family, and 0.6B indicates roughly 600 million parameters. A saved set of model weights is called a checkpoint; the laboratory chapter identifies the exact checkpoint we use so readers can reproduce the experiments.

A smaller model may need less memory and may reduce the time or cost of handling a request. Those possibilities make it worth investigating, but they are not results we can infer from its name or parameter count.

For the bookshop, three questions matter together:

• Correctness: Does it send requests to the right destination?
• Speed: Does the customer receive a useful answer soon enough?
• Cost: Are the computing and maintenance costs reasonable for the shop?

Suppose the small router often sends order questions to GENERAL_LLM. A quick, inexpensive decision would not help: the application would still lack the order records it needs. Likewise, if most requests require another model to correct the route, the extra step may make the assistant slower and more expensive.

Our aim is to find the smallest model that reliably meets the task’s requirements — a larger model remains an option when the smaller one cannot do the job.

Make improvement visible

Before training anything, we need to see how well prompting works. That first result is our baseline: the starting performance against which we compare later attempts.

We can begin by writing requests, deciding their expected destinations, and comparing the model’s answers with those decisions. Here is a made-up example to show the checking process — the responses are illustrative, not results from the book’s model.

Request	Expected route	Illustrative response	Correct?
Has order BK-907 been dispatched?	DATABASE	DATABASE	Yes
What is an invoice?	GENERAL_LLM	DATABASE	No
Find the total on this attached receipt.	DOCUMENT	DOCUMENT	Yes
What is 12 percent of 250?	CALCULATOR	CALCULATOR	Yes
Search the web for today’s library opening hours.	WEB	WEB	Yes

Four of the five responses match the expected route. Accuracy is the fraction that are correct: here, four divided by five, or 80 percent.

The mistaken invoice answer gives us something more useful than the percentage alone: the request asks for an explanation, but the response sends it toward private records. That gives us a specific mistake to investigate. The table does not tell us why the model made it; we would need further examples or comparisons to explore that.

We also check the form of each answer, because the program expects DATABASE, not a paragraph explaining why a database might help. An answer must both use the agreed label format and choose the right destination, and we keep malformed answers among the failures when counting correctness.

Five examples are enough to explain this process, but far too few to establish that a router will handle a shop’s traffic reliably. Our experiments will use separate requests for testing, kept out of the examples supplied to the model. Chapter 5 introduces the measured baseline and the tools for understanding its errors.

Training then has a clear question to answer: does it improve on that baseline enough to be useful? If a simple prompt already works well, we may have solved the problem, and fine-tuning would need another reason — such as reducing the repeated instructions while preserving correctness. The cost of preparing examples and maintaining a trained model belongs in that decision too.

Our first training comparison answers a narrower question: can training improve routing when the prompt contains the policy and no examples? In Chapter 7 the unchanged model routes 30 of 50 fresh test requests correctly; after a full fine-tune that took under a minute on the author’s machine, it routes all 50. Chapter 8 reaches the same result by training only a small add-on to the model, called an adapter, of about 1.1 million values. These results show an improvement over that no-example prompt on a small synthetic test. The stronger example-based prompts have not been evaluated on those same fresh requests, so whether training earns its cost over them remains open.

The bookshop grows three specialists

Routing is our first project because each answer is small and easy to inspect. The next two projects gradually ask the model to produce more.

The extractor receives document text and returns selected fields. For example, it might read a supplier invoice and return the invoice number, vendor, amount, and payment terms. We can check those values against the source document. This extends the task from choosing one label to finding several pieces of information. It also changes what “correct” means: a neatly structured answer that gives the wrong amount is still wrong. The structure can be checked by a program; the amount has to be checked against the invoice.

The SQL specialist receives a question and a description of the database’s tables and columns. It proposes a SQL query: an instruction for retrieving data from that database. We then check whether the query produces the answer the question requires. A query that runs without error can still answer the wrong question: filtering by invoice date instead of payment date can give a total for the wrong month.

Each project raises what we must check before trusting an answer:

Specialist	What it returns	Easy to check by program	What actually has to be true
Router	One destination label	The output is one of the five labels	It is the right destination
Extractor	Several fields from a document	The fields are well-formed	The values match the document
SQL specialist	A query	The query runs	Its results answer the question

These basic checks can reject some bad outputs. Passing them still does not establish that the answer is correct. The fourth column describes what we ultimately need to measure.

The three projects can eventually cooperate: for a request about unpaid invoices, the router chooses DATABASE and the SQL specialist proposes how to retrieve the records. For an attached receipt, the router chooses DOCUMENT and the extractor finds the requested fields. The surrounding application controls access and checks the results.

We will build toward that system one job at a time. The fine-tuning-specialists companion contains the code, example data, and saved experiment results; Chapter 4 sets up the environment. For now, the important first step is having a task whose answers we can recognize and check.

Exercises

1. Separate the route from the answer

A request says, “How much did customer C-615 spend last year?” What should the router output? What work remains before the application can answer?

Answer

The router should output DATABASE, because the answer depends on private purchase records. The application must check that the requester may access those records, retrieve or calculate the total, and use the result to answer. The route itself contains neither the amount nor permission to read it.

2. Choose by the information needed

Assign routes to “Explain what a discount is,” “What is 10 percent of 80?”, and “What discount did we apply to order BK-907?” Explain what distinguishes them.

Answer

The routes are GENERAL_LLM, CALCULATOR, and DATABASE. The first needs a general explanation. The second supplies the numbers needed for arithmetic. The third needs an order record, even though a discount can involve a calculation.

3. Locate where the examples are used

One developer puts five solved routing examples in the text sent with each new request. Another uses request-and-route pairs in a training process that adjusts model parameters. Which is prompting, and which is fine-tuning? Can both models have the job of routing?

Answer

The first is prompting: the examples accompany the request. The second is fine-tuning: the examples are used during training to adjust parameters. Both models can be routing specialists, because specialization describes their assigned job.

4. Check a result before celebrating it

In the illustrative table, suppose the invoice response changes from DATABASE to GENERAL_LLM, while every other answer stays the same. How many answers are now correct? Does that establish that the router is ready for real customers?

Answer

All five answers are now correct, giving 100 percent accuracy on these five examples. That establishes success on this small set only. We still need separate test requests that cover more of the shop’s work, and measurements of speed and cost. Fixing a known example is useful, but it does not show how the router handles new requests.

A small job we can learn from

We now have a first task: read a request and choose the destination with the information or capability needed to handle it. We can explain the routing decisions ourselves, specify the expected answer, and count mistakes.

That gives us a practical way to investigate specialization: start with an existing model, try instructions and examples, and measure what happens. Before we train it, the next chapter examines what we can accomplish through prompting and supplying the right information.

Next: Do You Actually Need Fine-Tuning?