Classify at Volume: Embeddings Beat an LLM Call

If you are sorting incoming items into a fixed set of labels, support tickets into topics, leads into tiers, emails into routes, and you call an LLM to do it on every item, you are paying the most expensive way to classify. When the label set is stable and closed and you have a few example items per category, embedding similarity does the same job for roughly one to two orders of magnitude less money, ten to eighty times faster, and deterministically. You embed a handful of labeled examples once, then assign each new item the label of its nearest neighbors. No fine-tuning, no per-item chat call.

Most automation advice skips this because it treats "use AI" as "call a chat model." Classification is the one job where that reflex is wrong most often. There is a tier between a keyword rule and an LLM call, and for high-volume sorting it is usually the right one.

The missing middle tier between a rule and an LLM

Operators tend to see two options for deciding what bucket something goes in: write a rule, or ask a model. Rules are free and instant but brittle, they only catch labels you can pin to an exact token or a deterministic condition. So the moment a label depends on meaning ("is this ticket about billing or about cancellation"), people jump straight to an LLM call. That jump skips the tier that fits most classification work.

Tier	What it is	Cost and speed	Use it when
Rule or keyword	Exact match, regex, lookup table	Free, instant	The label is a literal token or a deterministic condition (contains "invoice", amount over a threshold)
Embedding similarity	Embed labeled examples once, classify each new item by nearest neighbors	About 0.02 dollars per million input tokens, 10 to 80 times faster than a chat call, deterministic	A stable closed label set decided by meaning, and you have example items per category
LLM call	Prompt the model with the labels and let it pick	One to two orders of magnitude more per item, slower, nondeterministic	The label needs reasoning or policy, the output is open-ended, you have no labeled examples, or volume is tiny

The embedding tier is the one people forget. An embedding turns a piece of text into a vector, a list of numbers, positioned so that things with similar meaning sit close together. Classification then becomes geometry: embed ten to thirty real examples of each label, store those vectors, and for every new item, embed it and find which labeled examples it lands nearest. Take the majority label of the closest few. That is the whole method. We covered where the rule-versus-AI line actually falls in when should an automation use AI instead of a rule; this is the tier that sits just above the rule and below the chat call.

Why it wins: cost, latency, and determinism

The gap is not small, and it is measured, not hand-waved.

Cost. As of July 2026, OpenAI's text-embedding-3-small runs 0.02 dollars per million input tokens, and embedding calls bill input only, with no output charge. A short ticket is roughly 200 tokens, so a hundred thousand of them is 20 million tokens, about 40 cents. Run those same hundred thousand tickets through a per-item call to even a cheap chat model and you are paying for the ticket, a prompt that lists the labels, and a few output tokens on every single one, which lands in the single-digit to tens-of-dollars range per month depending on the model. A Thumbtack engineering study put the annual figure at 9,179 dollars for embeddings versus 92,325 dollars for GPT-4o mini prompting across 63 million requests, a clean ten times. A separate cost-aware study of a fixed 14-class ontology task found encoder-based classifiers cost 7 to 11 dollars per million requests against 463 to 2,702 dollars for LLM prompting.

Latency. In the same Thumbtack study, classifying text by embedding took 0.0153 seconds against 1.22 seconds for prompting, about 81 times faster, and images 14 times faster. The cost-aware study measured encoder median latency at 133 to 206 milliseconds versus 406 to 1,128 milliseconds for the LLM. When a person is waiting on a route or a queue is draining overnight, that difference compounds.

Determinism. The same input embeds to the same vector, so nearest-neighbor classification gives the same answer every run. An LLM classification can flip between two labels on identical input, because a hosted model is not run-to-run identical. That means the embedding classifier can be unit-tested with exact-match assertions, and its confidence score (the similarity distance) actually tracks how close the call was, unlike a chat model's self-reported number. The Thumbtack study found the embedding approach was better calibrated, a Brier score of 0.78 against 0.93 for prompting.

And accuracy holds up. This is the part that surprises people. On fixed-label classification where you have real labeled examples, embeddings are at least competitive and often ahead. The cost-aware study found fine-tuned encoders reached 99.40 macro-F1 against a best LLM zero-shot score of 98.83 on the same 14-class task. The reason is blunt: for a closed, learnable label set, a bigger generative model does not have anything extra to contribute, it just costs more and varies more. Your own examples encode your taxonomy better than the model's general world knowledge does.

A worked example: routing 100,000 support tickets a month

Say you route a hundred thousand tickets a month into eight categories: billing, cancellation, bug report, feature request, shipping, account access, refund status, and general. Each ticket is about 200 tokens.

The LLM-per-ticket build sends each ticket plus a prompt describing the eight labels and asks the model to return one. With a cheap model that is a few hundred input tokens and a handful of output tokens per call, times a hundred thousand, landing somewhere between roughly 8 dollars a month on a nano-class model and about 40 dollars on a small workhorse model like Claude Haiku 4.5. It also runs at chat-call latency and can return a different label on a re-run.

The embedding build starts with setup: collect 15 to 30 real tickets you have already labeled for each of the eight categories, embed them once, and store the vectors. That one-time embed is a few hundred short texts, pennies. Then in the live flow, each incoming ticket gets one embedding call and a nearest-neighbor lookup against the stored examples. The monthly embedding cost is that 20 million tokens at 0.02 dollars per million, about 40 cents, and you can halve it with the batch endpoint for any non-urgent backlog. Same eight-way routing, one to two orders of magnitude cheaper, faster, and repeatable.

The synthesis is the best part. You do not have to pick one tier and eat its weakness. Run the embedding classifier first and read the similarity margin: if the nearest label is a clear winner, keep it; if the top two labels are close or nothing is near, escalate just that item to the LLM. That is a cascade, and it is exactly the pattern in cut AI costs with a two-tier model cascade, with the embedding classifier as the cheap first tier instead of a small chat model. Most of your volume never touches the expensive call.

When to still call the LLM

The embedding tier is not a universal replacement, and pretending it is will burn you. Here are the cases where it breaks, and what to do about each.

1. You have no labeled examples yet. Nearest-neighbor needs reference points. On a brand-new label set with zero examples, you cannot embed your way in. Use the LLM zero-shot to get started, and even to help label the first batch, then switch to embeddings once you have 15 to 30 solid examples per category.
2. The label needs reasoning, not similarity. "Is this a refund request that falls outside our 30-day window" is not one classification, it is a classification plus a policy check against an order date. Similarity cannot apply a rule. Split it: classify the intent with embeddings, then apply the date rule in a normal condition, or hand the whole judgment to a model.
3. Category boundaries genuinely overlap. If two labels mean almost the same thing, the nearest neighbor flips between them on tiny wording changes. The fix is a confidence margin, if the top two are within a small distance of each other, treat it as ambiguous and route that item to the LLM or a human rather than guessing.
4. The label is defined by a keyword, not meaning. If "urgent" is literally the word urgent in the subject line, a rule beats both embeddings and an LLM. Do not reach for a vector when a string match is the actual definition.
5. The signal is buried in a long document. Embedding a 60-page contract into one vector blurs everything into an average and the class signal disappears. Embed the relevant section, not the whole file. The reason is the same context-dilution effect we described in running AI over a long document.

One plumbing note that trips people up. If your text generation runs on Claude, your embedding step will not. Anthropic does not ship its own embedding model as of 2026 and points to third-party providers like Voyage AI, while OpenAI offers embeddings directly. It is a two-vendor setup, and worth deciding on purpose rather than discovering mid-build. This is a different use of the same primitive than a knowledge base, embeddings here decide a label, not fetch a passage, which is why this is not the same call as do you need a vector database for AI on your docs.

What to do this week

Open your highest-volume classification step, the one that runs on every ticket, lead, or message, and ask three questions. Is the label set closed and stable, or does it change constantly? Is the label decided by meaning, or by a keyword you could match with a rule, or by a policy that needs reasoning? Do you have, or can you quickly assemble, 15 to 30 already-labeled examples per category? If the set is stable, meaning-based, and you can gather examples, that step is an embedding classifier, not a chat call, and moving it will cut its cost by an order of magnitude while making it faster and testable.

Sorting incoming work into the right lane is most of what a real customer acquisition and routing system does, and we build those on the cheapest tier that classifies each item correctly, rule, embedding, or model, in that order of preference. If you want a read on which of your classification steps are overpaying for an LLM call they do not need, send us the flow and we will mark each one rule, embedding, or model, with the monthly number next to it.