Best-of guide
Best AI models for classification
A working shortlist for classification jobs: decision models, small classifiers, and LLMs with schema constraints. Jev is one candidate, not the default winner.
| Task | Decision model | General LLM |
|---|---|---|
| Classification | Structured decision with a bounded label set | Possible, usually via generated text or JSON schema |
| Model routing | Native fit: pick a route, return a typed choice | Possible, but you pay generation cost for a decision |
| AI agents | Use as a judge / router / gate, not as the actor | Use as planner, writer, tool-caller, and explainer |
| RAG | Score, filter, or route retrieved chunks | Synthesize an answer from retrieved context |
| Chat | Not the job. No free-form conversation | Primary job |
| Writing | Not the job. No prose generation | Primary job |
| Coding | Not the job, unless you only need a pass/fail or route | Generate, explain, and iterate on code |
What “best” means here
Not a trophy. Classification is a decision: a label, a score, or a reject. The best model is the one that returns that decision on your taxonomy at a price and latency you can live with. Jev is on the list because TypeSafe trained it for that job. It is not the default winner.
The shortlist
| Option | Use when | Watch |
|---|---|---|
| Jev (decision model) | Closed labels, high QPS, next hop is code | Vendor-heavy evidence; no generation; text/JSON in only |
| GPT / Claude / Gemini + schema | Fuzzy labels, low QPS, or you already pay for that LLM in-request | You pay generation for a decision; keys can still drift |
| Classical classifiers / small encoders | Stable, tiny taxonomy, you have labels | Worse at messy language until you invest in data |
| Vendor moderation / safety APIs | The taxonomy is “allowed / not” and you want a frozen product | You do not control the label set |
Who should start where
- Start frozen if the taxonomy is a handful of keywords. Do not buy a model to replace an if-statement.
- Start with the LLM you already have if volume is low or the class names keep changing.
- Start a Jev bake-off if the same closed enum is called often enough that output tokens show up on the bill. Use the pricing review for the arithmetic, then measure on your labels.
Evidence we still need
A public, multi-task classification suite with labeled methods. One LessWrong monitor writeup is not that suite. Until it exists, this page is a shortlist and a task frame — not a ranking you should paste into a purchase order.
Related: best for routing, Jev vs LLM.