We sent 45 messages to a fictional physiotherapy practice through four open decision models. Cloudflare's Clef-flash sorted 43 of them correctly, the larger Clef 42, Amazon's Strands Decider 2B 38 and Convai's Laya 33. A keyword rule scored 32, but only because it was written for the first 30 messages: on 15 new ones it got 4 right. We could not measure Jev ourselves, because OpenRouter does not serve it in its EU data region. In published comparisons Jev sits close to Clef. For routing customer messages written in German, an open decision model is now a serious option, as long as you test it on your own data.
As of 6 November 2026, Laya has been available since 18 September, Clef since 1 October, and Jev 1.14 since 14 October. All of them accept the same request: a state plus typed questions. How decision models differ from LLMs such as Claude, and what they cost, is covered in Jev vs LLM. This article only asks how well they sort real-looking requests.
The result at a glance
| Model | Correctly routed | Of the 15 new requests |
|---|---|---|
| Clef-flash (Cloudflare) | 43 of 45 | 14 of 15 |
| Clef (Cloudflare) | 42 of 45 | 14 of 15 |
| Strands Decider 2B (Amazon) | 38 of 45 | 12 of 15 |
| Laya (Convai) | 33 of 45 | 11 of 15 |
| Keyword rule | 32 of 45 | 4 of 15 |
| Jev (TypeSafe) | not measured | close to Clef in third-party tests |
How we tested
We wrote 30 messages to a made-up practice: 24 in German and 3 each in English and Russian, including typos, sarcasm, praise and spam. Before running anything, we fixed the right answer for each one: appointment, billing, treatment, complaint or other. A second question asked whether the practice has to react today.
Next we wrote a simple keyword rule, the kind you build in ten minutes, with words like “Rechnung” (invoice) or “Schmerz” (pain). Only after that did we write 15 more messages that had no influence on the rule or the questions. That held-out set is the more honest half of the test.
Every model got the same questions in this format, with no fine-tuning and no examples. The questions were in German, as the practice would set them up:
{
"state": "Super, schon wieder eine Absage von Ihrer Seite. Dritter Termin in Folge. Toller Service.",
"questions": {
"kategorie": {
"type": "choice",
"instructions": "Worum geht es in der Nachricht an die Physiotherapie-Praxis?",
"criteria": {
"termin": "Termin vereinbaren, verschieben oder absagen",
"abrechnung": "Rezept, Verordnung, Rechnung, Zuzahlung, Krankenkasse, Gebühren",
"behandlung": "Fragen zu Beschwerden, Übungen, Ablauf oder Wirkung einer Behandlung",
"beschwerde": "Unzufriedenheit, Kritik an Personal, Wartezeit oder Service",
"sonstiges": "Öffnungszeiten, Anfahrt, Bewerbungen, Lob, Werbung und alles andere"
}
},
"dringend": {
"type": "noul",
"instructions": "Muss die Praxis noch heute reagieren, etwa weil ein Termin heute betroffen ist, eine Frist heute endet oder akute Beschwerden nach einer Behandlung auftreten?"
}
}
}
The example state reads, roughly: “Great, another cancellation from your side. Third appointment in a row. Lovely service.” Laya and Strands Decider ran locally on an M4 Mac with 24 GB of memory. Clef and Clef-flash ran on Cloudflare Workers AI, measured from our machine in Germany. The limits: 45 messages is a small sample, we wrote them ourselves, and each model ran once. Two of the answers are debatable, as shown below.
Why the keyword rule only looked good on paper
On the 30 messages it was written for, the rule got 28 right, almost level with Clef. On the 15 new messages it dropped to 4. “Könnten Sie mich für Mittwoch früh eintragen?” (could you put me down for Wednesday morning?) contains no word like “Termin”, and a message about a “Physio-Verschreibung” never says “Rezept”. The models without any training barely moved: Clef-flash scored 29 of 30 and 14 of 15.
The same trap applies to prompts and question wording you tune against a model. Always check the result on messages nobody looked at while writing the setup. Otherwise you measure how well your rule knows your examples, not how it handles a normal week.
Where the models disagreed
| Message | Correct | Who got it wrong |
|---|---|---|
| Sarcastic complaint about a third cancelled appointment (quoted above) | Complaint | Laya and Strands: appointment |
| “I sat in the waiting room for 25 minutes and then was told my appointment wasn't booked. Unbelievable.” (German) | Complaint | Laya: appointment, confidence 1.00 |
| “Thanks for the great care from Ms Weber, my knee is back to normal!” (German) | Other | Laya and Strands: treatment |
| “Since this morning my lower back is locked and I can't stand up straight. Can someone see me today?” | Appointment | Laya and Strands: treatment |
| “I forgot my appointment today, sorry! Do I have to pay a no-show fee?” (German) | Billing (debatable) | All models: appointment |
| “I still haven't been refunded for the cancelled sessions.” (Russian) | Billing (debatable) | All models: complaint |
We kept the last two cases in the score even though you could argue them either way. When you write questions for your own business, put such edge cases into the category descriptions, for example “no-show fees belong to billing”.
Clef-flash: most hits and honest uncertainty
Clef-flash routed 43 of 45 messages correctly and was never wrong with high confidence. Both errors came with a confidence below 0.65. For 9 messages it reported less than 0.5. With that threshold, 9 of 45 messages would go to a person, including one of the two errors. It caught all 10 urgent messages, with 2 false alarms.
According to Cloudflare, Clef-flash on Workers AI costs $0.038 per million input tokens, and output is free. The median was 0.18 seconds per request, network included. The weights are published under Apache 2.0, but the model has 9 billion parameters. Cloudflare tested it on an H200 GPU, so it will not run on an office laptop.
Clef: bigger, but not better here
The larger Clef got 42 of 45 and also caught every urgent message. It was slower, with a median of 0.31 seconds, and was wrong once with 0.86 confidence, on the debatable no-show fee. At $0.24 per million input tokens it costs more than six times as much as Clef-flash. For routing short customer messages, the bigger model brought no gain in our test.
Strands Decider 2B: runs on a Mac, misses sarcasm
The model from AWS Strands Labs routed 38 of 45 correctly. It read the sarcastic complaint as an appointment request and the praise as a treatment question. It was cautious about urgency: at a 0.5 threshold it caught only 4 of 10 urgent messages, at 0.4 it caught 7, with no false alarms either way. You will need to set the threshold on your own data.
Strands Decider is released under Apache 2.0 and needs no GPU server. Our requests took a median of 0.96 seconds on the M4. We ran it with --device mps; the maintainers say MLX is 1.4 to 1.6 times faster. Your data never leaves the machine.
Laya: fast, but often confidently wrong on German
Laya scored 33 of 45. For German and Russian messages, its built-in router picked the multilingual checkpoint with 322 million parameters. The kind of error matters more than the count: three times Laya was wrong with a confidence of 0.8 or higher, and it filed the waiting-room complaint as an appointment request with 1.00. No threshold catches that.
This matches what Convai publishes. Without fine-tuning, the Laya checkpoints score 0.342 to 0.362 on the Typed Decisions benchmark. A version trained on those tasks reaches 0.766, above the 0.727 Convai cites for Jev. So Laya is a starting point for your own model rather than a finished tool. At about 0.1 seconds per request on the Mac, it was by far the fastest model we ran.
And Jev?
Jev is closed and runs on TypeSafe's API, which OpenRouter also resells. Our OpenRouter workspace only allows the EU data region, and on 6 November OpenRouter rejected the request there: no provider serves Jev in that region. If your data has to stay in the EU, settle that with TypeSafe before you build on it. TypeSafe's documentation mentions a data processing agreement and zero data retention for enterprise customers.
That leaves other people's measurements. On the Typed Decisions benchmark, Convai cites 0.727 for Jev 1.13. An independent run from 16 October, using the same protocol, put Clef at 0.728 and Clef-flash at 0.710. In Cloudflare's own table, Jev leads on a tool-calling task (80.97 against 72.37 for Clef) and trails on banking intents (79.74 against 94.20). So Jev and Clef are close, and the real difference is how you run them: Jev costs $0.042 per million input tokens and is closed, while Clef-flash is open and slightly cheaper.
Which model for which situation
| Situation | Candidate |
|---|---|
| High accuracy without training, cloud is acceptable | Clef-flash |
| You already use Jev | Test Clef-flash alongside; Cloudflare says it speaks the Jev API |
| Data must not leave your premises, no GPU server | Strands Decider 2B with a tuned threshold |
| Plenty of labelled examples, you want your own model | Laya after fine-tuning |
| Categories can be recognised by fixed words | A rule first, see automation without AI |
How to repeat the test with your data
Take 50 to 100 real, anonymised messages and write down where each one belongs before you start. Put another 20 aside and only look at them after the questions are written. Put edge cases like the no-show fee into the category descriptions. Count separately how often a model is wrong with high confidence, because those errors reach the wrong team unchecked. Set the “unsure, send to a person” threshold on this data, and measure response time from your own infrastructure.
Frequently asked questions
Is Laya better than Jev?
Convai says yes, but only after fine-tuning: 0.766 against 0.727. Without fine-tuning, Laya trailed Clef and Strands Decider in our test. We could not measure Jev directly.
Can Clef replace Jev?
Cloudflare says Clef understands the Jev API. In published tests the two are close, and in our test Clef-flash was the strongest model. Only a comparison on your own messages shows whether that holds for you.
Which decision model runs locally on a Mac?
Laya and Strands Decider 2B both ran on an M4 Mac in our test. Clef-flash, with 9 billion parameters, is too large for that.
Do decision models understand German?
Clef and Clef-flash routed every unambiguous German message correctly in our test. Laya made confident mistakes exactly where German sarcasm and criticism came in. Test the languages your customers actually write in.
How much does Clef-flash cost?
On Workers AI, $0.038 per million input tokens, with free output. In our calculation for 100,000 requests a month that comes to $2.28.
If you want to see how a decision model copes with your customer messages, talk to us about AI automation with LindenTech. Bring 30 to 50 anonymised messages and note for each where it belongs and what a wrong routing costs you.