When an application needs to choose its next action, the useful output is often an option ID and a probability. Kodoom adapts a small decision model to Persian and English workflows: support triage, invoice processing, security incidents, and agent oversight.
The name comes from colloquial Persian: کدوم, meaning “which one?” Supply the evidence, ask a question, and define the available answers. The model scores those answers without generating an autoregressive response.
Model card and checkpoints on Hugging Face.
The engineering problem
A workflow can have a clear answer space while its evidence arrives as natural language or structured records. Should an agent continue, stop, or ask for human review? Which invoice-processing action fits the supplied context? These are decisions among known alternatives.
Kodoom builds on Laya-multilingual, a bidirectional mmBERT-base encoder with a decision head, approximately 322 million parameters in total. Candidate options are supplied per request rather than fixed to one classifier’s label vocabulary. The model returns a probability for each option, leaving action selection and review thresholds to the caller.
Bilingual data and fine-tuning
The work combined English–Persian training data, full encoder and decision-head fine-tuning, and evaluation tooling. The recorded run used the four workflow families in the typed-decisions dataset, with five decision questions per case. English and Persian variants retain aligned questions, option descriptions, and the same probability targets.
| Partition | Cases | Paired English–Persian questions |
|---|---|---|
| Fine-tuning | 960 | 4,800 |
| Development | 120 | 600 |
| Reserved calibration | 120 | 600 |
| Test | 400 | 2,000 |
The fine-tuning partition contains 9,600 language-specific examples. Cases and their language variants stay together across partitions to prevent an English case from entering training while its Persian translation enters evaluation.
Training ran on two T4 GPUs, with 50% Persian / 50% English replay, soft-target cross-entropy, an effective batch size of 32, and 600 optimizer updates over two epochs. Checkpoint selection used development criteria.
What the evaluation shows
The primary comparison uses 60 Persian questions from 12 held-out cases: three cases and 15 questions per workflow. All three models receive the same selected questions, option order, and Persian normalization.
| Workflow | Kodoom | Laya-multilingual | DibaOne X1 |
|---|---|---|---|
| Invoice processing | 73.3% | 33.3% | 33.3% |
| Security incidents | 80.0% | 60.0% | 46.7% |
| Agent trace observability | 66.7% | 46.7% | 40.0% |
| Customer service | 53.3% | 33.3% | 13.3% |
| All four workflows | 68.3% | 43.3% | 33.3% |
That is a 25.0 percentage-point gain over the base model and 35.0 points over DibaOne X1. A paired source-case bootstrap gives a 95% interval of +5.0 to +46.7 points for the gain over Laya-multilingual on this sample.
The targets are probability distributions. Kodoom also improves agreement with those targets: negative log-likelihood falls from 1.899 to 1.003, Brier score from 0.416 to 0.126, and 15-bin expected calibration error from 0.245 to 0.118 relative to the base model.
Choice, yes/no, and score questions all use a shared choice interface for this comparison. Yes/no becomes a two-option choice; a score becomes a choice among the supplied levels. These results measure answer selection through that conversion, rather than native scoring-head performance or continuous scoring.
How far the result travels
These are synthetic English scenarios with machine-translated Persian fields. The sample covers new cases within the same workflow families used for fine-tuning, and only 12 of the 400 test cases. The results are preliminary; matching synthetic targets does not establish production reliability.
Two external Persian samples give smaller observed gains: 58% versus 56% on 100 ParsiNLU paraphrase questions, and 73% versus 72% on 100 PersianQA answerability questions. Their paired accuracy-difference intervals include zero. PersianQA here tests whether a passage contains an answer, not answer extraction. These samples do not establish a general Persian-language advantage.
The model card also describes English gains, but its published numerical tables and confidence intervals cover Persian only. The figures above are Persian results.
Using the release
The evaluated checkpoint
is under laya-full-v1/best/, at revision
f315b5da129e2b1df55d971ddf2053c3cbfe5282. The latest/ directory is a
training-resume state. The repository requires a Hugging Face login and acceptance
of its access conditions to download files.
The measured setup uses a 1,024-token total input budget, including a 256-token question/options budget. Evidence can be lost to truncation. Review thresholds and probability calibration need evaluation on separate examples from the intended application; this experiment does not establish a latency or hardware-efficiency advantage.
The model repository declares Apache-2.0. The bilingual training dataset is planned for release at Farahani/kodoom-en-fa. The model card carries the detailed evaluation protocol, frozen model identities, evidence checksums, and attribution.