Post-training case study with AI21 Labs

This post is authored by AI21’s Eran Goldstein, Asaf Karnieli, Shaked Meirom and Joanna Kramer, in collaboration with Deel’s Abhijit Mehta, Anthony Li, Michael Matias and Ömer Gözüaçık.

Most of the time, AI gets “close enough.” But, in payroll, a wrong or incomplete answer can mean an incorrect paycheck, a missed filing, or compliance problems.

Thus, Payroll is where Emet started. Emet is Deel’s AI lab for model evaluation and training. Deel, the global HR and payroll platform, runs payroll for 45,000+ companies in 150+ countries. Its in-country payroll specialists know what a complete answer looks like, because they answer these questions every day.

It’s not a simple task to build a payroll model that is reliable and truthful across the globe. Every country has its own tax rules, leave laws and filing deadlines, and many change every year.

On a set of 30 payroll tasks, Claude Opus 5.5 met 64.0% of the expert criteria but fully solved only 26.3%. So we worked with AI21 Labs to change that. In three weeks, AI21 post-trained NVIDIA Nemotron 3.5 Super VL, an open model, against Emet’s expert criteria. On the same 30 tasks, measured by criteria met, the model achieved:

20.7-point quality improvement over the base model
8-point quality improvement over Claude Opus 5.5
50x cheaper model than Opus 5.5 (as measured in cost per 1M tokens)

Scatter plot of quality against cost per 1M tokens on a log scale. Post-trained Nemotron 3.5 Super VL scores 72.03, 20.7 points above its base model’s 51.30, at about $0.40. The other models sit on or below a Pareto frontier that tops out with Claude Opus 5.5 near 64 at about $20: Gemini 3.5 Flash near 61, Inkling-Small and DeepSeek-V4.1-Flash near 52, GPT-6 Luna near 46 and Qwen3.5-9B near 40. — Figure 1 · Post-training lifted Nemotron 3.5 Super 8 points above Claude Opus 5.5 on criteria met for global payroll questions, at 50x lower cost per 1M tokens. Dev-set results.

Below, we describe the evaluation setup and the two-stage post-training recipe. Emet’s expert criteria defined a complete answer. They served as both the filter for training data and the metric at every stage. AI21’s Forward-Deployed Researchers trained the model with supervised fine-tuning (SFT) on the model’s own filtered rollouts, then self-distillation policy optimization (SDPO).

Why global payroll questions are hard for models

The domain

Payroll questions span multiple regions and jurisdictions. The correct answers depend on local regulations, tax rules, employment contracts, and company policy, all of which change regularly. The 30 tasks in this project come from real questions Deel’s payroll teams answer every day, de-identified and rewritten to stand alone.

A typical task, usually a few sentences long, might be: “We have an employee in France who started maternity leave on 14 August. My understanding was that CPAM would fund this. How does the funding and the employer’s part work?”

In addition to getting the answer right, the model also has to phrase the answer in Deel’s terminology, policies and style. Most companies training their own model want the same: answers that sound like their own experts, not like a general model.

Evaluation setup

Emet built 600 payroll tasks with Deel’s in-country payroll specialists. Before training, we split them: 270 for training, 30 as a development set, and 300 held out. All 600 make up EmBench Payroll, Emet’s public payroll benchmark. The training tasks went to AI21 Labs, and both teams used the development set to track progress.

Each question has 3–6 rubric assertions, reviewed by Deel’s in-country payroll specialists. Some are factual (“states the daily cap for the CPAM allowance is 101.94 euros”), while others check what the answer covers or how it frames the situation. With GPT-6 Sol as the judge, answers were graded on two metrics:

Criteria met: the average share of a question’s assertions that the answer satisfies
All criteria met, or fully solved: the share of questions where every assertion is satisfied

Frontier models get most details right, but they’re expensive to run at scale. Claude Opus 5.5 met 64.0% of the criteria but fully answered only 26.3% of the dev questions. In contrast, open models cost far less, but generally start with lower quality results. The base version of Nemotron 3.5 Super scored just 51.3%.

Deel wanted an open model that gives the complete answer without being prompted, at a fraction of frontier cost. That became the brief for AI21 Labs’ Forward-Deployed Researchers. Post-training makes it possible to reach the desired cost-quality operating point.

Training methodology

Where the base model falls short

AI21’s Forward-Deployed Researchers evaluated base Nemotron 3.5 Super against Emet’s criteria and reviewed its failed rollouts. Most were incomplete, not wrong. The model hedged where the law is definite, gave no concrete figures, skipped framing facts, or mixed up related concepts. That pattern suggested it already had most of the knowledge. It just needed steering toward a complete answer.

Figure 2 shows a typical example. When a US employee asked how much sick time they would have by the end of April, the base model recognized that the answer depends on state law, yet hedged and referred the user to HR instead of stating the rule and the relevant figures. As a result, the answer met none of the three rubric criteria, and none of eight sampled answers to this question was fully correct.

Three panels. The question: a United States employee asks “How much sick time will I have by the end of April?”, which depends on federal and state law, the employer’s size and the hire date. The base model’s answer says it cannot tell exactly, that state-mandated paid sick leave laws may apply, and points to the pay stub and HR. Deel’s rubric expects the federal rule (no federal paid-sick-leave mandate), the accrual rate (New York: 1 hour per 30 hours worked) and the annual entitlement (New York, 100+ employees: 56 hours a year). Criteria met 0 of 3; fully correct 0 of 8 samples. — Figure 2 · How base Nemotron 3.5 Super answers a typical payroll question, scored against Emet by Deel’s rubric criteria.

How the model was trained

Step 1: Near-policy SFT

Since data close to the policy’s own distribution is easier to learn and causes less forgetting [3], AI21 kept training as on-policy as possible. However, the base model rarely produced rollouts strong enough to learn from. It fully answered only 10% of dev questions.

To raise the quality of its rollouts without leaving its distribution, or training “near-policy”, AI21 provided the model with hints during generation. The hints covered the model’s role, what the task expects of it, the level of detail a payroll answer needs, Deel’s terminology and answer structure, and the distinctions that users usually blur. While the hints were refined iteratively through error analysis, they added no new domain knowledge and only steered the model toward using what it already knew.

Next, AI21 sampled hint-guided rollouts from Nemotron 3.5 Super with reasoning turned on. They kept only the rollouts that met every one of Emet’s rubric assertions, including their reasoning traces. These formed the curated training set of only 412 examples, small by post-training standards. The model was trained with supervised fine-tuning (SFT) on this set, with hints withheld.

Near-policy SFT lifted performance on the dev set from 0.513 / 0.100 to 0.657 / 0.229 (criteria met / all criteria met).

Revisiting the sick-leave question shows the impact of this step. As seen in the figure below, the trained model states the federal rule, gives the figures for the states most likely to apply, and asks for the details it needs, raising its criteria met on this question from 0.13 to 0.79.

Three panels on the same sick-time question. Left, the question and the hints used to build the SFT data, which were not sent at inference. Middle, the base model knows the law is state-level but names no rule and no figure: 0 of 3 criteria met. Right, the post-SFT model states there is no federal paid sick leave law, gives New York’s figures and asks for the state and hire date. Criteria met 0.13 to 0.79; fully correct 0 of 8 to 4 of 8 samples. — Figure 3 · An example of how the same question from Figure 2 is answered by the Nemotron 3.5 Super base model, and then the Nemotron 3.5 Super model post-trained with SFT.

Step 2: SDPO with a privileged teacher

Once AI21 had established a stronger checkpoint with SFT, they turned to self-distillation policy optimization (SDPO) [1,2], a recent on-policy method. In this method, the model acts as both the student and its own teacher. The student answers a plain question, and the teacher (with access to privileged information) then scores the student’s tokens. The privileged information is the same set of hints from Step 1. This rewards desired traits and penalizes undesired ones, so the model doesn’t have to infer them from demonstrations alone.

AI21 trained on a dataset of 698 questions, a superset of the 412 used for SFT. That set is still small, but SDPO makes better use of the data. First, SDPO needs no gold answer. SFT can only learn from questions where the model produced a fully correct rollout. SDPO can also train on questions the model gets wrong, since the teacher scores the student’s tokens directly. Second, being on-policy, SDPO generates fresh rollouts every time it revisits a question. Each pass yields a new training signal, rather than a fixed target the model can memorize. When data is scarce, this is a large part of why on-policy training pays off.

As a result of training with SDPO, the model’s performance improved:

Using SDPO alone, dev-set results rose from 0.513 / 0.100 to 0.638 / 0.183, close to the SFT-trained scores (0.657 / 0.229) without any curated data.
Starting SDPO from the SFT checkpoint (SFT → SDPO) produced the best Nemotron 3.5 Super results: 0.720 / 0.339.

Post-training a model for a specific task can work with a small dataset. Like GRPO, SDPO relies on the student’s own capabilities to bootstrap performance and create a flywheel. The flip side is that on-policy training can only amplify what the student already produces: if the base model rarely produces good answers on its own, there is little signal to learn from. So it’s standard practice to first distill from a teacher, or otherwise improve the base model, before on-policy training begins. Here, off-policy distillation from a self-teacher, the model itself guided by hints, worked well as a precursor to SDPO. The combination of both methods improves over each one on its own.

Results

Line chart of score in percent at each training stage. Criteria met rises from 51.3 for the base model to 65.7 after near-policy SFT and 72.0 after SDPO, above Claude Opus 5.5’s 64.0. All criteria met rises from 10.0 to 22.9 and 33.9, above Claude Opus 5.5’s 26.3. — Figure 4 · Criteria met and all criteria met at each stage of post-training, with Claude Opus 5.5 for reference.

The sequential usage of SFT, followed by SDPO, produced the strongest model on both metrics, surpassing Claude Opus 5.5 by ~8 points on criteria met on the dev set.

Reaching frontier-level quality with SFT and SDPO

AI21’s Forward-Deployed Researchers trained Nemotron 3.5 Super with SFT, then SDPO. The model now scores 20.7 points above the base model on criteria met, and 8 points above Claude Opus 5.5.

Together, AI21 Labs and Emet by Deel demonstrated that, equipped with the right data and training recipe, companies don’t need to choose between a frontier model’s quality and an open model’s price. By post-training Nemotron 3.5 Super VL, Deel now has a payroll model that surpasses Claude Opus 5.5 on Deel’s payroll questions, while costing 50x less.

Payroll is the first domain. Emet is open to every model builder. If you want the same for your domain, talk to the Emet team.

Read the case study on AI21 Labs’ blog.

About AI21 Labs

AI21 Labs trains specialized AI models for companies through its lab-as-a-service approach. Its Forward-Deployed Researchers post-train models on a company’s own data and tasks, build the evaluation suite, and optimize the model for production, so the company owns the result. Learn more at ai21.com.

References

[1] Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, Aditya Grover. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. arXiv:2601.18734, 2026.
[2] Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, Andreas Krause. Reinforcement Learning via Self-Distillation. arXiv:2601.20802, 2026.
[3] Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, Qian Liu. Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning. ACL 2024, arXiv:2402.13669.

Post-training case study: Reaching frontier-level intelligence on global payroll with Emet by Deel