Research
Explanatory Reinforcement Learning.
Our central research direction. We believe a model learns better when it is told why it was corrected, not merely that it was. The method is in active development; the first model trained with it is Hikaru-1.
The problem with a binary reward
Reinforcement learning from human feedback, as it is most often practiced, reduces a trainer's judgement to a single bit. A response is good, or it is not. The model receives the verdict, but none of the reasoning that produced it.
This is a lossy signal. A response can be wrong in small ways or large ones; it can fail on tone while succeeding on fact, or the reverse. Collapsing all of that into one dimension throws away exactly the information a model needs to improve.
The method
Three changes to the reward signal
1. A graded scale
Instead of +1 or −1, the grader assigns a value on an
eleven-point scale: −5, −4, … 0 … +4, +5. Zero marks
a response that was neutral, neither good nor bad. The magnitude
tells the model how far off it was, and in which direction.
2. Per-section scores
A single score still hides too much. We grade dimensions separately, personality, helpfulness, and others, each on the same scale. A model can be told that it was helpful but cold, or accurate but evasive, and adjust each independently.
3. Grader explanations
The core of the method. A strong grader model writes an explanation for every score: where the response succeeded, where it failed, and why. The model under training learns the reasoning behind the correction, not just the correction itself.
Why we think this matters
A model that only learns "this was bad" learns to avoid a narrow pattern. A model that learns "this was bad because it hedged on a question where a direct answer was needed, and the user lost trust" learns a transferable principle. The explanation carries the generalization that a bare scalar cannot.
It is also a more honest record of training. When the grader's reasoning is preserved alongside the score, the training process becomes auditable rather than opaque. We can ask not only what a model was rewarded for, but why.
Where it stands
The method is still in development. Our first concrete test is
Hikaru-1, a finetune of IBM Granite 4.1 (8B)
trained with ERL, which entered development on June 16, 2026. Its
purpose is to measure whether a model trained this way behaves
measurably differently from one trained on a binary reward.
- Active: the ERL pipeline and the Hikaru-1 training run
- Planned: evaluations comparing ERL against baseline RLHF
- Future: applying the method to larger base models
Full papers will be published here when ready. We would rather share something correct than something first.