News / Models
A month into building Deriva-1, our early iterations surfaced real
problems in the Qwen3.8-27B base model: skewed reasoning
modes, heavy censorship, and a hallucination spike of our own making.
Here's where Deriva-0.4 stands and what's still ahead.
August 26, 2026
Over the past month we've worked through the first iterations of Deriva, and it came with real struggles. We originally planned to shape the model's personality with a mix of supervised fine-tuning (SFT) and our Explanatory Reinforcement Learning (ERL) method. We've since pulled back to SFT alone: a behavioral fine-tune gave us a more predictable way to shape personality, while ERL is still an unproven method at this scale. This is a sequencing decision, not a retreat from ERL; that work continues elsewhere, largely through Hikaru.
We evaluated a wide range of base models before settling on
Qwen3.8-27B. It's an unusually strong model for its
size; some third-party evaluations have it beating
Opus 4.6 MAX on select benchmarks, though we'd treat
that specific claim carefully, since the sources behind it carry
their own biases and we haven't reproduced it ourselves. Strong as
it is, the base model came with real issues, and we chose it
knowing we would need to fix them, not despite them.
Deriva-0.1, our first iteration, trained on a small corpus, and it showed us a problem immediately: hallucination rate rose more than 70% over the base model. The model would confidently assert things that were simply false, at one point claiming the Berlin Wall never fell and that we were still in the middle of the Second World War. What stood out was not just that it was wrong, but how it was wrong: no roleplay framing, no hedging, just a false claim stated as fact, from a model that appeared to already know better.
Deriva-0.2, a second small iteration, brought the hallucination rate down to about 40%, a real improvement over 0.1, and gave us a noticeably better personality. It was the first version that felt like genuine progress toward Deriva-1, though it was still not where we needed it to be.
Before covering where Deriva-0.3 and 0.4 landed, it's worth explaining the base model issues we were actually fixing.
Reasoning-mode task awareness.
Qwen3.8-27B ships with four reasoning modes:
Off, Low, Medium, and xHigh, its default, with support for adding
custom modes. The first issue is that in xHigh mode, the model
doesn't size its own thinking to the task. It can spend thousands
of tokens reasoning through something as trivial as 1+1, the kind
of task-awareness frontier models handle automatically.
Skewed mode quality. The second issue is that the modes themselves are skewed. In our internal testing, echoed by outside community testing, Off and Low produced nearly identical results. Medium gave lower-quality output, but quickly. xHigh, the default, gave output below Low's quality and took roughly five times longer while using more tokens to get there. Low mode turned out to be the best combination of quality, speed, and token budget, which is not what a mode named xHigh implies.
Censorship. The third issue is heavy censorship, common to models built under Chinese regulatory guardrails. Ask the base model something like what happened at Tiananmen Square, and it won't give a straight answer. It fabricates a response, deflects to an unrelated date, or avoids the question outright.
Deriva-0.3 addressed two of the three issues above: censorship and the reasoning-mode problems. With those fixed, hallucination also dropped back down to roughly the base model's own rate. That spike in 0.1 turned out to be mostly a data problem: our training corpus at that stage was synthetic, and synthetic data alone was introducing errors of its own.
Deriva-0.4, what we're building as of August 26, 2026, is aimed at the remaining gap and the data problem underneath it. The plan is to add stronger guardrails, generate synthetic data across multiple models instead of one, and bring in human-written data alongside real validation, rather than relying on synthetic data to check itself.
We'll share results once the Deriva-0.4 training runs are further along. For where Deriva-1 stands today, see the Deriva-1 model page and Deriva-1 in development.
