Cactus Hybrid
A small, on-device model is fast and private, but sometimes wrong.
At Cactus we post-train models to know when they are wrong: we ship probes
inside the checkpoint that score every answer with a confidence between
0 and 1, returned as structured data (never parsed out of the answer text).
Answer on-device when confidence is high; you can re-route to a bigger
model when it's low:
We start the rollout with Gemma 4 E2B Hybrid, all builds live in the
Cactus Hybrid collection
on Hugging Face.
Gemma 4 E2B hybrid, the smallest Gemma model, matches Gemini 3.1 Flash-Lite
on most benchmarks by routing only 15–35% of queries to the Gemini 3.1 Flash-Lite and
running the remnant itself.
N/B: Quantisation quality is measured on Cactus Quants
which performs well at uniform quantization.
Developers are encouraged to benchmark for Unsloth, GGUF, and MLX quantization independently.
Cactus
MLX
Transformers
Load the model with an explicit .to(device), not device_map="auto": the
probe scores generations outside the module forward() path, so weights that
accelerate offloads (left on the meta device) crash the confidence read.
llama.cpp
llama.cpp is C++, so the probe is a patch you compile into the engine (see
patches/llama.cpp/). Build the patched server once:
Then serve and query it like any llama-server — the response carries a
top-level confidence field:
Routing Quality (AUROC)
Gemma 4 E2B Hybrid AUROC measures how well the the separates wrong answers from right ones
(higher = better, 0.5 is random, 1.0 is perfect):
The strongest result: the probe was trained on zero audio data, yet achieves
0.79–0.88 AUROC on four audio benchmarks (two transcription, one audio MCQ, one
out-of-domain transcription).
This rules out surface-level explanations, the probe
is reading a modality-independent correctness signal from the hidden state, not
memorizing patterns from training data.
MIT-licensed. Gemma model use is subject to the Gemma terms.