Albedo

Training small, efficient language models that retain the reasoning, coding, and knowledge capabilities of larger teachers is expensive and hard to validate. Traditional benchmarks saturate quickly and miners overfit to static test sets, leaving held-out performance flat. Knowledge distillation can bridge the gap, but evaluating whether a distilled student actually generalizes requires a principled multi-axis scoring system that resists gaming.

Bittensor Subnet 97 uses an incentive-aligned network of validators to score distilled models on a composite of 25+ axes — including 11 procedurally-generated axes (math, code, reasoning, long-context, knowledge, calibration, consistency) that regenerate every round from the block-seed, so there is no static answer key to memorize. Miners commit distilled models to HuggingFace; validators pull them, verify architecture compliance (≤33B params, Kimi-family, same tokenizer as teacher), and score them on the composite. The ranking key blends the worst-3-axis mean with the weighted mean, so a model cannot camp specialists. Winner takes all: the highest-scoring model gets 100% of emissions that round.

The validator runs a split architecture: a secure host holds wallet keys and sets weights on-chain; a remote GPU pod (8× H200 tensor-parallel) runs inference — teacher generation via vLLM, student forward passes, KL computation. Miners train using standard distillation recipes (on-policy KL + SFT on quality traces + optional GRPO) and push to HuggingFace. Single-eval policy + paired king re-evaluation + 5% dethrone margin mean lucky rounds do not crown imposters, and a model's score reflects the same procedural items as its challengers.

Subnet 97 (Distil) is a Bittensor subnet for competitive distillation of moonshotai/Kimi-K2.6 (1T total / ~32B active MoE) into smaller student models (≤33B total params, Kimi-family architecture). Miners train and commit distilled models to HuggingFace; validators score them on a 25+ axis composite (11 procedurally-generated v31 axes covering math/code/reasoning/long-context/knowledge/honesty, plus distillation/judge/discipline tiers), with the king receiving 100% of emissions. The composite.final ranking key blends the worst-3-axis mean (75%) with the weighted-mean of all axes (25%), anti-Goodhart via procedural item generation seeded by block hash.