Question 1

How does Ralph measure whether a compressed model actually preserves capability?

Accepted Answer

Ralph scores by measuring downstream observer effect: from a trajectory prefix, both the pinned parent and the compressed model produce a step, then a judge model continues from the parent's step to establish a fixed continuation. The judge's distribution over that continuation is measured after seeing the parent step, the compressed model's step, and the prefix alone. The score rewards whether the compressed step moved the judge in the same direction and magnitude as the parent step moved it—this captures capability retention without rewarding style imitation. Every item is scored by all three configured judge models (SmolLM2-1.7B-Instruct, Phi-3-mini-4k-instruct, OLMo-2-1124-7B-Instruct), and the displayed metric is the mean of their worst-slice scores.

Question 2

What prevents miners from copying the reigning crown and winning again?

Accepted Answer

Exact copies have zero paired margin and cannot earn challenger share; previously scored bytes are skipped entirely from re-evaluation. The crown transitions on displayed-metric lead (0.02 in one round, or 0.01 in two consecutive rounds), so even if a copy were re-scored, it would tie its original and earn nothing. Coupled with the requirement that the earlier commitment wins when two hotkeys commit the same hash, copying is economically irrational: it provides no path to allocation.

Question 3

Can I verify a round's score myself without running the full scoring infrastructure?

Accepted Answer

Yes, at four levels of cost. L0 recomputes the arithmetic (score, margins, weights) from published measurements with no GPU required. L1 derives which items were scored from the post-commit round nonce and verifies the exam was neither pruned nor padded. L2 recomputes a recorded judge's distributions over the frozen text in the record. L3 loads the actual checkpoint and regenerates the model steps to bind the record to the artifacts. The `eval.rerun` tool supports all four; skipping expensive levels reports the round as INCOMPLETE, never as verified.

Question 4

What bit budgets do the four tiers enforce, and how is the budget measured?

Accepted Answer

The tiers are binary (1.15 bits/weight max), ternary (1.75), sub2 (2.3), and sub4 (4.0). For GGUF, the code bits come from the per-tensor type name (Q1_0 = 1 bit, Q2_0 = 2 bits, IQ2_XXS = 2 bits) mixed by parameter count, and the container bits account for overhead. For safetensors, distinct quantization levels per group are counted. Both budgets bind—a 4-bit model inside a 16-bit container is rejected because it exceeds the container cap. Formats like TQ1_0 and TQ2_0 are rejected because they lack required mainline Metal and CUDA kernels, not because of the bit count.

Question 5

How does the commit-reveal mechanism prevent miners from gaming the exam?

Accepted Answer

Miners commit a hash of their exact model bytes (sealing it on chain) before the round's nonce is drawn. The nonce, derived from a future block hash after commitments close, selects which trajectory items are scored. Because the nonce is drawn after the commit window closes, miners cannot know which items will be evaluated and thus cannot tune their compression specifically for them. Miners reveal their artifact URI only after the round opens, and validators verify that the fetched bytes hash to the committed value. Mismatch between committed and served bytes is recorded as a failure.

Question 6

What does it mean if a crown 'held on the floor rule'?

Accepted Answer

The floor rule requires that no slice of a challenger's score fall below the incumbent's worst slice under the same judge. If a challenger has a higher overall metric but at least one judge-language-depth combination scores lower than the incumbent's minimum on that same judge, the challenger cannot dethrone. This prevents reshaping scores—a strong slice cannot buy a weak one.

Ralph

Ralph

Hardware Requirements

Hardware Requirements

Miner Setup

Validator Setup

Participate

Participate

Hardware Requirements

Hardware Requirements

Miner Setup

Validator Setup