Current chain: Teutonic-XXIV. Verify this is still current by fetching /dashboard.json and inspecting king.repo.

Mining Teutonic-XXIV (Quasar 8B-active / 24B-total MoE)

The Teutonic SN3 king is now unconst/Teutonic-XXIV, a freshly-initialised SILX-AI Quasar hybrid MoE: ~8B active per token, ~24B total parameters, RoPE θ=1e6, vocab 262144 (same Hippius shards as before — no retokenization).

This guide tells you how to:

1. Set up your environment.
2. Build / train a challenger.
3. Submit it on-chain.

If you only want to play with random noise, jump to Quick start (noise miner). If you want to actually dethrone the king, see Real training.

0. The mechanism in one paragraph

The validator pulls every reveal commitment from chain, downloads the challenger from HF, runs a paired cross-entropy test against the king on a random Hippius shard, and crowns the challenger if its bootstrap LCB on the per-token NLL improvement clears delta = 1/N. Winner takes 100% of SN3 emission until dethroned. Full mechanism in DESIGN.md.

The architecture lock is enforced by validate_challenger_config in validator.py: your challenger's config.json must match the king on every key in CONFIG_MATCH_KEYS (vocab, dims, MoE shape, looped depth, latent-memory shape, RoPE, …) and must not ship any *.py files or set auto_map. Vendored modeling code only.

1. Environment

You need Python 3.12, CUDA 12.8, PyTorch 2.11 (cu128 wheel — earlier wheels do not have B200 / sm_100 kernels), transformers >= 5.5, and our flash-linear-attention fork that ships the Quasar/GLA layers.

python3.12 -m venv .venv
. .venv/bin/activate

pip install --index-url https://download.pytorch.org/whl/cu128 torch
pip install transformers accelerate safetensors huggingface_hub bittensor numpy

# The Quasar attention layers + GLA cache live in this fork pinned by SILX:
pip install "flash-linear-attention @ git+https://github.com/SILX-LABS/quasar-flash-linear-attention.git@84ad1cc5a7428609d7e0e56d4041a775cd19b7bb"

Clone the validator/miner code so you have the vendored Quasar package locally:

git clone https://github.com/unarbos/teutonic
cd teutonic

Sanity-check the model loads without trust_remote_code:

python -c "
import sys; sys.path.insert(0, '.')
import teutonic.quasar
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained('unconst/Teutonic-XXIV', torch_dtype='bfloat16', device_map={'': 'cuda:0'})
print('loaded', sum(p.numel() for p in m.parameters())/1e9, 'B params')
"

attn_implementation will fall back to eager for Quasar layers — that's expected (FA2 / SDPA upstream do not yet support QuasarForCausalLM).

2. Architecture you must match exactly

The king config is locked at:

If any of these drift, the validator rejects with "<key> mismatch".

If your repo contains *.py or your config has auto_map, the validator rejects with "repo ships *.py files" or "auto_map present in config.json". The vendored teutonic/quasar/ module is the only path the network accepts — your weights must load via plain AutoModelForCausalLM.from_pretrained(...) after import teutonic.quasar.

3. Repo naming and anti-impersonation

Your HF repo MUST match ^[^/]+/Teutonic-XXIV-.+$ AND embed the first 8 ss58 chars of your coldkey somewhere in the full repo id (case insensitive, in either the namespace or the model basename). Examples for coldkey 5DhAqMpdABCDEFG…:

• ✅ myaccount/Teutonic-XXIV-5DhAqMpd-v3
• ✅ 5DhAqMpd/Teutonic-XXIV-noise01
• ❌ myaccount/Teutonic-XXIV-v3 (no coldkey prefix)

This is the anti-impersonation gate added 2026-04-29.

4. Quick start — noise miner

For testing only; will almost never dethrone but verifies your end-to-end pipeline works.

. .venv/bin/activate
export HF_TOKEN=hf_...                 # write access to your HF org
export BT_WALLET_NAME=mywallet         # registered on SN3
export BT_WALLET_HOTKEY=default

python miner.py \
    --hf-account myaccount \
    --suffix 5DhAqMpd-noise-01 \
    --noise 1e-4

Under the hood miner.py:

1. Pulls the king at its pinned commit SHA.
2. Adds Gaussian noise of stdev --noise to every learnable tensor — but skips SMEBU global bias / momentum / max_vio buffers and the latent memory state (perturbing those collapses routing or destroys memory).
3. Runs the same validate_local_config checks the validator runs.
4. Uploads to myaccount/Teutonic-XXIV-5DhAqMpd-noise-01.
5. Submits the on-chain reveal commitment.

You can watch the validator pick it up at https://teutonic.ai/dashboard.json.

5. Real training

You need to lower the king's per-token NLL by more than delta nats on a random unseen Hippius shard, with a one-sided 99.9% bootstrap LCB > delta. Right now the king is uniform over 262144 tokens (ln(262144) ≈ 12.48), so the first real training run will dethrone.

A reasonable starting point uses scripts/mining/train_challenger.py which:

1. Reads the king repo + revision from the live dashboard.
2. Pulls the king and a few Hippius shards (already tokenized to vocab 262144).
3. Trains a LoRA adapter (default targets cover Quasar's q/k/v/o_proj, ffn.gate/up/down, w_down_proj/w_up_proj).
4. Merges LoRA into the base weights → standalone candidate.
5. Runs an offline paired-CE test against the king to estimate mu_hat before burning a HF push and chain reveal.

torchrun --nproc-per-node=8 scripts/mining/train_challenger.py \
    --upload-repo myaccount/Teutonic-XXIV-5DhAqMpd-v1 \
    --noise-only false \
    --max-iters 3

Notes specific to Quasar:

• nn.Parameter blocks like experts_w12 and experts_w3 are NOT Linear layers, so PEFT/LoRA cannot target them. Train them with full SGD if you want to move the routed experts.
• The SMEBU bias buffer (model.all_moe_bias) is updated by the model itself only when model.training=True. Don't manually overwrite it unless you understand what the routing-stability path is doing.
• Latent memory state is reinitialized every forward call when memory_states is None (default), so you don't have to manage it during training — but DO call model.eval() before paired evaluation so SMEBU doesn't shift bias mid-test.
• attn_implementation="eager" is currently the only supported path for QuasarForCausalLM under transformers ≤ 5.7. SDPA / FA2 will be wired up later.

After training, run the offline paired test the harness emits — if your estimated mu_hat is at least 2-3x the offline delta, push and submit. Otherwise re-run with more steps / different seed / different data weighting.

6. What the validator will tell you

Verdicts you might see in dashboard.json under history[*]:

• accepted: true, verdict: "challenger" — you are king.
• verdict: "king" — you didn't beat the king (LCB ≤ delta).
• verdict: "error" with error_code: "config_mismatch" — validate_challenger_config rejected your repo (read error_detail).
• verdict: "error" with error_code: "eval_error" and "could not load model with any attention implementation" — your safetensors didn't load on the eval server. Most common cause: you perturbed SMEBU buffers or shipped a config with auto_map.
• rejection_reason: "untrainable:seed0(...):loss_non_finite:nan" — the trainability probe took one SGD step on your model and got NaN. Means your weights are pathological (often: noise too large, or projections collapsed). Lower --noise or check the reparam-trick guard.

7. Useful links

• King model: https://huggingface.co/unconst/Teutonic-XXIV
• Live dashboard: https://teutonic.ai
• Live JSON: https://teutonic.ai/dashboard.json
• Source: https://github.com/unarbos/teutonic
• Discord: γ・τeuτonic・3 (ARbos answers technical questions there)
• SILX Quasar docs: https://huggingface.co/silx-ai/Quasar-3B-A1B-Preview

8. FAQ

Q: Can I just upload my own MoE / dense / Mamba checkpoint? A: No. The validator pins model_type=quasar and the full Quasar dim set. Cross-architecture submissions are rejected at config-match.

Q: Why isn't FlashAttention-2 used? A: QuasarForCausalLM doesn't yet have an FA2 path in upstream transformers. The eval server falls back to eager. This roughly doubles forward wall vs FA2 but is otherwise correct.

Q: Can I train and submit a quantized challenger? A: Eval loads in bf16. Quantized weights would dequant on load — usually fine for storage savings, but be careful about bias drift. safetensors only, no pickle.

Q: How do I reset my submission if I made a mistake? A: You can't dethrone yourself. Wait for the next reign and submit again. The validator de-dupes per-hotkey within a reign.