State tracking (A5 / S5)

FPRM (FPTRM-singlez) trained on permutation-group state tracking over the alternating group A5 and the symmetric group S5. Trained at sequence length 32, evaluated for length generalisation up to k = 128. Three seeds per group.

Results at k = 128 (mean ± sample std over 3 seeds)

group	accuracy	best	worst	mean L-level calls / sample
A5	98.1 ± 2.2	99.9	95.5	60 ± 9
S5	98.8 ± 0.9	99.7	97.9	70 ± 7

Eval uses fixed-point halting with max_iter_eval=160 (per-sample compute ceiling), stepsize_decay=0.95, and decay_patience=5 (A5) / 10 (S5). For comparison, a TRM baseline under the same 160-call budget reaches 45.8 ± 3.9 (A5) and 39.4 ± 1.9 (S5).

Layout

state_tracking/
├── {a5,s5}_seed{1,2,3}/
│   ├── step_97656           # final model weights (load these)
│   ├── all_config.yaml      # exact training config
│   ├── fp_trm_singlez.py    # model source at training time
│   └── losses.py
├── eval_results/            # per-k CSVs (k = 1..128), one per run
└── reproduce/               # per-k eval driver, sbatch scripts, eval-only patches

Unlike the other folders, these runs were trained with ema: false, so step_97656 holds the raw (non-EMA) weights, and no *_train_state.pt is included.

eval_results/*.csv columns: family, group, seed, k, acc_halt, mean_halt_steps, never_halted_frac, l_level.

Reproducing the eval

From the repo root (branch dev/vera/fptrm_new or a descendant):

git apply state_tracking/reproduce/*.patch
cp state_tracking/reproduce/eval_fp_per_k.py scripts/

The patches are eval-only and default to no-ops. per_k_node.sbatch and submit_per_k.sh show the exact per-k eval invocation used for the CSVs.