Skip to content

Commit 21a31fa

Browse files
author
Ronald Tse
committed
docs(wO47): metric root cause — docs 0.52% is corpus-level CER; dual-metric eval launched
1 parent df1b5e4 commit 21a31fa

1 file changed

Lines changed: 11 additions & 0 deletions

File tree

‎TODO.sota/47-fas-diac-release.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,3 +84,14 @@ Epochs/bs/backbone all matched. **run-079-fas-diac-docslr launched**
8484
with the full docs optimizer recipe (trainer now has U_LR/U_WARMUP/
8585
U_SCHED/U_WD knobs). Matrix so far, same split/corpus:
8686
small-3ep 0.65 · small-6ep 0.60 · base-3ep@3e-4 0.63 · docs@5e-4 0.52?
87+
88+
### run-079 (docs-exact optimizer): 0.64% MISS — the metric was the gap
89+
90+
base @5e-4/linear/w500 lands 0.64% row-mean — indistinguishable from
91+
base @3e-4/cosine (0.63). Optimizer eliminated. Final root cause, from
92+
modal_app_diacrit.py evaluate() itself: **run-001's "0.52% CER" is
93+
CORPUS-level (total_ed / total_gold_chars)**, while every rerun gate
94+
reported the **row-mean** (mean of per-row ed/len). Same predictions,
95+
different aggregation. run-079-metrics (l4) re-scores run-079/best
96+
under BOTH conventions with predictions saved; its corpus-level number
97+
is the docs-comparable verdict.

0 commit comments

Comments
 (0)