HIT-ASR is compared with the best single expert, a random choice, two transcript-fusion methods
that run every expert, and two pooled routers, on four English corpora, each with its own trio of experts. Every number
on this page is generated from the paper's tables.
Each corpus is split once: a random quarter is held out for the hyperparameter search, where every model gets the
same budget of ten random draws. On the remaining clips, every model is trained and tested on the same five
repetitions of two-fold cross-validation (5×2), and the tables give the mean and standard deviation over the ten test
folds.
The systems compared:
Best single expert: the expert with the lowest WER on the training half, used for every clip.
Random: an expert drawn uniformly for every clip.
ROVER (confidence-weighted) and CN-MBR: transcript-level fusion of all three experts' outputs, which needs
every expert to decode every clip.
MLP-pool: a neural router on the experts' encoder states averaged over time, trained with all three terms of
HIT-ASR's objective and the same search budget, three seeds averaged as well.
ADASTT: gradient-boosted trees on the same pooled states, trained with cross-entropy.
Oracle: an expert with the fewest errors on each clip, the upper bound for picking one expert.
HIT-ASR has a lower WER than every baseline on every corpus and closes 18.2 to 34.1 % of the oracle gap. Under a paired significance test on the same folds, its WER is significantly lower (p < 0.05) than that of the best single expert, of MLP-pool and of both transcript-fusion baselines on all four corpora.
How much of the gap between the best single expert (left end) and the per-clip oracle (right end) each router closes on the test folds; the ends give the two WERs in %.
System
AMI-SDM
Earnings-22
People's Speech
AfriSpeech-200
Best single expert
18.42 ± 0.22
12.79 ± 0.24
17.39 ± 0.26
25.82 ± 1.03
Random
21.42 ± 0.48
13.51 ± 0.32
18.32 ± 0.89
25.69 ± 0.94
ROVER (confidence-weighted)
20.03 ± 0.37
13.60 ± 0.36
18.15 ± 0.23
31.17 ± 1.27
CN-MBR
23.99 ± 0.44
14.06 ± 0.40
19.12 ± 1.17
31.71 ± 1.36
MLP-pool
17.44 ± 0.21
12.14 ± 0.36
16.29 ± 0.32
23.71 ± 0.79
ADASTT
18.23 ± 0.22
12.82 ± 0.33
16.78 ± 0.23
24.84 ± 1.36
HIT-ASR
17.19 ± 0.30
11.98 ± 0.27
15.80 ± 0.19
22.93 ± 0.59
Oracle
11.68 ± 0.13
9.00 ± 0.18
12.74 ± 0.13
17.51 ± 0.38
Oracle (1-word tolerance)
13.67 ± 0.15
9.90 ± 0.18
13.54 ± 0.13
18.54 ± 0.36
Oracle (2-word tolerance)
15.15 ± 0.15
10.78 ± 0.20
14.31 ± 0.13
20.21 ± 0.38
Test WER in %, mean ± standard deviation over the ten folds; the lowest of each corpus in bold. The tolerant oracles keep the best single expert unless another expert saves more than one or two words.
The share of the oracle gap each system closes, and how often it picks a best expert:
AMI-SDM
Earnings-22
People's Speech
AfriSpeech-200
System
GC
Acc
GC
Acc
GC
Acc
GC
Acc
Best single expert
0.0
73.6
0.0
68.0
0.0
67.3
0.0
54.3
Random
−44.5
68.8
−19.0
64.2
−19.7
59.4
1.1
53.2
ROVER (confidence-weighted)
−23.8
—
−21.3
—
−16.5
—
−65.1
—
CN-MBR
−82.7
—
−33.3
—
−36.7
—
−71.6
—
MLP-pool
14.5
78.4
17.4
73.1
23.6
70.5
24.8
62.9
ADASTT
2.9
74.9
−0.6
68.9
13.1
69.2
11.8
61.6
HIT-ASR
18.2
79.7
21.4
73.2
34.0
71.6
34.1
63.4
Oracle
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
Oracle (1-word tolerance)
70.4
84.8
76.2
83.8
82.8
83.9
87.5
83.1
Oracle (2-word tolerance)
48.5
79.2
52.9
75.8
66.0
76.1
67.0
69.3
GC: the share of the gap between the best single expert and the per-clip oracle that a system closes. Acc: the share of clips on which it picks a best expert. Both in %; the fusion methods write a new transcript and have no Acc.
A controlled check of the central design choice. Each synthetic clip switches from one acoustic regime to another at
a random point, and which expert is best depends on both regimes and their order, so averaging the frames over time
removes the information the choice needs. HIT-ASR runs here with its full design: shared Stage 1, cross-attention
bridge and a Stage 2 transformer, trained on expected WER plus soft cross-entropy.
At every pool size HIT-ASR picks a best expert for every test clip (selection accuracy 100.0 %), so its WER equals the oracle's. MLP-pool, which sees the same frames averaged over time, closes 29.0 to 37.9 % of the gap.
Synthetic regime switch: the share of test clips on which each method picks a best expert, as the pool grows. HIT-ASR's line is also the oracle's.
K
Method
WER
Acc
GC
3
Random
24.92 ± 0.20
33.3 ± 0.0
−15.1
3
Best single expert
22.24 ± 0.41
42.3 ± 1.3
0.0
3
MLP-pool
17.10 ± 0.79
58.1 ± 1.3
29.0
3
HIT-ASR
4.49 ± 0.27
100.0 ± 0.0
100.0
3
Oracle
4.49 ± 0.27
100.0 ± 0.0
100.0
5
Random
29.00 ± 0.12
20.0 ± 0.0
−19.7
5
Best single expert
25.06 ± 1.15
34.0 ± 1.4
0.0
5
MLP-pool
19.14 ± 0.42
51.3 ± 0.7
29.2
5
HIT-ASR
4.86 ± 0.43
100.0 ± 0.0
100.0
5
Oracle
4.86 ± 0.43
100.0 ± 0.0
100.0
10
Random
32.04 ± 0.10
10.0 ± 0.0
−11.9
10
Best single expert
29.10 ± 0.25
18.4 ± 0.6
0.0
10
MLP-pool
19.76 ± 0.48
50.4 ± 1.3
37.9
10
HIT-ASR
4.49 ± 0.33
100.0 ± 0.0
100.0
10
Oracle
4.49 ± 0.33
100.0 ± 0.0
100.0
WER and selection accuracy (%) as mean ± sd over the seeds; GC in %.
Which expert each router picks, and how often the pick is a best one.
HIT-ASR has the highest selection accuracy of the three routers on every corpus. ADASTT sends 87.0 to 96.1 % of the clips to one expert; HIT-ASR uses all three experts of each trio.
Where each router sends the test clips, per expert of each corpus's trio; the ring is where the per-clip oracle sends them.
Corpus
Router
E1
E2
E3
Acc
AMI-SDM
MLP-pool
74.8
21.0
4.3
78.4 ± 0.5
ADASTT
96.1
3.8
0.1
74.9 ± 0.4
HIT-ASR
39.7
54.1
6.2
79.7 ± 0.6
Oracle
73.6
20.8
5.6
100.0 ± 0.0
Earnings-22
MLP-pool
66.7
18.2
15.1
73.1 ± 0.6
ADASTT
89.8
2.6
7.6
68.9 ± 0.8
HIT-ASR
73.5
12.0
14.6
73.2 ± 0.7
Oracle
64.1
20.9
15.0
100.0 ± 0.0
People's Speech
MLP-pool
69.8
15.1
15.1
70.5 ± 0.7
ADASTT
91.7
6.6
1.7
69.2 ± 0.7
HIT-ASR
45.5
22.1
32.4
71.6 ± 0.4
Oracle
67.3
18.3
14.3
100.0 ± 0.0
AfriSpeech-200
MLP-pool
63.1
24.4
12.6
62.9 ± 0.6
ADASTT
87.0
10.0
3.0
61.6 ± 0.5
HIT-ASR
52.8
27.6
19.6
63.4 ± 0.9
Oracle
60.6
24.9
14.4
100.0 ± 0.0
Share of the test clips sent to each expert (%) and selection accuracy (%, mean ± sd over the folds). E1–E3: AMI-SDM: Cohere-Transcribe / Granite-Speech-4.1-2b / Parakeet-TDT-0.6b-v3; Earnings-22: Cohere-Transcribe / Kyutai-STT-2.6b / Parakeet-TDT-0.6b-v3; People's Speech: Granite-Speech-4.1-2b / Parakeet-CTC-1.1b / Parakeet-TDT-0.6b-v3; AfriSpeech-200: Cohere-Transcribe / Qwen3-ASR-1.7B / Voxtral-Mini-3B.
An exploration on the hold-out data drew HIT-ASR configurations at random, with each design choice as a switch.
Grouping its configurations by one switch at a time shows what that switch does across all the others.
The training objective matters most. Trained on the expected-WER term alone, 86 % of the configurations stay on the best single expert; with soft cross-entropy targets alone, 7 % do, and with WER plus hard cross-entropy, 8 %.
Share of the explored HIT-ASR configurations that closed at most one percent of the gap, that is, stayed on the best single expert, by training objective (hold-out exploration on three corpora).
Design switch
Setting
Collapsed (%)
GC median (%)
GC best (%)
Selected
Stage 2 fusion
cross-attention bridge
48
6.6
33.2
0/3
concatenation
28
18.6
31.3
1/3
mean
22
20.5
43.2
2/3
transformer
33
21.9
39.6
0/3
Stage 1 shared across experts
no
30
23.1
42.3
2/3
yes
35
16.7
41.6
1/3
Training objective
WER + hard CE + soft CE
24
22.6
31.2
0/3
WER + soft CE, τ = 0.1
39
18.4
37.0
0/3
hard CE only
30
14.6
26.4
0/3
soft CE only
7
26.2
38.5
2/3
WER + hard CE
8
27.1
35.6
1/3
WER + soft CE, τ = 1
17
15.8
35.1
0/3
WER only
86
−7.0
18.0
0/3
Input normalisation
none
36
10.7
39.8
2/3
scale
35
20.4
39.5
0/3
standardise
25
23.0
40.9
1/3
Every HIT-ASR configuration of the hold-out exploration on AMI-SDM, Earnings-22 and People's Speech, grouped by one design switch. Collapsed: the share that closed at most one percent of the gap, that is, stayed on the best single expert. GC median and best: the share of the gap closed by the typical and the best configuration. Selected: on how many of the three corpora the hold-out search picked that setting for the main results.
Measured on raw audio, one clip at a time on one GPU, with a router of the main experiment deployed in front of the
real experts.
Against decoding every expert, HIT-ASR saves 40 to 90 % of the time per clip. Against the best single expert alone its time ranges from 0.11 to 2.52 times that expert's, depending on the corpus.
Milliseconds per clip on raw audio, one clip at a time; each corpus is drawn on its own scale. HIT-ASR runs every encoder, the router and one decoder.
Corpus
Encoders
Router
Decoder
HIT-ASR
Best single
Decode all
× best single
Saved (%)
AMI-SDM
51.3
7.7
47.4
106.4
42.2
177.8
2.52
40
Earnings-22
49.8
6.2
161.0
217.0
1965.3
2106.3
0.11
90
People's Speech
51.1
5.6
99.8
156.5
215.1
298.9
0.73
48
AfriSpeech-200
65.5
8.2
59.3
133.0
271.5
658.8
0.49
80
Milliseconds per clip on raw audio, one clip at a time on one GPU. HIT-ASR runs every encoder, the router and the chosen expert's decoder; decode all runs every expert in full, then fuses the transcripts. × best single: HIT-ASR's time over the best single expert's. Saved: the share of decode all's time that HIT-ASR saves.
The People's Speech trio grown to five and to ten experts, adding at each step the expert that lowers the oracle's
WER most, with HIT-ASR and MLP-pool refitted on the hold-out at each size:
K
Router
GC
WER
Acc
3
HIT-ASR
58.1 ± 2.6
15.92
71.2
3
MLP-pool
52.1 ± 2.1
16.34
72.6
5
HIT-ASR
0.2 ± 4.2
16.01
60.3
5
MLP-pool
−6.5 ± 8.0
16.32
61.3
10
HIT-ASR
9.8 ± 4.5
15.59
54.8
10
MLP-pool
3.0 ± 4.5
15.96
54.6
The People's Speech trio grown to five and ten experts, on the hold-out; GC (%) is the mean ± sd over the model seeds, WER and Acc in %.