A hackathon research record · in-silico (T0) evidence only · no candidate promoted

Before the implant, there is the bench.

A small, strictly causal neural model that tries to pull one voice out of a crowd for cochlear-implant users. It is judged on a synthetic bench, in pre-registered generations, by rules written down before each run. It passes 13 of 14 rows. This site is about the one it doesn't.

The short version Ask the data

Graham's story · in his dad's words

The day Graham was born, there was a jackhammer outside.

"Someone was using a jackhammer right outside our hospital room. It was so loud I could feel the reverberations shaking my feet underneath his bed.

He was asleep the whole time."

Weeks of tests

All we knew for a long time was that something was wrong.

"During his hearing test, I saw that he was getting almost no sound. They ran more blood tests, urine tests, MRIs and echocardiograms over the course of weeks. We had multiple doctor's appointments every week. We haven't gotten any definitive answers, other than that he has bilateral sensorineural hearing loss."

What the doctors said

"The doctors told us that, one day, he may grow to enjoy music."

Graham, laughing, hugs his little sister Maggie on a picnic blanket outdoors. — Graham and his sister Maggie.

Part 1 · The crowd

Music and conversation in a crowd are still the hardest thing.

Graham was born with bilateral, sensorineural, severe-to-profound hearing loss. He wears bilateral Cochlear Nucleus 6 processors; that is what he uses, and nothing on this site has ever been used by him. His dad, who built this project, has normal hearing.

Crowds are where cochlear-implant users struggle most. A dance class on a lawn, a birthday table, a playground: many voices at once, and the one you need is no louder than the rest. That is where the −5 dB wall lives, and it is the only thing this research is about.

A cochlear implant delivers sound as electrical pulses on a short row of electrodes. In quiet, that works remarkably well. In a room full of other voices, the thing a listener needs most, one voice standing apart from many, is exactly what the signal makes hardest to recover.

Graham's dad with Maggie and Graham on a tree-lined path, all three laughing. — Built by Graham's dad.

Why this exists

This project exists because of Graham.

Captions on. Music is playing in this clip.

"Turns out, he does love music. And he's pretty great."

Graham, shaking his booty with the gray hairs at an outdoor workout class.

Part 2 · The model

A model that can never listen ahead.

A hearing device cannot wait for the end of a word to decide what the start of it was. So the model is strictly causal: every output sample depends only on audio that has already arrived.

Gen 19, the current baseline, is a causal Conv-TasNet-style enhancer. Exact causality is re-verified on the final weights after every training run: feed it two signals that match for the first P samples and the outputs match for exactly those P samples, to the bit.

—

parameters

—

ms algorithmic latency (excluding compute)

samples of look-ahead

—

max prefix difference in causality test

How 5.88 ms compares

The KPI contract allows up to 10 ms of algorithmic latency. At 16 kHz, 5.88 ms is 94 samples.

5.88 / 10 ms

What the implant does next

One of the three intelligibility proxies simulates the implant's own strategy: of 22 electrode channels, the 8 with the most energy are selected each frame. The bench scores how well the enhanced signal's electrode pattern matches the clean target's.

22 channels · 8 selected per frame

Part 3 · The wall

On crowd noise overall, Gen 19 improves SI-SDR by +3.51 dB (three-seed mean) against a +3.0 requirement. At 0 dB SNR, where the crowd is as loud as the voice, it gains +4.65 dB. At −5 dB, where the crowd is louder than the voice, it gains +1.19 dB. The rule says +1.5.

Every attempt since has been aimed at that number. None has moved it. Drag the slider to walk the bench.

Signal-to-noise ratio of the crowd mixture

SNR+10 dB

Gen 19 ΔSI-SDR—

classical causal gate—

non-causal oracle—

verdict—

Table: crowd ΔSI-SDR by SNR (three-seed mean, dB vs unprocessed)

The short version

Thirteen rows pass. One does not. Nothing is promoted.

The question

Can a tiny, strictly causal neural model reduce crowd noise for cochlear-implant users enough to earn a listening test, judged on a synthetic bench under rules fixed in advance?

What we found

Gen 19 passes 13 of 14 implemented scorecard rows, including +3.51 dB crowd ΔSI-SDR. It fails crowd noise at −5 dB: +1.19 dB against a +1.5 requirement. Four further generations tried to fix that row. Three are filed as nulls; one is running.

What it does not mean

Nothing here was tested against the Nucleus 6's own processing, on any implant, or with any listener. Vocoder audio is not what a CI user hears. This is in-silico evidence, verdict FAIL — keep optimizing, promoted FALSE.

13/14

scorecard rows pass (Gen 19, 3 seeds)

+3.51

dB crowd ΔSI-SDR vs unprocessed

+1.19

dB at −5 dB; +1.5 required

—

generations filed as nulls since Gen 19

candidates promoted

The scorecard

Fourteen rows, written before the result.

The scorecard is computed by kpi_scorecard.py from the committed bench JSONs of three seeds plus a hash-matched spatial run. It reports INCOMPLETE unless it has all of them. Gen 19 is the first generation with a complete verdict. The failing row is highlighted and marked FAIL in text.

Gen 19 KPI scorecard, three-seed mean
Row	Actual	Required	Status	Explain

The generations ·

An evolutionary loop where the dead branches are kept.

Each generation changes one thing, is pre-registered, trains on fixed budgets, and is judged on its final checkpoint only, on three seeds, by a script. An independent AI reviewer (Codex Astra) audits each result before the next generation trains. Branches that fail stay on the tree.

Trying to break the wall

The −5 dB row has not moved.

Gen 21: more optimization is not the answer

Two policies, 48k steps from scratch (S48) and the Gen 19 parent plus a 24k restart (R48), on three seeds. Qualification required a +0.30 dB seed-mean gain at −5 dB on both the validation panel and the bench. Overall crowd gains rose to +3.79 and +3.67 dB. The −5 dB row did not qualify on either panel for either arm; bench gains were negative.

Table: Gen 21 −5 dB gains over same-seed Gen 19 parent (dB)

Gen 22: a capped loss tracks the baseline exactly

A 30 dB soft cap on the SI-SDR loss was meant to stop quiet examples from dominating gradients. On the validation panel the −5 dB trajectory shadows Gen 19 at every 2k-step checkpoint. Final gains: +0.149 dB on VAL, −0.010 dB on the bench, against +0.30 on both. The cap is not retained.

Table: VAL −5 dB ΔSI-SDR by training step, seed 17 (dB)

When Gen 23 completes, its trajectory can be added to data/val_traj.json and will be drawn here.

Ask the data · powered by Muse, Meta's AI model

Ask the research record a question in your own words.

Muse works as a research assistant over the full record: pre-registrations, results, scorecard rows, reviewer notes. It decides what to look up, runs those lookups, and you can see each step it took. It is bound by the same rules as this page: in-silico evidence only, no claims about any listener.

The Ask and Explain features need the site's Muse endpoint, which is not reachable right now. Everything else on this page is static and does not depend on it.

What this could mean for Graham · if it passes the next gates

Could, not does.

Nothing on this page has been tested with Graham's implants, or with any listener. His Nucleus 6 is what he uses, and its own processing has never been compared. What follows is what the bench says the model does in simulation, and what it could look like if every gate below were passed.

1 · Hearing better in noise

A tiny program would sit between the microphone and the implant processor. Every 5.88 ms of sound, it guesses which part is the voice and turns the rest down, using only sound that has already happened. It has 602,510 parameters, small enough to imagine on a phone or a processor; its streaming speed on a CPU is not yet measured.

What the bench shows (simulated, Gen 19, 3 seeds). In crowd noise it improves a speech-clarity score by +3.51 dB overall and by +4.65 dB when the crowd is as loud as the voice (0 dB). In simulated electrode patterns of an 8-of-22 ACE-style processor, correlation with clean speech improved by +0.072. Quiet speech is left alone (quiet SI-SDR 36 dB). In the hardest case, crowd louder than the voice at −5 dB, it helps only +1.19 dB, short of the +1.5 target. That is the wall.

crowd ΔSI-SDR at −5 dB+1.19 dB crowd ΔACE electrode correlation+0.072

What it could look like. A birthday party, a cafeteria, a family dinner: the chatter turned down a little, the voice he is facing coming through clearer, quiet speech untouched. Possible paths, stated as possibilities only: processing in a phone or clip-on accessory that streams to his processors, or inside a future processor. Each would need the manufacturer, audiologists and regulators. No timelines.

2 · Knowing where a voice comes from

With two implants, each side processes sound mostly on its own. Brains find direction partly from small differences between the ears: which ear is louder (ILD) and which hears it first (ITD). CI users rely mostly on the loudness difference. If each side's noise reduction turns sound down by different amounts, it can blur that clue, so a voice on his left might seem to come from the middle.

What the bench shows (seed 17, four simulated scenes, assumed head model).

target-ILD error (lower is better)5.47 vs 5.96 dB unprocessed ITD error (lower is better)4.0 vs 5.75 samples ILD error in the −5 dB scene10.29 → 11.22 dB, worse

Nothing here shows a directional-hearing benefit. The registered next step is a binaural linked-gain model: both sides share one decision about how much to turn down, so the left-right loudness clue survives noise reduction.

What linked ears could look like

An illustration of the idea, not measured data. His teacher calls from his left in a noisy classroom.

Before any of this could reach Graham

Pass all 14 bench KPIs on 3 seeds
Gen 19 passes 13. The −5 dB crowd row is the one left, and Gens 20–22 did not move it.
13 of 14 · verdict FAIL — keep optimizing · promoted FALSE
Fresh-material confirmation
12 new speakers and genuinely distinct-voice crowds the model has never seen. Not yet built.
A staged listening test
Only with audiologist oversight, Graham's assent and MAP-safe levels. Not attempted.
Comparison against what he already has
His Nucleus 6's own processing has never been compared. Until it is, nothing here can be called better than what he uses today.

How we keep ourselves honest

The rules were written before the runs.

Pre-registration

Each generation files one change, its budget, its panels and its pass rules before training. Gen 24 is drafted but unregistered until the Gen 23 verdict and its review are both filed.

Three seeds, seed-mean

KPIs are judged on the mean of three seeds with distinct checkpoint hashes. A single-seed screen can only advance a generation to replication, never qualify it.

Final checkpoint only

Interim checkpoints are benched and reported, but the verdict uses the final one. No picking the best step after the fact.

Bit-identical replay

When a Gen 21 run was killed at 12.5k steps by a session ending, it was relaunched with the same seed and its losses and validation items matched the original bit for bit before the result counted.

Independent review, quoted

Codex Astra, a different model family, recomputes rows from raw data between generations. Reviews are filed verbatim with a disposition, including the ones that required fixes before launch.

"Nothing here implies a directional-hearing benefit."SUBMISSION.md, disclosures, after the Astra review of Gen 19

Nulls are filed

Gen 16, 20, 21 and 22 are on the tree with their numbers. "Null for this policy" is the registered label; none of them establishes a zero effect, and the record says so.

Limits and disclosures

Stated plainly.

The Cochlear Nucleus 6's own processing was not tested. Offline bench results cannot establish superiority over it, and nothing here compares against it.
No listener benefit is claimed for anyone. Vocoder listening by a normal-hearing adult cannot predict an individual CI user's benefit; vocoder audio is not what a CI user hears.
No candidate has earned a listening test. That gate requires all required KPIs plus confirmation on fresh material (new speakers, genuinely distinct-voice crowds). Implant-involved testing would additionally need assent, audiologist oversight and MAP-safe levels. None was attempted.
Quiet PESQ and CPU streaming real-time factor are required by the KPI contract but remain unestablished.
White-noise vocoded ESTOI decreases in all three Gen 19 seeds (mean −0.059), even though the registered white-noise SI-SDR row passes.
Direction-cue rows are one seed on four simulated scenes. In the −5 dB scene, target-ILD error worsens (10.29 → 11.22 dB). Nothing here implies a directional-hearing benefit.
Earlier generations were less disciplined. Gen 17 changed the speaker pool, mixtures, SNR weighting and budget together.

How this was made

A changelog, mistakes included.

The loop
An evolutionary research loop, run by Claude, reviewed by Astra
The experiments were designed, trained and filed by Claude (Anthropic) working under the author's direction, with Codex Astra (a different model family) as the independent reviewer between generations. Meta's models did not do the research.
Mistake
Gen 17 changed four things at once
Speaker pool, mixtures, SNR weighting and budget moved together, so its +0.26 dB gain cannot be attributed. Later generations change one thing each.
Mistake
A 10-step smoke run once overwrote Gen 16's final weights filename
Now guarded: smoke runs must write to a distinct output path. Interim checkpoints are all benched so nothing can be cherry-picked.
Correction
Gen 21's seed-19 run was killed mid-training and replayed
A session ended at 12.5k steps. The run was relaunched with the same seed and verified bit-identical (losses and validation items) before it counted.
Correction
A per-speaker caveat was corrected after review
Astra found that Gen 21's S48 validation gain was 87% from one of two target speakers. The results file was corrected and the caveat kept.
Determinism
Gen 23's spectral loss runs on CPU on purpose
MPS STFT gradients were not bit-deterministic across identical calls, so the spectral term is computed on CPU and its gradients cross back to the model on MPS.
This site
Numbers come from one script
site/tools/extract.py runs the project's own kpi_scorecard.py and reads the committed bench, training-log and judge JSONs into site/data/. The generation cards are driven by data/generations.json so results landing tonight can be published by editing one file.
This site
Muse, Meta's AI model, powers Ask and Explain
Questions go to a small server endpoint that lets Muse (muse-spark-1.3, via Meta's API) look things up in the research record and show its steps. It answers; it did not run the experiments. If the endpoint is unreachable, those features hide themselves.
This site
Animation
The jackhammer's reverberation field with its still centre, the crowd resolving into one voice, and the wall it hits, are one GPU particle system (three.js r170, vendored) morphing between three target shapes as you scroll. It caps pixel ratio, pauses when off-screen, and renders a single static frame when your system prefers reduced motion. Techniques after Codrops' scroll-driven shader articles and this author's earlier site, itsthemedium.com.

Methods & data

In simple terms.

The bench mixes clean read speech (LibriSpeech, speaker-disjoint train and test sets) with noise: a crowd of 8 to 12 talkers drawn from six held-out interferer identities, a single competing talker, speech-shaped noise, held-out white noise, and quiet. Each mixture is scored before and after the model under three intelligibility proxies: waveform STOI, vocoded ESTOI and a simulated 8-of-22 ACE electrodogram correlation, plus SI-SDR. A spatial screen checks that direction cues (ILD, ITD) are not damaged on simulated scenes.

Technical details

Model: causal Conv-TasNet-style enhancer, 602,510 parameters, past context 1.02 s, no look-ahead; 16 kHz.
Training (Gen 19): 24k steps, batch 8 × 2 s, 10% quiet examples, Adam, cosine 1e-3 → 1e-4, 3 seeds (20261117/18/19), 251-speaker train-clean-100 pool.
Bench v2: n = 6 crops per cell; SNRs −5/0/5/10 dB; systems: unprocessed, classical causal gate, non-causal oracle, candidate.
Scorecard: 14 required rows plus one diagnostic; verdicts PASS / FAIL / INCOMPLETE from kpi_scorecard.py; +3.0 dB crowd requirement is about a third of the oracle headroom.
Screens (Gen 21–24): candidate minus same-seed Gen 19 at −5 dB must be ≥ +0.30 dB on both the fixed VAL panel and the bench, with 0 dB, quiet and other-row safeguards.
Tests: 17 bench/spatial tests, 42 trainer/judge tests across Gen 20–24.

Limitations of the bench itself

Synthetic mixtures of read speech; six interferer identities in the crowd pool.
Spatial scenes use an assumed head model; one seed.
The vocoder listening-pack builder does not yet enforce the current scorecard, checkpoint identity or fresh-material clearance.

Sources

SUBMISSION.md, the judges' summary, treated as canonical wording here.
STATUS.md, the running log, newest first.
Gen 19 results · Gen 21 results · Gen 22 results · Gen 23 pre-registration · Gen 24 draft
KPI contract and scorecard script.

Downloads

scorecard.json14 rows, Gen 19 seed-mean
bench_snr.jsonper-SNR and per-condition deltas
val_traj.jsonVAL trajectories, seed 17
gen21_replication.jsonjudge output, verbatim
gen22_screen.jsonjudge output, verbatim
generations.jsondrives the tree and cards
model.jsonparams, latency, causality
tools/extract.pyreproduces all of the above

No audio is published. The repository's example and listening-pack folders are not part of this site.