Model · speech-to-text

Parakeet TDT 0.6B v2: English–Hebrew code-switching

Speech-to-text that gets the Hebrew words right when an English speaker in Israel drops them into English: "pick up some bamba from the makolet".

Model on Hugging Face Try the demo Download Project tracker

What it fixes

Stock Parakeet v2I want to go to the Micollette today to buy my groceries.
This fine-tuneI want to go to the makolet today to buy my groceries.
Makolet (מכולת) is the Israeli corner grocery shop. Stock v2 has never heard the word, so it guesses an English-looking name. Recorded in the demo Space, 2026-10-15.

About the model

Stock Parakeet v2 is very good at English, but it has never seen romanized Hebrew. A Hebrew word in English speech comes out as whatever English words sound closest. The v2 tokenizer has no Hebrew characters, so this fine-tune teaches the model one romanized spelling for each Hebrew word.

It was trained on read-aloud sentences that I recorded on the Android phone that runs the model, plus some plain English so that it keeps its English. It runs fully offline on a phone or a Linux desktop.

This is release v2 of the fine-tune (2026-10-15), trained from stock Parakeet v2 on 120 minutes of code-switched speech.

I chose v2 over Parakeet v3 on purpose. In my own dictation v2 was more accurate for English; v3 adds 25 languages and gives up some English accuracy.

Which file should I download?

If unsure, take Q8_0. On desktop CPUs it was also faster than Q4_K_M (about 3× realtime against 1.4× on 8 cores), and both gave the same transcripts as the full model on the test clips.

Recommended

GGUF Q8_0 730 MB

Android phone or Linux desktop. Take this one if unsure.

Run with Offline Voice Input (Android) / Handy (Linux) Download GGUF Q8_0 →

GGUF Q4_K_M 475 MB

Phones with 4 GB of RAM or less, or if Q8_0 is too slow.

Run with Offline Voice Input Download GGUF Q4_K_M →

.nemo checkpoint 2.47 GB

Python, GPU inference or further fine-tuning, at full precision.

Run with NVIDIA NeMo Download .nemo checkpoint →

On Android

  • Install Offline Voice Input (source), download the Q8_0 GGUF on the phone, import it in the app and select it.
  • Use it as the system voice input, or through a keyboard whose mic key opens it: SwiftKey, HeliBoard or FlorisBoard. Gboard's mic button cannot hand speech to it.
  • Set the app's battery optimisation to Unrestricted so Android does not kill it between dictations.

On Linux

  • Handy loads the same GGUF. Copy it into ~/.local/share/com.pais.handy/models/ under its own file name.

Results

v1, run r2-en30, evaluated 15 October 2026. Lower WER is better; a higher share of terms correct is better.

Test setStock v2Fine-tune
Code-switching, held-out test split, 121 clips (WER)9.9%4.8%
Code-switching test, phone recordings only (WER)9.4%4.3%
Code-switching test, desktop-mic recordings only (WER)13.8%8.6%
Hebrew terms never seen in training: share correct29%50%
Hebrew terms seen in training, in test sentences: share correct38%75%
Speaker's own plain English (WER)2.7%2.4%
LibriSpeech test-clean, 200 utterances (WER)1.2%1.1%
Long form, 5 files of ~3 minutes (WER)7.8%4.5%
  • WER is lowercase with punctuation removed. Both models used the same NeMo harness (bf16, greedy decoding).
  • The test split is term-disjoint: about 1 Hebrew term in 12 is held out of training entirely, and the "never seen" row scores only those terms.
  • v1 (earlier on 2026-10-15) scored 9.5% → 4.5% WER and 30% → 55% on unseen terms, but on the older 85-clip test split, so its figures are not directly comparable with these.
  • English did not get worse. The jump on unseen terms suggests the model learned how this speaker's Hebrew maps to the romanization, not only the training words.
  • Against other models: on my English–Hebrew code-switching ASR benchmark (test set v1, 84 clips no model trained on, 2026-10-15), release 2 of the fine-tune (built on Parakeet v2) gets 65.7% of Hebrew words right, ahead of Whisper-Hebrish (54.1%), Whisper Large V3 (45.9%), Whisper Large V3 Turbo (40.1%) and stock Parakeet v2 (32.6%). See the leaderboard and write-up linked below.
Word error rate, stock Parakeet v2 against the v2 fine-tune, on each test setShare of Hebrew terms transcribed correctly, stock v2 against the v2 fine-tune, for seen and unseen termsHow stock v2 handles 615 Hebrew terms before fine-tuning: always right, sometimes right, never right

Training

  • 120 min of code-switched speech (1,248 clips: phone recordings plus an earlier desktop-mic set).
  • 29 min of the speaker's own monolingual English and 17 min of LibriSpeech-PC replay, so about 72% code-switching and 28% English. Fine-tuning on code-switched speech alone is known to damage a model's English.
  • Full fine-tune from stock v2 (not continued from v1) in NeMo 2.7.3: AdamW, lr 3e-5, cosine schedule with 10% warmup, 20 epochs, batch 16, bf16. 23 minutes on one L40S; best checkpoint by validation WER at epoch 17. Run name r2-en30.
  • Data pinned to Read-Aloud-Voice-Sentences at revision 3bdaa2a50d52.

Training data

Checkpoints and roadmap

Where the model is now and the checkpoints still to come. Planned stages are the current plan, not promises.

  1. Baseline Done 15 October 2026

    Stock v2 got 28% of Hebrew term occurrences right across 615 terms.

  2. v1 pilot (pilot-en30) Done 15 October 2026

    First release: 77 min of code-switching, 30% English. Superseded by v2; kept in the Hugging Face repo history.

  3. GGUF export Done 15 October 2026

    transcribe.cpp v0.1.3; identical transcripts to PyTorch on 5 test clips.

  4. v2 retrain (r2-en30) Done 15 October 2026

    Released: 120 min of code-switching, 28% English, trained from stock v2. Same files and download links; .nemo plus GGUF Q8_0 and Q4_K_M.

  5. Data collection In progress

    Recording continues, weighted toward the terms v2 still gets wrong.

  6. v3 at 180 min Planned

    Retrain at 180 min of code-switching audio, comparing 0%, 15% and 30% English and keeping the mix that does best on both code-switching and general English.

  7. On-phone test Planned

    Measure accuracy and speed of the GGUFs on the target phone.

  8. ONNX export Planned

    For sherpa-onnx apps such as Voxtype and Dictus.

Limitations

  • One speaker: an English speaker with an Irish accent living in Jerusalem. Gains on other voices are untested.
  • About 2 hours of code-switched audio, with most Hebrew terms recorded only once or twice. Unseen terms are still right only about half the time.
  • Spellings follow one convention (the Hebrew-Latin-Token-Classifier lexicon), so the model may prefer it over other romanizations.
  • On-device accuracy and phone speed are not measured yet beyond a five-clip GGUF check.

Related

ASRParakeetNeMocode-switchingHebrewEnglishon-deviceGGUF