Dataset · token classification · Hebrish
Hebrew-in-Latin Code-Switching Corpus
English sentences with Hebrew words written in Latin letters, every Hebrew span marked token by token, for finding those words in text before text-to-speech mispronounces them. No corpus like it had been published.
The dataset in numbers
Counted from the dataset files at revision 2364d2f2654c (built 2026-10-15; unchanged since apart from the card). Splits: train 3,802 sentences / 987 seed terms, validation 467 / 124, test 468 / 124. 3,855 marked spans; 216 sentences carry masked spans.
About the dataset
English text-to-speech reads romanized Hebrew as English: makolet comes out as "MACK-oh-let". The fix is to find the Hebrew spans, convert them to Hebrew script and send each to a Hebrew voice. This corpus trains the classifier that does the finding. There was no heb_Latn label in GlotLID v3, lid.176 or lid218e, no token-level code-switching model covering Hebrew, and no Hebrew-English code-switching corpus of any kind.
The labelling test is pronunciation, not etymology: would an English voice mispronounce this word, and would treating it as Hebrew fix that? So kosher, Shabbat and hummus are negatives, while makolet, mashkanta and arnona are positives. 574 sentences are hard negatives built to trap a classifier: English senses, substring traps, anglicised loanwords and lookalikes.
Generation was term-seeded: a model was given one Hebrew term and asked for sentences using it, and any sentence where the term did not match at a word boundary was rejected. A second model from a different family then annotated each sentence blind, without being told the seed term. Agreements (87.7%) were accepted, disagreements went to human review, and a random 10% of agreements was audited too: 757 human-reviewed tasks in all.
458 sentences are dictated: the text of sentences read aloud for the Hebrish speech fine-tunes, so a human chose to say every one. Their labels are recomputed against the term inventory, not copied. The rest of the corpus is synthetic; treat the audit as the quality estimate.
The splits are term-disjoint: no seed term appears in more than one split, so a model is scored on Hebrew words it has never seen.
Checkpoints and roadmap
Each training run is pinned to a dataset revision, so every model checkpoint can be traced to the exact data it saw. Planned stages are the current plan, not promises.
- Corpus built and published Done 15 October 2026
4,737 sentences over 1,235 seed terms, term-disjoint train / validation / test splits, plus terms.csv.
- Token classifier v1 trained on it Done 15 October 2026
xlm-roberta-base fine-tune: span F1 0.941 on the test split, 0 false positives on the 58 hard-negative test sentences.
- Moved to the Hebrish organisation Done 19 October 2026
Now Hebrish/hebrew-latin-code-switching; data unchanged.
Layout and columns
One JSON Lines file per split, data/{train,validation,test}.jsonl, plus terms.csv: the term inventory with category, status and accepted spelling variants. load_dataset("Hebrish/hebrew-latin-code-switching") works directly. load_dataset("Hebrish/hebrew-latin-code-switching") works directly.
| Column | What it holds |
|---|---|
id | Row id (g… for generated) |
text | The English sentence |
source | generated or dictated |
seed_term / category | The term the sentence was generated around, and its topic (bureaucracy, food_shopping, … or negative:<kind> for hard negatives) |
spans | Hebrew spans as {start, end, surface, term}, character offsets into text |
masked | Character spans with no label: model disagreements no human has ruled on, and Israeli person names. Ignore their tokens when training or scoring (label -100) |
tokens | [token, tag] pairs, tags O / B-HE / I-HE; multi-word terms such as bituach leumi are B-HE I-HE |
Models trained on it
Related
MIT. Built in Hebrish-ASR/Hebrew-Latin-Token-Classifier, which holds the generation, annotation and review code and the annotation policy.