Making comprehensible input compelling

While walking through Chicago one day I came across a used bookstore and saw a book about learning Spanish by reading stories. It caught my interest for two reasons.

The first is that I'd tried learning a language the traditional way. I did three years of high school Spanish and have lived in Southern California my entire life, and yet when I travel to a Spanish-speaking country I barely follow what's said to me beyond the basics: ordering food, or asking if I can go to the bathroom (¿puedo ir al baño?, iykyk).

The second is that I remembered reading about this. As an avid consumer of the internet I'd come across comprehensible input: the idea that input is king, but not just any input. Comprehensible input means taking in language where you understand most of it, but not all of it. You fill the small gaps from context, and it stays easy enough that you keep going.

So I bought the book, $8.00, convinced this would finally be the start of my bilingual journey.

That was optimistic. My Spanish was much worse than I'd imagined. I could get the rough idea of the first story, but it was grueling: stopping to look up every other word, often several in a row. I was probably at about 40% comprehension. (I made a tool that shows what each level feels like in English: how many words you need to know.) That's input, but not comprehensible input. By the time I hobbled to the end of the first story I was exhausted, demotivated, and still not sure what it was about.

About a year later the idea came back to me. With where the technology is now, maybe I could redo the spirit of that book with better execution, tailored to the reader. The book had three problems:

• The stories were too hard.
• They weren't written for my vocabulary, just any vocabulary.
• I didn't care what happened in them.

It wasn't just me

Once I started digging, it turned out these weren't just my feelings. There's a whole body of research behind them.

The first problem, the stories being too hard, is probably the best studied. Researchers have actually tried to put a number on "understand most of it". The best-known study gave readers the same story at 80%, 90%, 95% and 100% of the words known, and found that while a few people got by at 95%, most needed about 98% to read it on their own.^[1] Later work settled on 95% as the bare minimum and 98% as the sweet spot,^[2] and found that it isn't a cliff you fall off so much as a slope: the more of the words you know, the more you understand.^[3] To be fair, that 98% number is shakier than it usually gets credit for, and a 2023 replication couldn't fully reproduce the original results.^[4] But nobody is arguing the direction. At my 40%, I wasn't really reading, I was decoding.

You can feel the difference yourself. Here's the opening of Alice in Wonderland with the rarest words swapped for made-up ones, at different levels of known words:

The second problem was that the book wasn't written for my vocabulary, and the research has something to say here too. Running into a new word a few times doesn't do much. In one study, three months after reading a graded reader, learners could recall the meaning of only about 4% of its new words, and of none they'd met fewer than eight times.^[5] Another found that even ten encounters gets you to recognizing a word, but not yet to using it.^[6] So a story that throws a new word at you once and moves on isn't teaching you that word, it's just in the way.

The third problem, not caring what happened in the stories, sounds like the softest of the three, but it might matter the most. Krashen, the linguist behind comprehensible input, later argued that input shouldn't just be comprehensible but compelling: interesting enough that you forget you're reading in another language at all.^[7] Which, looking back, was never going to happen with that book.

So the three problems with the book map onto three requirements:

1. Hit about 95–98% known words, for this reader.
2. Bring a few new words, on purpose, and use each more than once.
3. Be about something the reader actually cares about.

What I built

An engine for comprehensible and compelling input. I built what I was looking for: a core loop that could deliver that, while also supporting the best practices from industry-standard tools like Anki. From that came Kin.

Attempt one: just ask the AI

The obvious version: tell the model "write a story at A2 level, 95–98% words the learner knows". The stories looked fine. Then I measured one. A real A2 story came out at about 78% known words. Models are bad at judging what's easy for a particular learner, and "A2" means nothing about which 2,000 words this person knows.

And just like that I was right back where I started, only now I had spent a bunch of time building it... great. But luckily I didn't give up there.

The lesson: you can't prompt your way to a coverage number. You have to measure it.

Modelling what one learner probably knows (no AI)

To measure coverage you need a guess, for every word, at whether this reader knows it. Kin does that without a model call:

• Start from a size, not a list. Placement gives an estimate: "about 800 words". Each word's chance of being known comes from its frequency rank on a curve centred on that estimate:

The Nth most common word is a coin flip, the word at N/2 is about 85%, the word at 2N about 15%.

• The learner's own record overrides it word by word. Marked "known": 0.97. Tapped but not yet reviewed: 0.2. Reviewed words: whatever the spaced-repetition memory model says their recall is now.
• Forms map to dictionary words. "llueve" counts as "llover", from a forms table built once per language and committed, so there's no AI at runtime.
• Cognates get credit. A word spelt like its English meaning ("chocolate", "conservar" → "to conserve") counts as at least 80% likely known. It's spelling only, so it sometimes lets a false friend through ("carrera" isn't "career").
• Names and numbers don't count, as in the research.

The estimate learns from every story. After a story, the words new to the learner's record that they tapped were unknown, and the rest were probably known. "Probably", because people read past words they don't know, so untapped words get half weight. A grid search finds the vocabulary size that best explains that pattern, and the estimate moves a damped step toward it, 60% of the way after the first story and 25% from the fifth on. It never moves up after a failed comprehension check.

Write, measure, rewrite

1. Write. The writer is given the words this learner very probably knows (odds of 70% or more, up to 1,200 words) and asked to use only a few less common words, each twice.
2. Choose the new words afterwards. Kin picks them from what the story actually leans on: the unfamiliar words it uses most. They're taught before reading (an index card each), so they count as known. Choosing new words up front didn't work: they rarely fit the subject and the writer crammed them in.
3. Measure. Coverage is computed against the learner's word odds.
4. Rewrite if it's under target. A second prompt replaces the specific words costing the most, at most twice, keeping the best version. The target is 97% for the first three stories, 96% up to the sixth, then 95%. Too easy is pleasant; too hard is how people quit.

Here's how that worked out on six real stories:

It's still short of 98%, and small vocabularies rarely get there: with 300 words there isn't much to write with. So the reader shows the honest number before you start: "You'll know about N% of the words."

Then real users said "too hard" anyway

The first two learners to rate a story both said Too hard, and three of four stopped after one story. And really, that was just another signal of what I already knew. The stories need to be easy. If they're even slightly too hard, it's too painful to continue long enough to actually gain something. Two fixes:

• The first story aims low no matter what placement said: it's measured against 0.6× the estimate and aims for 99%. The first story is where someone decides whether this works.
• "Make it easier": a button in the reader that rewrites the same story more simply, keeping the same facts and photo. Pressing it also counts as a Too hard rating and lowers the estimate.

Making them worth reading

Finally, and potentially the most important factor, the content needed to be worth reading. I've read plenty of "The hat is red" or "The man is hungry". Nice, just about as fun to read as the instruction manual for my toaster. My time is precious to me, and if I'm going to be consuming input, especially in a language that's a little foreign to me, I need it to be something I care about reading.

For me that means keeping up with motorsports, more specifically F1, IMSA and WEC. I like following these anyway, so reading about them in a different language might take me a couple of minutes longer, but it doesn't feel like an entirely new allocation of time, since it's something I'd already want to read. So Kin pulls real stories for me to read.

What's next

Now all that's left to do is use it. I guess that's the benefit of being my own ICP! I've committed to 15 minutes a day, but I'm really trying to push it to 30 minutes if I can, and fixing any issues I find along the way. Hopefully this post is followed up one day with a discussion about how I became bilingual, but for now I'll continue to read about my favorite little racecars at the vocab level of a toddler.

Sources

1. Hu, M. & Nation, P. (2000). Unknown vocabulary density and reading comprehension. Reading in a Foreign Language 13(1). ↩
2. Laufer, B. & Ravenhorst-Kalovski, G. C. (2010). Lexical threshold revisited. Reading in a Foreign Language 22(1). ↩
3. Schmitt, N., Jiang, X. & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. Modern Language Journal 95(1). ↩
4. Kremmel, B., Indrarathne, B., Kormos, J. & Suzuki, S. (2023). Unknown vocabulary density and reading comprehension: replicating Hu and Nation (2000). Language Learning. ↩
5. Waring, R. & Takaki, M. (2003). At what rate do learners learn and retain new vocabulary from reading a graded reader? Reading in a Foreign Language 15(2). ↩
6. Webb, S. (2007). The effects of repetition on vocabulary knowledge. Applied Linguistics 28(1). ↩
7. Krashen, S. & Bland, J. (2014). Compelling comprehensible input, academic language and school libraries. Children's Literature in English Language Education 2(2). ↩