In July 2024, the French lab Kyutai unveiled Moshi, one of the first real-time conversational AI models you could actually talk to. Most voice assistants of the time were a relay race: speech-to-text, then an LLM, then text-to-speech. Each hand-off added latency, so you waited a couple of seconds for every reply.
Moshi skipped the relay. It took audio in and produced audio out directly, as continuous streams. It could listen while it talked, so you could interrupt it, and it answered in roughly 200 milliseconds, about the speed of human conversation.
It was also, frankly, not very smart. The model was small and lightly trained next to the giants of the day, and the real-time constraint left little room for deep reasoning. Early users noticed quickly:
just tried it and... it's really, really bad. leaves me wondering if the demo was fake? https://t.co/N5LY2AL61a
— ben hylak (@benhylak) July 3, 2024
Still, Moshi showed that voice AI didn’t have to be a slow pipeline, and it was open-sourced so others could build on it. It’s still around, and still worth a chat.