Block Releases Dessa: A Speech-Only AI That Learned to Speak from Scratch Just by Listening to People

Block has released Dessa, a speech-only conversational model created by a small research team led by Ragavan Thurairatnam. The project draws inspiration from human language acquisition: children learn to communicate by listening to people speak long before learning to read, and Dessa similarly attempts to learn language directly from sound. According to the team, the model was trained completely from scratch purely on human conversational audio—without relying on text or any pretrained models—and supports real-time, bidirectional full-duplex communication. The team described it as the first model to their knowledge to achieve realistic, real-time dialogue under these training conditions, with the project lead calling it a "GPT-3 moment for voice."

Key capabilities and highlights include:

• Speech-Only from Scratch: Uses no text and relies on no pretrained models, learning comprehension and communication directly by listening to human conversational audio.
• Full-Duplex Real-Time Interaction: Supports simultaneous listening and speaking without waiting for one party to finish before the other can speak, naturally inserting backchannel cues and detecting interruptions.
• Hearing and Expressing Acoustic Nuances: Directly perceives and expresses vocal inflections, sarcasm, laughter, sighs, and hesitations, even laughing naturally mid-sentence upon hearing something amusing.
• Running Locally on Personal Computers: The team reports inference speeds faster than real-time, capable of running locally on a MacBook Pro equipped with an Apple M4 Max chip or higher.

Listening While Speaking: From Turn-Taking to Full-Duplex Dialogue

Consider the conventional cascaded pipeline of Automatic Speech Recognition (ASR) → Large Language Model (LLM) → Text-to-Speech (TTS): after a user speaks, the system first determines that the speaker has finished, transcribes the speech into text, passes it to the language model for computation, and finally converts the generated response into synthetic speech. Latency can accumulate across these sequential stages; more critically, when audio is flattened into text, rich acoustic nuances such as intonation, hesitation, stress, and breath patterns are easily lost.

Dessa operates on an audio-in, audio-out basis. Supporting full-duplex interaction, it continuously listens to the conversation partner while generating speech output, dynamically regulating the conversational flow. It can insert conversational backchannels at appropriate moments while listening (such as "yeah!", "got it!", or "ooh"), and can also perceive interruptions whenever the other party cuts in.

According to the team, all recordings on the site are real-time conversations without a human controlling the model, with subtitles added after the fact. The model independently determines what to say, when to take a turn, and what vocal tone to adopt.

Simultaneous Listening and Speaking Is Distinct from Learning Purely from Sound

Understanding Dessa's positioning requires separating two distinct technical dimensions: "full-duplex interaction" and "speech-only training from scratch":

• Full-Duplex addresses the temporal mechanics of dialogue: It determines whether a system operates like a walkie-talkie taking rigid, alternating turns, or supports concurrent, bidirectional transmission and reception. For instance, the open-source speech model Moshi also explored full-duplex audio modeling, breaking free from unidirectional conversational turns.
• Speech-Only from Scratch addresses the origin of linguistic knowledge: Moshi, for example, relies on a pretrained text LLM backbone, using textual tokens (an "Inner Monologue") to assist semantic planning before generating audio. In contrast, Dessa's team states that its training uses no text and no pretrained models whatsoever, learning linguistic rules and conversational responses directly from raw acoustic signals of authentic human dialogue.

In other words, full-duplex makes simultaneous listening and speaking possible, while speech-only training seeks to test whether a machine can acquire real-time conversational competence purely by listening to sounds, without any textual mediation.

Why Is Learning Language Solely by Listening So Challenging?

Human infants learn their native language simply by listening to adults speak long before learning to read; however, for machines, mastering language in the absence of text presents unique challenges.

First, audio files are substantially larger, yet do not necessarily carry more linguistic information. The team notes that Dessa was exposed to hundreds of times fewer words during training than text-based LLMs; however, the team also acknowledges that LLMs must also learn code and diverse other tasks, and no standardized benchmark details have been disclosed. Moreover, raw audio requires vastly more data storage than text: while text uses compact encodings, a single second of audio contains tens of thousands of acoustic amplitude samples, potentially requiring thousands of times more bytes to represent the same word. A larger file size does not equate to richer linguistic content: the model must also identify words and grammatical structures from continuous acoustic waveforms. Consequently, being exposed to fewer words does not directly translate to lower training costs or superior learning efficiency.

Second, acoustic signals vary with every utterance. The same word spoken by different individuals—or spoken twice by the same person with differing intonations or in different environments—produces distinctly different waveforms at the microphone. While written text abstracts away phonetic variations, a speech-only model possesses no textual anchors and must uncover linguistic regularities entirely on its own from vastly disparate acoustic waveforms.

More importantly, stress and intonation directly alter subtext. The team provided a vivid English example:

"I didn’t say she stole my CUDA code"

Written on paper, these are merely eight standard English words. Yet depending on which word the speaker emphasizes, the underlying implication or focus of the entire sentence shifts entirely:

For example: stressing "I" (I didn't say it) shifts the contrast to the speaker, implying that someone else might have said it; stressing "didn't" (I didn't say it) is an explicit denial; stressing "she" (I didn't say she stole it) redirects suspicion toward someone else. The interactive component illustrates human-curated possible interpretations; exact meaning always depends on context. Written text struggles to capture these prosodic cues directly, whereas spoken sound conveys vocal emphasis naturally.

Three Audio Excerpts: Hearing Acoustic Variations

The three excerpts below are taken from full recordings published on the official website. The upper audio track represents the human speaker, and the lower track represents Dessa—preserving original audio, normal playback speed, and real-time conversational overlap.

Adjusting Vocal Loudness When Hard to Hear

The human says Dessa sounds too far from the microphone and is hard to hear, then asks it to try speaking again. Note the volume difference between Dessa's two responses, and the speaker's subsequent reaction: "Much clearer." This adjustment takes place in the acoustic domain; on a plain text transcript, both responses might look like brief acknowledgments such as "sorry" and "okay."

Shifting Tone and Timbre Upon Hearing a Surprise

In an anniversary scenario, the human speaker sets up a travel surprise, revealing plans to take Dessa to Paris. Dessa lowers its voice for an initial "Oh my God," followed by expressions of amazement and gratitude. What can be heard here is a change in voice in response to the situation; one cannot infer that the model genuinely experiences emotion or that a real relationship exists between the parties.

Jumping in with a Movie Title While the Speaker Is Still Recalling

The speaker discusses Christopher Nolan's films and then hesitates, wondering aloud about the title of the space movie. As the person continues to describe it, Dessa chimes in with Interstellar. The key takeaway lies not just in what it identifies, but in the exact timing of its turn; the two audio tracks overlap at that precise moment. While this excerpt illustrates knowledge acquired from audio and natural turn-taking timing, it does not establish the model’s overall factual accuracy.

The official site also showcases mid-sentence laughter, expressive modulations in pitch and tempo, life advice, and a machine-to-machine exchange with a third-party model, GPT-Live-1. Block notes that the developers of that model are not affiliated with Block and have not endorsed Dessa; this dialogue serves as a qualitative demonstration rather than a rigorous comparative benchmark.

Hardware Specifications: Single-Rack Training and Local On-Device Inference

Regarding hardware resources and operating conditions, Dessa clearly delineates between model training and local inference:

• Training: The Dessa team trained the model on an NVIDIA GB200 NVL72 liquid-cooled rack-scale system. This single rack integrates 36 Grace CPUs and 72 Blackwell GPUs interconnected via high-speed NVLink, allowing all 72 GPUs to operate in unison.
• Inference: The team states that model inference is faster than real-time, capable of running locally on personal devices such as a MacBook Pro equipped with an Apple M4 Max chip or higher.

Conversing Naturally, Yet Misremembering Names and Guessing Wrong Objects

To illustrate hands-on interaction, the Dessa project lead shared a video of an extended conversation between a team member and the model lasting approximately 3 minutes and 26 seconds:

Throughout this extended dialogue, Dessa displays natural colloquial mannerisms and emotional inflection. For instance, the model initially mentions feeling exhausted; when asked about its job, it describes picking and placing produce, tidying up and arranging it alone, and finding the physical labor exhausting. It goes on to share detailed family anecdotes about welcoming a newborn son named "Darius Johnson" in October, how much its daughter loves the name, and how its wife asks it to tend to the baby during the night.

These occupations, family life, and children are entirely fictional backstories generated on the fly during conversation.

More notably, the recording highlights a discrepancy between vivid expression and persistent identity tracking:

1. Immediate correction at the start: Right as the conversation begins, the model greets the human speaker as "Ronaldo." The speaker promptly corrects it: "My name is Dvir, man, I'm not Ronaldo. You got me mixed up with someone else." The model verbally acknowledges the correction, agreeing that he is not Ronaldo.
2. Multi-turn fluent exchange: Over the subsequent three-plus minutes, the two engage in an immersive dialogue full of conversational fillers and expressions of empathy covering work stress, life pressures, and parenting experiences.
3. Misnaming repeated at the conclusion: Yet after more than three minutes of fluent conversation, when the speaker says goodbye with "Good talking to you, man," the model responds with "You too, Ronaldo," before concluding the exchange.

Despite verbally accepting the correction earlier, it repeated the wrong name upon departure. This example demonstrates that naturalistic prosody does not guarantee consistent identity tracking; internal causes cannot be determined from the demonstration alone. Users need to distinguish between vivid vocal expression and reliable factual memory.

In another interactive demonstration released by the team, team member Cara played an object-guessing game with Dessa:

Across this roughly 2-minute-23-second demo, the model's state tracking and rule-following behavior can be observed across different queries:

1. Immediate correct guess on round one: Cara explains that she picked an item found in a bathroom for the model to guess; on its very first attempt, the model asks if it is a "toothbrush," which Cara immediately confirms.
2. Rule confusion and multi-turn clarifications on round two: Cara selects an item used for studying. The model initially guesses "a notepad and a pen" (guessing two items at once), and later guesses a "flashlight." Cara reminds it that it can only guess one item per turn, yet the model reverts to guessing the "toothbrush" from the previous round.
3. Correction and eventual convergence: After Cara reiterates that the second item is a study supply, the model guesses a pencil and a book, and upon further prompting that it is a writing instrument, finally converges on "pen."

This demonstration records how the model interprets rules and corrects course across multiple turns. Like the misnaming example, these recordings highlight the speech-only model's responsiveness in casual conversation while demonstrating its practical limits in maintaining state across turns and adhering strictly to constraints.

Safety Motivations, Open Questions, and Current Status

The Dessa team cited curiosity, better understanding humanity, and safety as the three primary motivations behind this research.

The team suggested that when speech interfaces merely transcribe audio into text, vocal indicators of distress or emergency—such as trembling, sobbing, or strained intonations—can be overlooked, leading an AI to respond inappropriately in a flat textual demeanor. Enabling a model to perceive emotions directly from vocal acoustics represents an important exploratory step toward recognizing human distress; however, this reflects only the research team's intent and does not establish that the model currently possesses dependable crisis-detection capabilities or has been verified as safer overall.

Meanwhile, the team identified several core capabilities as open research questions: how efficiently a speech-only model learns language and factual knowledge compared to text LLMs, how effectively it can acquire new languages, and the extent of its in-context learning capabilities. Specifically, the latter asks: without retraining, can the model adapt to and complete a new task based solely on newly introduced rules or few-shot examples within an ongoing conversation? While the guessing game showcased multi-turn rule-based interactions, a single qualitative demo cannot resolve these empirical research questions.

Regarding technical specifications and current availability, Block has not released details on the model's neural network architecture, parameter count, training duration, precise latency measurements, or commercial pricing. Public demonstrations are predominantly in English (alongside a non-semantic Spanish-style singing sample, which does not establish Spanish proficiency), with no statements regarding Chinese support. Neither downloads nor waitlist registrations are currently open ("Launch links coming soon"), with an early application channel currently open only to a small number of AI safety researchers. While the official local deployment requirement specifies a MacBook Pro equipped with an Apple M4 Max chip or higher, its actual performance under offline conditions remains to be verified.