Turning a Person's Story into an Animated Face: Inside Mosaic Studio's Design Method

Agu Seguí released Mosaic Studio, a web-based generative animation engine custom-built for the design interview show First of Kind (Season 3, Episode 1 featuring Vercel founder Guillermo Rauch). The engine arranges key words representing the interviewee into an animated typographic portrait, powering both real-time animation on the web and frame-by-frame rendering of high-definition video assets for show intros and social media promotion.

The visual presents a profile portrait composed primarily in red, white, and black. Pulled back, the camera reveals the full facial contour; zoomed into a close-up, the light and shadow are formed not by smooth color gradients, but by tightly packed rectangular text blocks of varying lengths containing uppercase English words such as BORGES, LANUS, ARGENTINA, alongside DEPLOY, VELOCITY, SLOP, and LATENT.

Words Up Close, a Face from Afar: Why Does It Work?

A typographic portrait works at two scales: individual words remain readable up close, while their arrangement forms a face from a distance.

Viewed up close, each word block appears as an independently recognizable English word; pulled back, the viewer's gaze ceases to focus on individual letters, perceiving instead overall tonal contrasts and color blocks. Here, the position, size, text color, and background tone of each block collectively define the contours, light, and shadow of the face, behaving much like brushstrokes in painting.

In his retrospective, creator Agu Seguí noted that this word-as-image visual structure reminded him of Chuck Close's gridded portraits, Georges Seurat's pointillism, and Vincent van Gogh's brushstrokes: each individual symbol maintains its discrete local form, yet through arrangement in position and value, they coalesce into a coherent human face.

Word Meanings Determine Dynamic Skins

Treating words merely as static text blocks would yield nothing more than a typographic collage. Mosaic Studio stands out by assigning visual "skins" based on semantic meaning: words in the same category share the same drawing effects and motion rules.

Throughout the interview, Guillermo Rauch reflects on his upbringing, learning to code, and thoughts on front-end development and AI. Agu Seguí realized that words carry distinct emotional resonances: reading BORGES (Argentine author Jorge Luis Borges) or LANUS (Lanús in Buenos Aires Province) stirs humanistic and cultural memories, whereas DEPLOY, SLOP, JSON, and LATENT evoke engineering and computational vocabulary.

The system organizes these associations along a semantic axis spanning humans and machines: extending from personal experiences, memories, and handcrafted judgment to business, audiences, code tools, and machine intelligence, with speed and noise (friction) acting as two external forces spanning both ends. These categories stem from predetermined design conventions. The author bound corresponding shader skins to each category, granting word blocks distinctly different visible dynamics:

• BORGES (Past Memories → emulsion): Echoes cultural memory through film grain. Backgrounds and letter edges consist of fine particles; on entrance, grain clusters appear first before gradually revealing the word, resembling a photographic developing process.
• CRAFT (Artisanship → ink): Reflects human handcrafted aesthetic and judgment. Viewed closely, it features fine paper grain textures and bleeding ink edges; on entrance, it sweeps in from left to right like a brushstroke, unaffected by glitches spreading from neighboring blocks.
• VELOCITY (Speed & Cadence → smear): Reflects the pace and speed of engineering delivery. The visual generates dynamic motion smearing along the horizontal X-axis; on entrance, the word block sweeps in from the left with an elongated trail smeared to the left, backed by horizontally stretched noise stripes and intermittent sweeps of bright light streaks.
• SLOP (Frictional Drag → corrupt): Captures disorder, noise, and friction within the system. The screen exhibits glitch blocks, row sliding, and RGB color separation, which propagate outward to neighboring blocks during dynamic events.
• JSON (Code Tools → cursor): JSON is a ubiquitous programming data format, visualized here through the appearance of terminal typing. A block cursor types out words character by character; when swapping words, it can highlight the old word before typing the new one; once typing finishes, select focused word blocks retain a blinking cursor.
• LATENT (Machine Intelligence → latent): Points to the unformed latent space of models. It manifests as an unstable dot matrix in constant micro-motion; on entrance and transitions, words progress from coarse blocks to sharp letterforms, revealing preset segmented seams (such as LAT|ENT)—a visual metaphor for machine intelligence.

In this way, a word carries not only text content, but also an associated set of concrete presentation rules. When moving to another guest, these categories and skins can be reused simply by populating words and portrait layouts for the new episode.

The categorization rules in the code represent predetermined design conventions, establishing explicit data mappings between existing words and skins. The public materials do not disclose the specific process by which vocabulary from interview recordings was selected, extracted, and categorized.

How Do Shaders Turn Rules into Visuals?

On standard web pages, typesetting numerous individual HTML elements and animating them simultaneously tends to create browser performance bottlenecks. Mosaic Studio employs WebGL2 to drive its entire typographic grid. At its core are shaders—rendering programs running on the GPU designed to ingest state data for the current frame and compute per-pixel color fills, contour distortions, glows, and noise effects.

The system's entire rendering pipeline is coordinated between a JavaScript scheduler (scheduler.js) and a WebGL renderer (renderer.js), following a division of labor: "calculate each word's current-frame state first, then batch render":

1. The scheduler calculates state first: Operating on the CPU, the JavaScript scheduler determines what word each block displays on this frame, whether it is entering or exiting, and the intensity of jitter and glow, passing these batched results to the graphics card.
2. The renderer draws the scene: The system pre-renders commonly used letters onto a shared sheet called a glyph atlas. When the GPU renders a word block, it retrieves the required glyphs by index, arranges them by position and dimension, and applies the colors and transformation rules of the matching skin. The same rectangular template batch-renders numerous word blocks without maintaining an individual DOM element for every word.

Dynamic Events and Visual Propagation

In the scheduler's event table, word blocks do not merely sit static or flicker independently; certain visual effects propagate between adjacent blocks. For instance, jitter events belonging to the corrupt skin spread to surrounding blocks. These behaviors are governed by preset animation rules:

• When encountering a block with the ink skin, the program skips it directly (continue in code), leaving it untouched by glitch events;
• When encountering a block with the latent skin, the system triggers a regeneration animation (Regen), causing the block to re-transition from coarse blocks to sharp text;
• Blocks tagged with detox follow glitch reception with an erasure-like sweep animation (Sweep), clearing the corrupt artifacts.

Camera Motion Blur

Early pan and zoom transitions felt stiff and mechanical. To enhance dynamism, the author added camera motion blur (motion.js) to the rendering pipeline: during virtual camera pans or zooms, the shader takes multiple samples along the motion trajectory and blends them, giving fast-moving text a natural sense of direction; when the camera is static or moving negligibly, the system automatically skips the blur pass, preserving crisp typography while conserving render overhead.

How Does the Same Animation Serve Both Web and Video?

In typical digital creative workflows, web animations and promotional videos are handled by different disciplines: web teams write interactive animations in front-end code, while video teams re-layout, keyframe, and render inside motion design software. Whenever copy or visual layouts change, both sides must update independently.

Mosaic Studio supports both online website displays and high-definition video exports through a deterministic frame-state mechanism:

• The state evaluation function evalState(schedule, frame, out) and draw function draw(f) do not rely on the browser clock (such as Date.now()); instead, state is determined entirely by fixed scene configurations, pre-generated event schedules, deterministic random seeds, and the current frame number;
• All random events are derived from seed-based deterministic pseudorandom hash streams; motion blur sampling similarly reads deterministic fractional-frame camera parameters before and after;
• Given identical scenes, event schedules, random seeds, and frame numbers, the system consistently computes the exact same animation state.

This mechanism enables a single animation codebase to output across diverse media:

• Running on the web: Pushed forward along the timeline via the browser's render loop (such as requestAnimationFrame), calculating and drawing to the canvas in real time. Readers can visit the live website First of Kind to experience the real-time effect;
• Exporting video: Unlike web playback, which requires real-time frame rates, video rendering advances frame by frame. It can wait for one frame to finish calculating before proceeding to the next; slower computations simply increase export time. Once frames are rendered, they are encoded—such as the ProRes output noted by the author—before final adaptation to platform specifications and post-production workflows.

Interactive Storytelling on the Web

On the web, the animation is not a one-way linear video, but an interactive narrative orchestrated by a state machine:

1. The page opens on a single focal word: DRAGON;
2. A first click (or pressing Enter or the spacebar) triggers a ripple, expanding outward from that word into a background wall of words;
3. A second interaction introduces the show title FIRST / OF / KIND, after which key interview phrases play automatically in sequence with reading pauses after each phrase, highlighting associated word blocks around them;
4. As the narrative concludes, a color palette ripple sweeps across the canvas, the virtual camera pulls back, background filler words fade out, and the remaining red, white, and black blocks coalesce into Guillermo Rauch's portrait face, revealing show details and entry links.

If no action is taken during the first two steps, an idle timeout of roughly 3 seconds advances automatically; the opening can also be progressed via Enter or the spacebar, while subsequent phrases play automatically; a REVEAL button bypasses the narrative directly to the portrait, and sound can be toggled independently.

Landscape, Portrait, and Camera Reframing

While deterministic rendering ensures identical states upon re-evaluating the same frame, varying devices and formats still require intentional layout and framing design.

On the live site, wide desktop screens use a horizontal split with the portrait on the left and the info panel on the right; narrow mobile screens adopt a vertical stack with the info panel on top and the portrait spanning the full screen width. Rather than mechanically scaling down the entire image, the system recalculates portrait proportion, camera framing center, and viewport cropping for different viewports. The author also demonstrated multi-aspect-ratio previews in the workbench. A composition exercise below uses a static portrait frame from the original video. Switch between landscape and portrait formats to observe how the subject and title areas rearrange; zoom into the portrait to compare the full face with localized text.

Try the Composition Tool

When the aspect ratio changes, where do the face and title go?

Switch aspect ratios to see layout rearrangement, then zoom into the portrait to see how 'the whole face' resolves into 'individual words.'

16:9 Landscape9:16 Portrait

Reset Composition

Portrait Area

FIRST OF KIND · S03/E01One Stroke
DragonGUILLERMO RAUCHShow Title / Guest / Credits

Portrait zoom

1.0×

16:9: Portrait on the left, show info on the right. 1x shows the overall silhouette.

The base image is a high-resolution static frame from the original thread's second video, with the original top header cropped out. It is rearranged here as an independent teaching model: landscape and portrait versions share the subject and copy while arranging zones independently; values are not Studio export presets, and the original animation engine is not running.

Built in a Weekend: What Foundations Made It Possible?

Faced with such a polished system, one might easily assume that feeding a simple prompt to an AI automatically generated the whole codebase.

Agu Seguí described the project as a "low-effort" weekend build. In his view, existing design assets had already completed roughly 80% of the heavy lifting. This assessment reveals the true prerequisite for rapid development: when he stepped in, the visual language was already fully established.

1. A pre-established graphic design system: The original visual identity and design language were crafted by designers Oliver Hamrin and @haris_chc, establishing foundational layout standards and typography for the series. The system translated existing typography into on-screen motion rather than inventing a visual style from scratch.
2. AI's specific scope and role: The creator devised prompts grounded in his own understanding of graphics concepts and engineering; during video production, he delegated most encoding decisions for the audio/video tool FFmpeg to Claude Opus 5.5, without providing an itemized breakdown of which shader and math implementations involved AI.
3. Interoperable creative toolchains: Designers laid out word blocks in Paper, which the system parsed via HTML canvas and imported into Mosaic Studio for rendering; generated portraits could also be fed back into Paper via an MCP interface for further adjustment. Most visual skins originated from the Paper Shaders library, with the author extending special credit to developer @uuuuuulala for engineering contributions.

The editor's workspaces break production down into distinct tiers. From the demos, one can trace an inspection path from part to whole: first verifying whether a single word's skin and motion work properly in Cell Lab, then moving to Portrait to check full-face legibility, and finally reviewing multi-purpose segments and aspect ratio previews across workspaces like Plate, Intro, and Outro. This workflow is derived from the interface; the author did not provide a button-by-button manual, and the specific responsibilities of Plate remain undescribed.

These layers demand continuous iteration: a word's flicker or smear might look striking in isolation, yet overwhelm facial shading once placed back into the full portrait. Evaluating the macro-form first before returning to tune down local effects and adjust positioning ensures the balance between "words up close, a face from afar." Subsequent episodes can retain the skins and motion rules while adjusting portrait layouts, core vocabulary, and show metadata.

Additionally, the system leverages the Web Audio API to couple sound design with word semantics: state changes trigger micro-audio grains whose timbre corresponds to matching shader skins. The soundscape is structured across three layers: foreground audio for focal words and titles, mid-ground swells for camera translations and palette ripples, and subtle background grains from surrounding words forming ambient texture; the author explicitly stated that this audio system remains a work in progress (WIP) and will continue evolving with future episodes.

In post-production, collaborating creative studio Room3 received dynamic assets generated by the engine to edit, refine, and export final marketing assets within professional production pipelines.

Walk Through the Method: From Three Words to Two Compositions

Try the following exercise using this article's word effect and composition tools before applying this inspection method to your own material. This is an exercise designed from the case study; both interactive tools demonstrate motion rules and composition, without features for automatically generating portraits from custom photos or exporting video files. Mosaic Studio's public editor, installation packages, and full prompts have not been released.

1. Write down three words and what you want viewers to associate with them. You might choose "personal notes, computational logic, command-line interface" as NOTEBOOK, ALGORITHM, and TERMINAL. Add an intended meaning to each word, avoiding adding buzzwords merely because they are trendy.

2. Compare three entrances in the word effects tool. Scroll up to "What happens when you add motion rules to the same word?" and select BORGES, JSON, and LATENT in turn. Drag appearance progress to 0%, 50%, and 100% to observe the grain reveal, character-by-character typing, and resolution from blocks to sharp text; adjust effect strength from 0% to 75% to compare against the static reference block. You can also click Play to see continuous entrance motion. The key criteria: does the motion reinforce the word's meaning, and does the word remain legible upon arrival?

3. Manually assign skins to your words. Fill out three rows following the table below, then refer back to the word effects tool for corresponding examples. These mappings represent design choices for this exercise; a different context might call for different decisions, and this tool will not automatically extract or categorize your words.

4. Examine the face first, then the text. In the "When the aspect ratio changes, where do the face and title go?" composition workbench, set portrait zoom to 1x, then push to 3x. Use 1x to inspect facial features, silhouettes, and lighting; use 3x to ensure text remains legible and see how word blocks construct facial shading. When evaluating your own work, check across both distances rather than focusing solely on an appealing local effect.

5. Create composition sketches for both landscape and portrait formats. Switch between "16:9 Landscape" and "9:16 Portrait", resetting portrait zoom to 1x. Check the horizontal split in landscape, and the raised title with reframed portrait in portrait mode. Sketch both layouts on paper or in a design tool: determine portrait proportion, title placement, and credit locations. Verify that text is not clipped and the subject is not squeezed too small. Note that these aspect ratios are exercise canvases, not original engine export presets.

6. Apply this inspection sequence to your own workflow. Start with a static typographic portrait in your design tool, then implement motion for only three representative words. Once individual word effects work, evaluate them in the context of the full face, and finally produce intros, outros, and multi-format layouts. Save sample frames at identical timestamps across versions to verify title, word, and subject consistency. For web, test clicks, keystrokes, and viewport resizing; for video, verify start/end frames, subtitle/logo safe areas, and playback cadence before moving into post-production editing. Completing word mappings and composition sketches during this exercise helps surface design issues early without waiting for a fully completed engine.

Developer Practice: Getting Started with Paper Shaders

To explore dynamic shader textures in your own front-end projects, start with the open-source Paper Shaders. Taking an existing Vite + React project as an example, install the dependency in your project root. The official documentation recommends pinning exact versions using --save-exact in package.json:

npm i --save-exact @paper-design/shaders-react

Create src/ShaderDemo.jsx and insert the complete component below:

import { MeshGradient } from '@paper-design/shaders-react';

export default function ShaderDemo() {
  return (
    <MeshGradient
      colors={['#5100ff', '#00ff80', '#ffcc00', '#ea00ff']}
      distortion={1}
      swirl={0.8}
      speed={0.2}
      style={{ width: 300, height: 300 }}
    />
  );
}

Then import ShaderDemo in src/App.jsx and return <ShaderDemo />. Run npm run dev and open the local URL shown in your terminal. You should see a 300 × 300 animated color gradient region.

By adjusting the speed parameter from 0 to 0.2, you can observe the transition from a static color layout into smooth fluid motion. This serves as a foundational exercise for rendering backgrounds with shaders; advanced features such as text masking, multi-block state scheduling, and camera motion blur must be implemented on top of this using Canvas and custom state logic.

You can visit the live First of Kind website to experience the full interactive piece, or explore additional effects in the official Paper Shaders documentation.