LTX 2.3 Lip Sync Advanced · Image + Audio to Video

ABOUT THE WORKFLOW

Generate a Lip-Synced Video from Image and Audio
Upload a portrait and an audio recording. The workflow separates vocals from the audio, encodes them through the LTX Audio VAE, and conditions the video generation so the character's mouth movements match the speech. A static camera LoRA keeps the frame locked so the face stays centered throughout. The three-pass pipeline generates at 768x512 base, upscales x2, then upscales x1.5 to reach full 1080p. NAG and an IC LoRA Detailer sharpen fine facial detail. The output is a 10-second 1080p MP4 at 24fps with your audio embedded.

Model

• LTX Video 2.3 22B Dev by Lightricks. An open-source audio-visual video model with native audio encoding for lip-sync conditioning, paired with a Gemma 3 12B text encoder.

• Camera Control Static LoRA (strength 1.0). Locks the camera position so the face stays centered and stable during speech.

• IC LoRA Detailer (strength 0.75). Enhances facial detail, skin texture, and sharpness.

• LTX 2.3 Distilled LoRA (strength 0.5). Reduces step count while maintaining quality.

• LTX 2.3 Spatial Upscaler x2 and x1.5. Dedicated latent upscalers for the three-pass pipeline.

• MelBandRoFormer. Separates vocals from background noise in the uploaded audio before encoding.

HOW IT WORKS

Step 1. Upload your portrait
A clear, front-facing image with visible mouth, jaw, and facial features. This becomes the character in the video.
Works great with: headshots · portraits · AI-generated faces · character renders

Step 2. Upload your audio
A recording of speech, narration, or singing. The workflow separates vocals from background noise automatically and trims to 8 seconds.
Works great with: voiceovers · dialogue recordings · singing tracks · podcast clips

Step 3. Write your prompt
Describe the character, scene, and performance. "Cinematic anime mecha pilot speaking with focused determination, accurate lip sync, natural blinking and subtle head movement" tells the model what to generate around the lip-synced mouth.

Step 4. Hit run
Three passes generate and upscale the video. Audio encodes and conditions the lip-sync. The workflow produces a 1080p MP4 at 24fps with your audio.

Step 5. Download
The output is a 1920x1080 MP4 with your uploaded audio embedded.
Ready for: Premiere · DaVinci Resolve · After Effects · YouTube · TikTok · course platforms

First time? Upload a front-facing portrait and an audio clip. Edit the prompt to describe your character. Leave everything else as-is.

RECOMMENDED SETTINGS

Quick-start guide. Find the goal that matches yours and copy the settings.

• Standard lip-synced video — Default settings (three-pass, 241 frames, 24fps, NAG 11, all LoRAs active). Upload portrait and audio, write a prompt, run.

• Different audio segment — Adjust the TrimAudioDuration start and end values (default 0 to 8 seconds) to select a different section of your recording.

• Shorter clip — Reduce frame length from 241. At 24fps, 121 frames gives ~5 seconds. Shorter clips render faster.

• Stronger lip-sync — The audio conditioning drives the mouth movement. Cleaner audio with isolated speech produces tighter sync. MelBandRoFormer handles separation automatically, but starting with clean audio helps.

• Face looks distorted — Use a higher-quality, front-facing portrait with even lighting. The IC LoRA Detailer at 0.75 sharpens detail, but the source image quality sets the ceiling.

• Camera moves during speech — The Camera Control Static LoRA (strength 1.0) locks the camera. If drift occurs, check that the prompt does not include camera movement language like "push-in" or "orbit." Describe only the character's performance.

• More detail in the face — Raise the IC LoRA Detailer strength from 0.75 toward 1.0 for sharper skin, eye, and mouth detail.

Prompt: Describe the character and performance, not camera movement. "Anime pilot speaking with determination, accurate lip sync, natural blinking, cockpit lights flickering" is specific. "Person talking" gives the model nothing to work with. Include "accurate lip sync" in the prompt to reinforce the audio conditioning.

LEARN

📹 Videos

• Intro to Floyo

• ComfyUI 101 Free Course ft. Sebastian Kamph

• Floyo 101 for Team Collaboration

✨ Quick links

• Help docs

• LoRA Training guide

• Discover more workflows

USE CASES

🎤 Audio-Driven Talking Head Videos
Upload a voiceover and a portrait to produce a talking character clip at 1080p with precise lip-sync, natural blinking, and subtle head movement.

🎵 Singing and Music Performance
Upload a vocal track and a character image to generate a singing performance video with matched lip movement at broadcast quality.

🎓 Educational and Course Content
Generate a consistent talking instructor from a single portrait across full lectures and explainers without filming on camera.

🎬 Character Animation for Film
Animate illustrated or AI-generated characters with dialogue-driven lip-sync for animated shorts, trailers, and proof-of-concept reels.

WHAT WORKS BEST / WHAT TO AVOID

✅ Works great

• Front-facing portraits with visible mouth, jaw, and chin

• Clean audio with isolated speech or vocals

• Prompts that describe the character performance and include "accurate lip sync"

• Static camera framing (the LoRA is tuned for this)

⚠️ May produce softer results

• Profile or extreme-angle portraits where the mouth is partially hidden

• Audio with heavy background music, reverb, or overlapping voices

• Prompts with camera movement language that conflicts with the static camera LoRA

• Very long audio beyond the 8-second trim window (adjust the trim for different segments)

FAQ

How does LTX 2.3 lip-sync work?
The workflow separates vocals from the uploaded audio using MelBandRoFormer, encodes them through the LTX Audio VAE into latent space, and uses a latent noise mask to condition the video generation. The model reads the vocal features and generates corresponding mouth movements, timing, and jaw articulation. The Camera Control Static LoRA keeps the frame locked so the face stays centered during speech.

Does the lip-sync work with any language?
Yes. The audio encoding reads acoustic features from the waveform, not language-specific phonemes. Lip-sync works across languages including English, Mandarin, Japanese, Korean, Spanish, and others.

What is the three-pass upscaling pipeline?
Pass 1 generates at 768x512 base with 8 sampling steps. The LTX Spatial Upscaler x2 doubles the latent. Pass 2 refines at the upscaled resolution. The x1.5 upscaler then scales further. Pass 3 adds a final refinement. The result is ~1920x1080 video with detail preservation across all three stages.

What does the Camera Control Static LoRA do?
It locks the camera position so the face stays centered and stable throughout the clip. Without it, the model might add camera drift or movement that pulls the face off-center during speech. The LoRA is set to strength 1.0 for maximum stability.

Can I use this for text-to-video without a portrait?
Yes. Toggle "Turn it to true for T2V" from False to True. The workflow generates from the prompt alone, but lip-sync requires an audio input regardless of mode.

Is LTX Video 2.3 licensed for commercial use?
LTX Video 2.3 is an open-source model by Lightricks. The LoRAs and supporting models have their own license terms. Check each component's license on its model page for commercial use in your specific project.

How to run LTX 2.3 lip-sync video generation online?
You can run LTX 2.3 lip-sync video generation online through Floyo. No installation, no setup, no local GPU needed. Open the workflow in your browser, upload your inputs, and hit run. Free to try.

WHY FLOYO?

Floyo is the only platform with team collaboration for ComfyUI in the browser. You run workflows with no install. You share run history, assets, and models across your team. You pay only when you generate. Floyo supports open-source and closed-source models.

A designer runs an edit and likes the result. A teammate opens that exact run from shared history and keeps going. No file handoffs. No version confusion.

For studios and enterprise teams, Floyo adds private workspaces, pooled resources, and a team usage dashboard. Other ComfyUI cloud tools run for one person at a time. Floyo runs for the whole team, with transparent per-generation costs.

Ready to try it?
Upload a portrait and an audio clip, describe the character, and hit run.

→ Launch Workflow, Free

Questions? Watch the free course or check the FAQ above.