About this paper
LiveFoley improves streaming video-to-audio generation by using a trailing sliding window for visual alignment and expanded temporal denoising windows for audio quality with minimal latency.
Streaming video-to-audio (V2A) generation is essential for immersive interactive applications. However, real-time constraints leave existing streaming V2A methods far behind offline models: (1) visual frontends shrink their receptive fields to cut look-ahead latency, severely degrading audio-visual alignment; (2) causal autoregressive diffusion backends denoise each audio patch in isolation, impairing audio quality. To this end, we present LiveFoley, a streaming causal autoregressive diffusion architecture that addresses both bottlenecks with imperceptible latency. For the visual frontend, we introduce a Trailing Sliding Window that converts non-causal segment-level encoders into streaming…
streaming video-to-audiocausal autoregressive diffusionTrailing Sliding WindowUnified Varhop Frameworklocal diffusion transformerLocDiTHistory LocDiTFuture-Warmup LocDiT
- Upvotes
- 5
- Authors
- 9
- Published
- Nov 4, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
LiveFoley: Dynamic Real-Time Video-to-Audio Generation for Immersive Interactive Worlds was first seen on Hugging Face · Daily papers on Nov 6, 2026. Since then it has appeared in 13 checks on 3 different days. Its best position was #29 of 30, reached on Nov 6, 21:28 UTC. It is #29 right now, climbing from #30 to #29 over the last 6 checks.