AI Models
SANA-Video 2.0： Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
SANA-Video 2.0：混合线性注意力与注意力残差实现高效视频生成
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
arXiv.orgWe introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Open sourceRecommended because
This is worth tracking because it is a concrete model capability signal, not just a passing headline. The source preview points to a change in model capability, availability, benchmark behavior, or developer access. For builders and operators, "SANA-Video 2.0： Hybrid Linear Attention with Attention Residuals for Efficient Video Generation" can be used as a checkpoint for model selection, product roadmaps, eval planning, and timing decisions. I keep this thread indexed so future searches around AI model updates, capability shifts, and developer adoption can land on a source-linked page instead of disappearing into a fast-moving feed from arXiv.org.
What to take from this signal
Context
"SANA-Video 2.0： Hybrid Linear Attention with Attention Residuals for Efficient Video Generation" is archived here as a source-linked AI signal from arXiv.org. The useful part is the connection between SANA-Video, Hybrid, Linear, Attention, Residuals and model selection, product roadmaps, eval planning, and timing decisions, which makes the item more actionable than a normal feed headline. The source context says: We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Builder takeaway
For an AI builder, the main takeaway is to watch how this signal changes practical decisions around model quality, latency, cost, eval coverage, and release timing. It can inform what to test next, which product surface to compare, and whether the underlying workflow is ready for real users.
Source context
arXiv.org remains the authoritative source for the original claim. This page adds a stable archive URL, a short builder interpretation, and related search language so the item can be found later when the original feed has moved on.
Search angles
- SANA-Video 2.0： Hybrid Linear Attention with Attention Residuals for Efficient Video Generation AI Models context
- arXiv.org AI model releases
- SANA-Video, Hybrid, Linear, Attention, Residuals builder takeaway
- AI model updates, capability shifts, and developer adoption
This page keeps a source preview and a stable archive URL for search discovery. The original source remains authoritative.