实用学习型图像压缩的关键要素

2026-05-07 08:00·39天前

精选理由

Apple 这篇调研把感知质量和运行效率同时拉进实做框架，做 codec 或端侧推理的人值得认真读一下。

AI 摘要

学习型编解码器相比传统硬编码方法的显著优势在于能直接针对人类视觉系统进行优化，但目前尚未出现兼具感知质量与实用性的图像编解码方案。本研究通过全面分析关键建模选择，旨在填补这一空白，探索在感知质量与运行效率间的联合优化方案，并在消融实验中引入了若干新技术。研究进一步采用性能感知的神经架构优化方法，为构建真正实用化的学习型图像压缩系统提供了系统性的设计指南与实验基准。

原文 · 未翻译

What Matters in Practical Learned Image Compression

AuthorsKedar Tatwawadi, Parisa Rahimzadeh, Zhanghao Sun, Zhiqi Chen, Ziyun Yang, Sanjay Nair, Divija Hasteer, Oren Rippel

View publication

View source code (GitHub)

Copy Bibtex

One of the major differentiators unlocked by learned codecs relative to their hard-coded traditional counterparts is their ability to be optimized directly to appeal to the human visual system. Despite this potential, a perceptual yet practical image codec is yet to be proposed. In this work, we aim to close this gap. We conduct a comprehensive study of the key modeling choices that govern the design of a practical learned image codec, jointly optimized for perceptual quality and runtime — including within the ablations several novel techniques. We then perform performance-aware neural architecture search over millions of backbone configurations to identify models that achieve the target on-device runtime while maximizing compression performance as captured by perceptual metrics. We combine the various optimizations to construct a new codec that achieves a significantly improved tradeoff between speed and perceptual quality. Based on rigorous subjective user studies, it provides 2.3–3x bitrate savings against AV1, AV2, VVC, ECM and JPEG-AI, and 20–40% bitrate savings against the best learned codec alternatives. At the same time, on an iPhone 17 Pro Max, it encodes 12MP images as fast as 230ms, and decodes them in 150ms — faster than most top ML-based codecs run on a V100 GPU.