skip to main content
10.1145/3503161.3547880acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation

Published: 10 October 2022 Publication History

Abstract

Learning-based methods have effectively promoted the community of image compression. Meanwhile, variational autoencoder(VAE) based variable-rate approaches have recently gained much attention to avoid the usage of a set of different networks for various compression rates. Despite the remarkable performance that has been achieved, these approaches would be readily corrupted once multiple compression/decompression operations are executed, resulting in the fact that image quality would be tremendously dropped and strong artifacts would appear. Thus, we try to tackle the issue of high-fidelity fine variable-rate image compression and propose the Invertible Activation Transformation(IAT) module. We implement the IAT in a mathematical invertible manner on a single rate Invertible Neural Network(INN) based model and the quality level(QLevel) would be fed into the IAT to generate scaling and bias tensors. IAT and QLevel together give the image compression model the ability of fine variable-rate control while better maintaining the image fidelity. Extensive experiments demonstrate that the single rate image compression model equipped with our IAT module has the ability to achieve variable-rate control without any compromise. And our IAT-embedded model obtains comparable rate-distortion performance with recent learning-based image compression methods. Furthermore, our method outperforms the state-of-the-art variable-rate image compression method by a large margin, especially after multiple re-encodings.

Supplementary Material

MP4 File (MM22_mmfp0539.mp4)
Presentation video

References

[1]
Eirikur Agustsson and Radu Timofte. 2017. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In CVPRW.
[2]
Eirikur Agustsson, Michael Tschannen, Fabian Mentzer, Radu Timofte, and Luc Van Gool. 2019. Generative Adversarial Networks for Extreme Learned Image Compression. In ICCV.
[3]
Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W Pellegrini, Ralf S Klessen, Lena Maier-Hein, Carsten Rother, and Ullrich Köthe. 2019. Analyzing Inverse Problems with Invertible Neural Networks. In ICLR.
[4]
Johannes Ballé, Valero Laparra, and Eero P Simoncelli. 2017. End-to-End Optimized Image Compression. In ICLR.
[5]
Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. 2018. Variational Image Compression with a Scale Hyperprior. In ICLR.
[6]
Jean Bégaint, Fabien Racapé, Simon Feltman, and Akshay Pushparaja. 2020. CompressAI: a Pytorch Library and Evaluation Platform for End-to-End Compression Research. arXiv preprint arXiv:2011.03029 (2020).
[7]
Fabrice Bellard. 2015. BPG Image Format. https://bellard.org/bpg/
[8]
Tong Chen, Haojie Liu, Zhan Ma, Qiu Shen, Xun Cao, and Yao Wang. 2021. End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling. IEEE Transactions on Image Processing, Vol. 30 (2021), 3179--3191.
[9]
Tong Chen and Zhan Ma. 2020. Variable Bitrate Image Compression with Quality Scaling Factors. In ICASSP.
[10]
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. 2020. Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules. In CVPR.
[11]
Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. 2019. Variable Rate Deep Image Compression with a Conditional Autoencoder. In ICCV.
[12]
Eastman Kodak Company. 1999. Kodak Lossless True Color Image Suite. http://r0k.us/graphics/kodak/
[13]
Ze Cui, Jing Wang, Shangyin Gao, Tiansheng Guo, Yihui Feng, and Bo Bai. 2021. Asymmetric Gained Deep Image Compression with Continuous Rate Adaptation. In CVPR.
[14]
Laurent Dinh, David Krueger, and Yoshua Bengio. 2015. NICE: Non-Linear Independent Components Estimation. In ICLRW.
[15]
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density Estimation Using Real NVP. In ICLR.
[16]
Jarek Duda. 2013. Asymmetric Numeral Systems: Entropy Coding Combining Speed of Huffman Coding with Compression Rate of Arithmetic Coding. arXiv preprint arXiv:1311.2540 (2013).
[17]
Google. 2010. Web Picture Format. https://chromium.googlesource.com/webm/libwebp
[18]
Zongyu Guo, Yaojun Wu, Runsen Feng, Zhizheng Zhang, and Zhibo Chen. 2020. 3-D Context Entropy Model for Improved Practical Image Compression. In CVPRW.
[19]
Leonhard Helminger, Abdelaziz Djelouah, Markus Gross, and Christopher Schroers. 2020. Lossy Image Compression with Normalizing Flows. In CoRR.
[20]
Yung-Han Ho, Chih-Chun Chan, Wen-Hsiao Peng, Hsueh-Ming Hang, and Marek Doma'nski. 2021. ANFIC: Image Compression Using Augmented Normalizing Flows. IEEE Open Journal of Circuits and Systems, Vol. 2 (2021), 613--626.
[21]
Yueyu Hu, Wenhan Yang, and Jiaying Liu. 2020. Coarse-to-Fine Hyper-Prior Modeling for Learned Image Compression. In AAAI.
[22]
Yueyu Hu, Wenhan Yang, Zhan Ma, and Jiaying Liu. 2021. Learning End-to-End Lossy Image Compression: A Benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
[23]
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR.
[24]
Shoma Iwai, Tomo Miyazaki, Yoshihiro Sugaya, and Shinichiro Omachi. 2021. Fidelity-Controllable Extreme Image Compression with Generative Adversarial Networks. In ICPR.
[25]
Nick Johnston, Damien Vincent, David Minnen, Michele Covell, Saurabh Singh, Troy Chinen, Sung Jin Hwang, Joel Shor, and George Toderici. 2018. Improved Lossy Image Compression With Priming and Spatially Adaptive Bit Rates for Recurrent Networks. In CVPR.
[26]
Joint Video Experts Team (JVET). 2021. VVC Official Test Model VTM. https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-12.1 accessed on April 5, 2021.
[27]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
[28]
Durk P Kingma and Prafulla Dhariwal. 2018. Glow: Generative Flow with Invertible 1x1 Convolutions. In NeurIPS.
[29]
Jooyoung Lee, Seunghyun Cho, and Seung-Kwon Beack. 2019. Context-Adaptive Entropy Model for End-to-End Optimized Image Compression. In ICLR.
[30]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In ECCV.
[31]
Jiaheng Liu, Guo Lu, Zhihao Hu, and Dong Xu. 2020. A Unified End-to-End Framework for Efficient Deep Image Compression. arXiv preprint arXiv:2002.03370 (2020).
[32]
Andreas Lugmayr, Martin Danelljan, Luc Van Gool, and Radu Timofte. 2020. SRFlow: Learning the Super-Resolution Space with Normalizing Flow. In ECCV.
[33]
Haichuan Ma, Dong Liu, Ning Yan, Houqiang Li, and Feng Wu. 2020. End-to-End Optimized Versatile Image Compression with Wavelet-Like Transform. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44 (2020), 1247--1263.
[34]
Yi Ma, Yongqi Zhai, Jiayu Yang, Chunhui Yang, and Ronggang Wang. 2021. AFEC: Adaptive Feature Extraction Modules for Learned Image Compression. In ACMMM.
[35]
Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. 2018. Conditional Probability Models for Deep Image Compression. In CVPR.
[36]
Fabian Mentzer, George D Toderici, Michael Tschannen, and Eirikur Agustsson. 2020. High-Fidelity Generative Image Compression. In NeurIPS.
[37]
David Minnen, Johannes Ballé, and George D Toderici. 2018. Joint Autoregressive and Hierarchical Priors for Learned Image Compression. In NeurIPS.
[38]
David Minnen and Saurabh Singh. 2020. Channel-Wise Autoregressive Entropy Models for Learned Image Compression. In ICIP.
[39]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS.
[40]
Yichen Qian, Ming Lin, Xiuyu Sun, Zhiyu Tan, and Rong Jin. 2022. Entroformer: A Transformer-Based Entropy Model for Learned Image Compression. In ICLR.
[41]
Majid Rabbani. 2002. JPEG2000: Image Compression Fundamentals, Standards and Practice. Journal of Electronic Imaging, Vol. 11, 2 (2002), 286.
[42]
Oren Rippel and Lubomir Bourdev. 2017. Real-Time Adaptive Image Compression. In ICML.
[43]
Myungseo Song, Jinyoung Choi, and Bohyung Han. 2021. Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform. In ICCV.
[44]
Zhenhong Sun, Zhiyu Tan, Xiuyu Sun, Fangyi Zhang, Yichen Qian, Dongyang Li, and Hao Li. 2021. Interpolation Variable Rate Image Compression. In ACMMM.
[45]
Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár. 2017. Lossy Image Compression with Compressive Autoencoders. In ICLR.
[46]
George Toderici, Sean M O'Malley, Sung Jin Hwang, Damien Vincent, David Minnen, Shumeet Baluja, Michele Covell, and Rahul Sukthankar. 2015. Variable Rate Image Compression with Recurrent Neural Networks. In ICLR.
[47]
George Toderici, Wenzhe Shi, Radu Timofte, Johannes Balle Lucas Theis, Eirikur Agustsson, Nick Johnston, and Fabian Mentzer. 2021. Workshop and Challenge on Learned Image Compression. http://www.compression.cc
[48]
George Toderici, Damien Vincent, Nick Johnston, Sung Jin Hwang, David Minnen, Joel Shor, and Michele Covell. 2017. Full Resolution Image Compression with Recurrent Neural Networks. In CVPR.
[49]
Gregory K Wallace. 1992. The JPEG Still Picture Compression Standard. IEEE Transactions On consumer Electronics, Vol. 38, 1 (1992), 18--34.
[50]
Dezhao Wang, Wenhan Yang, Yueyu Hu, and Jiaying Liu. 2022. Neural Data-Dependent Transform for Learned Image Compression. arXiv preprint arXiv:2203.04963 (2022).
[51]
Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. 2018. Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform. In CVPR.
[52]
Lirong Wu, Kejie Huang, and Haibin Shen. 2020. A Gan-Based Tunable Image Compression System. In WACV.
[53]
Mingqing Xiao, Shuxin Zheng, Chang Liu, Yaolong Wang, Di He, Guolin Ke, Jiang Bian, Zhouchen Lin, and Tie-Yan Liu. 2020. Invertible Image Rescaling. In ECCV.
[54]
Yueqi Xie, Ka Leong Cheng, and Qifeng Chen. 2021. Enhanced Invertible Encoding for Learned Image Compression. In ACMMM.
[55]
Fei Yang, Luis Herranz, Joost Van De Weijer, José A Iglesias Guitián, Antonio M López, and Mikhail G Mozerov. 2020. Variable Rate Deep Image Compression with Modulated Autoencoder. IEEE Signal Processing Letters, Vol. 27 (2020), 331--335.
[56]
Yulun Zhang, Kunpeng Li, Kai Li, Bineng Zhong, and Yun Fu. 2019. Residual Non-Local Attention Networks for Image Restoration. In ICLR.
[57]
Lei Zhou, Zhenhong Sun, Xiangji Wu, and Junmin Wu. 2019. End-to-End Optimized Image Compression with Attention Mechanism. In CVPRW.io

Cited By

View all

Index Terms

  1. High-Fidelity Variable-Rate Image Compression via Invertible Activation Transformation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '22: Proceedings of the 30th ACM International Conference on Multimedia
    October 2022
    7537 pages
    ISBN:9781450392037
    DOI:10.1145/3503161
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 October 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. fidelity maintenance
    2. image compression
    3. variable-rate

    Qualifiers

    • Research-article

    Funding Sources

    • the National Natural Science Foundation of China (NSFC)
    • the National Key Laboratory Foundation of China
    • the Special Project of Science and Technology Development of Central guiding LocalCentral Guidance on Local Science and Technology Development Fund of Hubei Province

    Conference

    MM '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)78
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media