research-article

Open access

Megapixel adaptive optics: towards correcting large-scale distortions in computational cameras

Authors:

Wolfgang HeidrichAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 37, Issue 4

Article No.: 115, Pages 1 - 12

https://doi.org/10.1145/3197517.3201299

Published: 30 July 2018 Publication History

Abstract

Adaptive optics has become a valuable tool for correcting minor optical aberrations in applications such as astronomy and microscopy. However, due to the limited resolution of both the wavefront sensing and the wavefront correction hardware, it has so far not been feasible to use adaptive optics for correcting large-scale waveform deformations that occur naturally in regular photography and other imaging applications.

In this work, we demonstrate an adaptive optics system for regular cameras. We achieve a significant improvement in focus for large wavefront distortions by improving upon a recently developed high resolution coded wavefront sensor, and combining it with a spatial phase modulator to create a megapixel adaptive optics system with unprecedented capability to sense and correct large distortions.

Supplementary Material

ZIP File (115-206.zip)

Supplemental files.

Download
1.05 MB

MP4 File (115-206.mp4)

Download
344.41 MB

MP4 File (a115-wang.mp4)

Download
245.14 MB

References

[1]

Mariana SC Almeida and Mario Figueiredo. 2013. Deconvolving images with unknown boundaries using the alternating direction method of multipliers. IEEE Trans. Image Process 22, 8 (2013), 3074--3086.

[2]

Nicholas Antipa, Sylvia Necula, Ren Ng, and Laura Waller. 2016. Single-shot diffuser-encoded light field imaging. In Proc. ICCP. IEEE, 1--11.

[3]

Bradley Atcheson, Ivo Ihrke, Wolfgang Heidrich, Art Tevs, Derek Bradley, Marcus Magnor, and Hans-Peter Seidel. 2008. Time-resolved 3d capture of non-stationary gas flows. ACM Trans. Graph. 27, 5 (2008), 132.

Digital Library

[4]

Jacques M Beckers. 1988. Increasing the size of the isoplanatic patch with multiconjugate adaptive optics. In Proc. ESOCW, Vol. 30. 693.

[5]

Jacques M Beckers. 1993. Adaptive optics for astronomy: principles, performance, and applications. Annu. Rev. Astron. Astrophys. 31, 1 (1993), 13--62.

[6]

Pascal Berto, Hervé Rigneault, and Marc Guillon. 2017. Wavefront sensing with a thin diffuser. Opt. Lett. 42, 24 (2017), 5117--5120.

[7]

Sébastien Bérujon, Eric Ziegler, Roberto Cerbino, and Luca Peverini. 2012. Two-dimensional x-ray beam phase sensing. Phys. Rev. Lett. 108, 15 (2012), 158102.

[8]

Thomas G Bifano, Julie Perreault, R Krishnamoorthy Mali, and Mark N Horenstein. 1999. Microelectromechanical deformable mirrors. IEEE J. Select. Topics Quantum Electron 5, 1 (1999), 83--89.

[9]

Martin J Booth. 2014. Adaptive optical microscopy: the ongoing quest for a perfect image. Light Sci. Appl. 3, 4 (2014), e165.

[10]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3, 1 (2011), 1--122.

Digital Library

[11]

Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV. Springer, 25--36.

[12]

Sunghyun Cho and Seungyong Lee. 2009. Fast motion deblurring. ACM Trans. Graph. 28, 5 (2009), 145.

Digital Library

[13]

Sunghyun Cho, Jue Wang, and Seungyong Lee. 2011. Handling outliers in non-blind image deconvolution. In Proc. ICCV. IEEE, 495--502.

Digital Library

[14]

Oliver Cossairt and Shree Nayar. 2010. Spectral focal sweep: Extended depth of field from chromatic aberrations. In Proc. ICCP. IEEE, 1--8.

[15]

E Diolaiti, R Ragazzoni, and M Tordi. 2001. Closed loop performance of a layer-oriented multi-conjugate adaptive optics system. Astron. Astrophys 372, 2 (2001), 710--718.

[16]

Jiangxin Dong, Jinshan Pan, Zhixun Su, and Ming-Hsuan Yang. 2017. Blind Image Deblurring with Outlier Handling. In Proc. CVPR. 2478--2486.

[17]

Edward R Dowski and W Thomas Cathey. 1995. Extended depth of field through wave-front coding. Appl. Opt. 34, 11 (1995), 1859--1866.

[18]

Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T Roweis, and William T Freeman. 2006. Removing camera shake from a single photograph. ACM Trans. Graph. 25, 3 (2006), 787--794.

Digital Library

[19]

Enrique J Fernández, Ignacio Iglesias, and Pablo Artal. 2001. Closed-loop adaptive optics in the human eye. Opt. Lett. 26, 10 (2001), 746--748.

[20]

B Hermann, EJ Fernández, A Unterhuber, H Sattmann, AF Fercher, W Drexler, PM Prieto, and P Artal. 2004. Adaptive-optics ultrahigh-resolution optical coherence tomography. Opt. Lett. 29, 18 (2004), 2142--2144.

[21]

Berthold KP Horn and Brian G Schunck. 1981. Determining optical flow. Artificial intelligence 17, 1--3 (1981), 185--203.

Digital Library

[22]

Lifa Hu, Li Xuan, Yongjun Liu, Zhaoliang Cao, Dayu Li, and QuanQuan Mu. 2004. Phase-only liquid-crystal spatial light modulator for wave-front correction with high precision. Opt. Express 12, 26 (2004), 6403--6409.

[23]

Zhe Hu, Sunghyun Cho, Jue Wang, and Ming-Hsuan Yang. 2014. Deblurring low-light images with light streaks. In Proc. CVPR. 3382--3389.

Digital Library

[24]

Julian Iseringhausen, Bastian Goldlücke, Nina Pesheva, Stanimir Iliev, Alexander Wender, Martin Fuchs, and Matthias B Hullin. 2017. 4D imaging through spray-on optics. ACM Trans. Graph. 36, 4 (2017), 35.

Digital Library

[25]

Mooseokjang, Haowen Ruan, Haojiang Zhou, Benjamin Judkewitz, and Changhuei Yang. 2014. Method for auto-alignment of digital optical phase conjugation systems based on digital propagation. Opt. Express 22, 12 (2014), 14054--14071.

[26]

Hui Ji and Kang Wang. 2012. Robust image deblurring with an inaccurate blur kernel. IEEE Trans. Image Process. 21, 4 (2012), 1624--1634.

Digital Library

[27]

Na Ji, Daniel E Milkie, and Eric Betzig. 2010. Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues. Nat. Methods 7, 2 (2010), 141--147.

[28]

Dilip Krishnan, Terence Tay, and Rob Fergus. 2011. Blind deconvolution using a normalized sparsity measure. In Proc. CVPR. IEEE, 233--240.

Digital Library

[29]

RG Lane, A Glindemann, JC Dainty, et al. 1992. Simulation of a Kolmogorov phase screen. Waves in random media 2, 3 (1992), 209--224.

[30]

Anat Levin, Rob Fergus, Frédo Durand, and William T Freeman. 2007. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26, 3 (2007), 70.

Digital Library

[31]

Anat Levin, Samuel W Hasinoff, Paul Green, Frédo Durand, and William T Freeman. 2009. 4D frequency analysis of computational cameras for depth of field extension. ACM Trans. Graph. 28, 3 (2009), 97.

Digital Library

[32]

Anat Levin, Peter Sand, Taeg Sang Cho, Frédo Durand, and William T Freeman. 2008. Motion-invariant photography. ACM Trans. Graph. 27, 3 (2008), 71.

Digital Library

[33]

Chao Li, Mingliang Xia, Quanquan Mu, Baoguang Jiang, Li Xuan, and Zhaoliang Cao. 2009. High-precision open-loop adaptive optics system based on LC-SLM. Opt. Express 17, 13 (2009), 10774--10781.

[34]

Junzhong Liang, David R Williams, and Donald T Miller. 1997. Supernormal vision and high-resolution retinal imaging through adaptive optics. JOSA A 14, 11 (1997), 2884--2892.

[35]

Gordon D Love. 1997. Wave-front correction and production of Zernike modes with a liquid-crystal spatial light modulator. Appl. Opt. 36, 7 (1997), 1517--1524.

[36]

Enrico Marchetti, Roland Brast, Bernard Delabre, Robert Donaldson, Enrico Fedrigo, Christoph Frank, Norbert Hubin, Johann Kolb, Jean-Louis Lizon, Massimiliano Marchesi, et al. 2007. On-sky testing of the multi-conjugate adaptive optics demonstrator. The Messenger 129, 8 (2007).

[37]

PN Marsh, D Burns, and JM Girkin. 2003. Practical implementation of adaptive optics in multiphoton microscopy. Opt. Express 11, 10 (2003), 1123--1130.

[38]

Kshitij Marwah, Gordon Wetzstein, Yosuke Bando, and Ramesh Raskar. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Trans. Graph. 32, 4 (2013), 46.

Digital Library

[39]

Tomer Michaeli and Michal Irani. 2014. Blind deblurring using internal patch recurrence. In Proc. ECCV. Springer, 783--798.

[40]

Kaye S Morgan, David M Paganin, and Karen KW Siu. 2012. X-ray phase imaging with a paper analyzer. Appl. Phys. Lett. 100, 12 (2012), 124102.

[41]

Ren Ng. 2005. Fourier slice photography. ACM Trans. Graph. 24, 3 (2005), 735--744.

Digital Library

[42]

Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, and Pat Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. CSTR 2, 11 (2005), 1--11.

[43]

Jinshan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang. 2016. Blind image deblurring using dark channel prior. In Proc. CVPR. 1628--1636.

[44]

Julie A Perreault, Thomas G Bifano, B Martin Levine, and Mark N Horenstein. 2002. Adaptive optic correction using microelectromechanical deformable mirrors. Opt. Eng 41, 3 (2002), 561--566.

[45]

Jérôme Primot and Nicolas Guérineau. 2000. Extended Hartmann test based on the pseudoguiding property of a Hartmann mask completed by a phase chessboard. Appl. Opt. 39, 31 (2000), 5715--5720.

[46]

JetLSOGNO Primot and L Sogno. 1995. Achromatic three-wave (or more) lateral shearing interferometer. JOSA A 12, 12 (1995), 2679--2685.

[47]

Roberto Ragazzoni. 1996. Pupil plane wavefront sensing with an oscillating prism. J MOD OPTIC 43, 2 (1996), 289--293.

[48]

Ramesh Raskar, Amit Agrawal, and Jack Tumblin. 2006. Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans. Graph. 25, 3 (2006), 795--804.

Digital Library

[49]

Hugues Richard and Markus Raffel. 2001. Principle and Applications of the Background Oriented Schlieren (BOS) Method. Meas. Sci. and Techn. (2001), 1576--1585.

[50]

Francois Roddier. 1988. Curvature sensing and compensation: a new concept in adaptive optics. Appl. Opt. 27, 7 (1988), 1223--1225.

[51]

Austin Roorda, Fernando Romero-Borja, William J Donnelly III, Hope Queener, Thomas J Hebert, and Melanie CW Campbell. 2002. Adaptive optics scanning laser ophthalmoscopy. Opt. Express 10, 9 (2002), 405--412.

[52]

Leslie K Saddlemyer, Glen Herriot, Jean-Pierre Véran, and Murray Fletcher. 1998. Design aspects of the reconstructor for the Gemini Adaptive Optics System (Altair). 3353 (1998), 150--159.

[53]

Roland V Shack and BC Platt. 1971. Production and use of a lenticular Hartmann screen. JOSA A 61, 5 (1971), 656.

[54]

Deqing Sun, Stefan Roth, and Michael J Black. 2010. Secrets of optical flow estimation and their principles. In Proc. CVPR. IEEE, 2432--2439.

[55]

Libin Sun, Sunghyun Cho, Jue Wang, and James Hays. 2013. Edge-based blur kernel estimation using patch priors. In Proc. ICCP. IEEE, 1--8.

[56]

Michael Reed Teague. 1983. Deterministic phase retrieval: a Green's function solution. JOSA A 73, 11 (1983), 1434--1441.

[57]

Fernando Vargas-Martin, Pedro M Prieto, and Pablo Artal. 1998. Correction of the aberrations in the human eye with a liquid-crystal spatial light modulator: limits to performance. JOSA A 15, 9 (1998), 2552--2562.

[58]

Ashok Veeraraghavan, Ramesh Raskar, Amit Agrawal, Ankit Mohan, and Jack Tumblin. 2007. Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. ACM Trans. Graph. 26, 3 (2007), 69.

Digital Library

[59]

Laura Waller, Shan Shan Kou, Colin JR Sheppard, and George Barbastathis. 2010a. Phase from chromatic aberrations. Opt. Express 18, 22 (2010), 22817--22825.

[60]

Laura Waller, Lei Tian, and George Barbastathis. 2010b. Transport of intensity phase-amplitude imaging with higher order intensity derivatives. Opt. Express 18, 12 (2010), 12552--12561.

[61]

Congli Wang, Xiong Dun, Qiang Fu, and Wolfgang Heidrich. 2017. Ultra-high resolution coded wavefront sensor. Opt. Express 25, 12 (2017), 13736--13746.

[62]

Li Xu and Jiaya Jia. 2010. Two-phase kernel estimation for robust motion deblurring. In Proc. ECCV. Springer, 157--170.

Digital Library

[63]

Li Xu, Shicheng Zheng, and Jiaya Jia. 2013. Unnatural 10 sparse representation for natural image deblurring. In Proc. CVPR. IEEE, 1107--1114.

Digital Library

[64]

Zhimin Xu, Jun Ke, and Edmund Y Lam. 2012. High-resolution lightfield photography using two masks. Opt. Express 20, 10 (2012), 10971--10983.

[65]

Yan Zhang, Barry Cense, Jungtae Rha, Ravi S Jonnal, Weihua Gao, Robert J Zawadzki, John S Werner, Steve Jones, Scot Olivier, and Donald T Miller. 2006. High-speed volumetric imaging of cone photoreceptors with adaptive optics spectral-domain optical coherence tomography. Opt. Express 14, 10 (2006), 4380--4394.

[66]

Changyin Zhou and Shree Nayar. 2009. What are good apertures for defocus deblurring?. In Proc. ICCP. IEEE, 1--8.

[67]

Lijun Zhu, Pang-Chen Sun, Dirk-Uwe Bartsch, William R Freeman, and Yeshaiahu Fainman. 1999. Adaptive control of a micromachined continuous-membrane deformable mirror for aberration compensation. Appl. Opt. 38, 1 (1999), 168--176.

Cited By

Lao DWang CWong ASoatto S(2024)Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02372(25107-25116)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02372
Suo JZhang WGong JYuan XBrady DDai Q(2023)Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile VisionProceedings of the IEEE10.1109/JPROC.2023.3338272111:12(1607-1639)Online publication date: Dec-2023
https://doi.org/10.1109/JPROC.2023.3338272
Chu FGuo YZhang YWang DWang Q(2023)Accurate Phase Modulation Scheme in a Micropixel Liquid Crystal on Silicon with Multiple Polymer WallsACS Applied Optical Materials10.1021/acsaom.3c000992:7(1330-1337)Online publication date: 3-May-2023
https://doi.org/10.1021/acsaom.3c00099
Show More Cited By

Index Terms

Megapixel adaptive optics: towards correcting large-scale distortions in computational cameras
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Computational photography

Recommendations

Wavefront Reconstruction Methods for Adaptive Optics Systems on Ground-Based Telescopes

The earth's atmosphere is not a perfect media through which to view objects in outer-space; turbulence in the atmospheric temperature distribution results in refractive index variations that interfere with the propagation of light. As a result, ...
Adaptive optics based telescope arrays receiver for compensation of optical turbulence effects in a deep-space optical communication link with Mars
GLOBECOM'09: Proceedings of the 28th IEEE conference on Global telecommunications

Future deep-space exploration missions demand a broadband and integrated communication infrastructure to transport the scientific data to Earth. Optical communication employing optical arrays receiver is a viable alternative to the current NASA deep-...
Fiber Optics: Principles and Advanced Practices, Second Edition

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 37, Issue 4

August 2018

1670 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3197517

Issue’s Table of Contents

Copyright © 2018 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2018

Published in TOG Volume 37, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

King Abdullah University of Science and Technology

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
980
Total Downloads

Downloads (Last 12 months)226
Downloads (Last 6 weeks)49

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lao DWang CWong ASoatto S(2024)Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02372(25107-25116)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02372
Suo JZhang WGong JYuan XBrady DDai Q(2023)Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile VisionProceedings of the IEEE10.1109/JPROC.2023.3338272111:12(1607-1639)Online publication date: Dec-2023
https://doi.org/10.1109/JPROC.2023.3338272
Chu FGuo YZhang YWang DWang Q(2023)Accurate Phase Modulation Scheme in a Micropixel Liquid Crystal on Silicon with Multiple Polymer WallsACS Applied Optical Materials10.1021/acsaom.3c000992:7(1330-1337)Online publication date: 3-May-2023
https://doi.org/10.1021/acsaom.3c00099
Aragonda HShaik KJain HShah S(2022)Accurate and Real Time Assisted Cataloging in E-commerce using Dual Images5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD)10.1145/3493700.3493738(265-269)Online publication date: 8-Jan-2022
https://dl.acm.org/doi/10.1145/3493700.3493738
Zhang YHuang HPlaku EYu L(2021)Joint computational design of workspaces and workplansACM Transactions on Graphics10.1145/3478513.348050040:6(1-16)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480500
Itoh YLanglotz TZollmann SIwai DKiyoshi KAmano T(2021)Computational Phase-Modulated EyeglassesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.294703827:3(1916-1928)Online publication date: 1-Mar-2021
https://doi.org/10.1109/TVCG.2019.2947038
Wang CFu QDun XHeidrich W(2020)Modeling classical wavefront sensorsOptics Express10.1364/OE.38134928:4(5273)Online publication date: 11-Feb-2020
https://doi.org/10.1364/OE.381349
He WJiang ZZhang CSainju AGupta RLiu YTang JPrakash B(2020)CurvaNet: Geometric Deep Learning based on Directional Curvature for 3D Shape AnalysisProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403272(2214-2224)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1145/3394486.3403272
Shamir AMitra NUmetani NKoyama Y(2020)Intelligent tools for creative graphicsACM SIGGRAPH 2020 Courses10.1145/3388769.3407498(1-11)Online publication date: 17-Aug-2020
https://dl.acm.org/doi/10.1145/3388769.3407498
Zhang XBelfer RKry PVouga E(2020)C-Space tunnel discovery for puzzle path planningACM Transactions on Graphics10.1145/3386569.339246839:4(104:1-104:14)Online publication date: 17-Aug-2020
https://dl.acm.org/doi/10.1145/3386569.3392468
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents