CoolerSpace: A Language for Physically Correct and Computationally Efficient Color Programming
Authors: 
Ethan Chen, Jiwon Chang, Yuhao ZhuAuthors Info & Claims
Ethan Chen, Jiwon Chang, Yuhao ZhuAuthors Info & ClaimsArticle No.: 301, Pages 846 - 875
Abstract
Color programmers manipulate lights, materials, and the resulting colors from light-material interactions. Existing libraries for color programming provide only a thin layer of abstraction around matrix operations. Color programs are, thus, vulnerable to bugs arising from mathematically permissible but physically meaningless matrix computations. Correct implementations are difficult to write and optimize. We introduce CoolerSpace to facilitate physically correct and computationally efficient color programming. CoolerSpace raises the level of abstraction of color programming by allowing programmers to focus on describing the logic of color physics. Correctness and efficiency are handled by CoolerSpace. The type system in CoolerSpace assigns physical meaning and dimensions to user-defined objects. The typing rules permit only legal computations informed by color physics and perception. Along with type checking, CoolerSpace also generates performance-optimized programs using equality saturation. CoolerSpace is implemented as a Python library and compiles to ONNX, a common intermediate representation for tensor computations. CoolerSpace not only prevents common errors in color programming, but also does so without run-time overhead: even unoptimized CoolerSpace programs out-perform existing Python-based color programming systems by up to 5.7 times; our optimizations provide up to an additional 1.4 times speed-up.
Supplemental Material
PDF File - CoolerSpace Supplementary Materials
Contains a comprehensive list of syntax and type rules for CoolerSpace. Also contains proofs of translational and type soundness, the equality saturation rules used in CoolerSpace, and a copy of the benchmarking programs.
- Download
- 7.24 MB
References
[1]
2011. Answer to ädding/mixing colors in HSV Space". https://stackoverflow.com/a/7388476
[2]
2014. Answer to "Interpolate from one color to another". https://stackoverflow.com/a/21010385
[3]
2016. Weird interpolation between colors in hsv? https://stackoverflow.com/q/37471461
[4]
2021. How to calculate (a physical) ratio of colors to achieve a target color? https://math.stackexchange.com/q/4335003
[5]
adriahf. 2016. Increase the velocity of the calculations. · Issue #302 · colour-science/colour. https://github.com/colour-science/colour/issues/302
[6]
Willow Ahrens, Fredrik Kjolstad, and Saman Amarasinghe. 2022. Autoscheduling for sparse tensor algebra with an asymptotic cost model. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation. 269–285. https://doi.org/10.1145/3519939.3523442
[7]
Eric Allen, David Chase, Victor Luchangco, Jan-Willem Maessen, and Guy L. Steele. 2004. Object-oriented units of measurement. In Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (Oopsla ’04). Association for Computing Machinery, 384–403. isbn:9781581138313 https://doi.org/10.1145/1028976.1029008
[8]
Simon Amarasingham. 2019. Red and green Eclectus Parrots. https://www.flickr.com/photos/22896868@N05/49257147352/
[9]
Luke Anderson, Andrew Adams, Karima Ma, Tzu-Mao Li, Tian Jin, and Jonathan Ragan-Kelley. 2021. Efficient automatic scheduling of imaging and vision pipelines for the GPU. Proceedings of the ACM on Programming Languages, 5, Oopsla (2021), 1–28. https://doi.org/10.1145/3485486
[10]
Roy S Berns. 2016. Color science and the visual arts: a guide for conservators, curators, and the curious. Getty Publications. https://doi.org/10.17613/d08z-ga34
[11]
G. Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools.
[12]
David H Brainard and Andrew Stockman. 2010. Colorimetry. The Optical Society of America Handbook of Optics, 3 (2010), 10–1.
[13]
Hans Brettel, Françoise Viénot, and John D Mollon. 1997. Computerized simulation of color appearance for dichromats. Josa a, 14, 10 (1997), 2647–2655. https://doi.org/10.1364/josaa.14.002647
[14]
Arthur D Broadbent. 2001. Basic principles of textile coloration. 132, Society of Dyers and Colorists Bradford.
[15]
Jiwon Chang and Ethan Chen. 2024. eggwrap. https://github.com/horizon-research/eggwrap
[16]
Ethan Chen. 2024. CoolerSpace. https://github.com/horizon-research/CoolerSpace
[17]
Ethan Chen. 2024. CoolerSpace Benchmarker. https://github.com/horizon-research/CoolerSpaceBenchmarker
[18]
Ethan Chen and Jiwon Chang. 2024. onneggs. https://github.com/horizon-research/onneggs
[19]
Ethan Chen, Jiwon Chang, and Yuhao Zhu. 2024. CoolerSpace Artifacts. https://doi.org/10.5281/zenodo.13621721
[20]
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, and Luis Ceze. 2018. $TVM$: An automated $End-to-End$ optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 578–594. https://doi.org/10.5555/3291168.3291211
[21]
Adam Chlipala, Leaf Petersen, and Robert Harper. 2005. Strict bidirectional type checking. In Proceedings of the 2005 ACM SIGPLAN International Workshop on Types in Languages Design and Implementation (Tldi ’05). Association for Computing Machinery, New York, NY, USA. 71–78. isbn:1581139993 https://doi.org/10.1145/1040294.1040301
[22]
Jeffrey Clark. 2023. Pillow. https://github.com/python-pillow/Pillow
[23]
Dimitrios Danopoulos, Christoforos Kachris, and Dimitrios Soudris. 2021. Utilizing cloud FPGAs towards the open neural network standard. Sustainable Computing: Informatics and Systems, 30 (2021), 100520. https://doi.org/10.1016/j.suscom.2021.100520
[24]
Colour Developers. 2015. Colour Science for Python. https://www.colour-science.org/
[25]
ONNX Runtime Developers. 2021. ONNX Runtime. https://onnxruntime.ai/ Version: 1.16
[26]
A Dreiheller, B Mohr, and M Moerschbacher. 1986. Programming pascal with physical units. ACM SIGPLAN Notices, 21, 12 (1986), Dec., 114–123. issn:0362-1340 https://doi.org/10.1145/15042.15048
[27]
DR Duncan. 1940. The colour of pigment mixtures. Proceedings of the Physical Society, 52, 3 (1940), 390.
[28]
Joshua Ebenezer. 2021. Computational speed for conversions · Issue #788 · colour-science/colour. https://github.com/colour-science/colour/issues/788
[29]
Mark D Fairchild and Lisa Reniff. 1995. Time course of chromatic adaptation for color-appearance judgments. Josa A, 12, 5 (1995), 824–833. https://doi.org/10.1016/s0042-6989(00)00050-x
[30]
Daniel J Fremont, Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Alberto L Sangiovanni-Vincentelli, and Sanjit A Seshia. 2019. Scenic: a language for scenario specification and scene generation. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 63–78. https://doi.org/10.1145/3314221.3314633
[31]
Dietrich Geisler, Irene Yoon, Aditi Kabra, Horace He, Yinnon Sanders, and Adrian Sampson. 2020. Geometry types for graphics programming. Proceedings of the ACM on Programming Languages, 4, Oopsla (2020), 1–25. https://doi.org/10.1145/3428241
[32]
Romain Guy. 2017. Understanding color. https://www.youtube.com/watch?v=r8NeG0wmFXM
[33]
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature, 585, 7825 (2020), Sept., 357–362. https://doi.org/10.1038/s41586-020-2649-2
[34]
Trish Hartmann. 2012. Eleuthera Sunset. https://openverse.org/image/d693e7e7-d7aa-4801-96c5-56674e5715c6
[35]
ST Henderson and D Hodgkiss. 1963. The spectral energy distribution of daylight. British Journal of Applied Physics, 14, 3 (1963), 125. https://doi.org/10.1088/0508-3443/15/8/310
[36]
Zhihao Jia, Oded Padon, James Thomas, Todd Warszawski, Matei Zaharia, and Alex Aiken. 2019. TASO: optimizing deep learning computation with automatic generation of graph substitutions. In Proceedings of the 27th ACM Symposium on Operating Systems Principles. 47–62. https://doi.org/0.1145/3341301.3359630
[37]
Tian Jin, Gheorghe-Teodor Bercea, Tung D Le, Tong Chen, Gong Su, Haruki Imai, Yasushi Negishi, Anh Leu, Kevin O’Brien, and Kiyokuni Kawachiya. 2020. Compiling onnx neural network models using mlir. arXiv preprint arXiv:2008.08272, https://doi.org/10.48550/arXiv.2008.08272
[38]
Ruth Johnston-Feller. 2001. Color science in the examination of museum objects: nondestructive procedures. Getty Publications. https://doi.org/10.1002/col.10107
[39]
Pramod G Joisha and Prithviraj Banerjee. 2006. An algebraic array shape inference system for MATLAB®. ACM Transactions on Programming Languages and Systems (TOPLAS), 28, 5 (2006), 848–907. https://doi.org/10.1145/1152649.1152651
[40]
Michael Karr and David B. Loveman. 1978. Incorporation of units into programming languages. 21, 5 (1978), 385–391. issn:0001-0782 https://doi.org/10.1145/359488.359501
[41]
Fredrik Kjolstad, Shoaib Kamil, Stephen Chou, David Lugato, and Saman Amarasinghe. 2017. The tensor algebra compiler. Proceedings of the ACM on Programming Languages, 1, Oopsla (2017), 1–29. https://doi.org/10.1145/3133901
[42]
Fredrik Kjolstad, Shoaib Kamil, Jonathan Ragan-Kelley, David IW Levin, Shinjiro Sueda, Desai Chen, Etienne Vouga, Danny M Kaufman, Gurtej Kanwar, and Wojciech Matusik. 2016. Simit: A language for physical simulation. ACM Transactions on Graphics (TOG), 35, 2 (2016), 1–21. https://doi.org/10.1145/2866569
[43]
Paul Kubelka. 1948. New contributions to the optics of intensely light-scattering materials. Part I. Josa, 38, 5 (1948), 448–457.
[44]
Paul Kubelka and Franz Munk. 1931. An article on optics of paint layers. Z. Tech. Phys, 12, 593-601 (1931), 259–274.
[45]
Daniël Lakens. 2017. Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social psychological and personality science, 8, 4 (2017), 355–362. https://doi.org/10.1177/1948550617697177
[46]
Siu Kwan Lam, Antoine Pitrou, and Stanley Seibert. 2015. Numba: a LLVM-based Python JIT compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC (Llvm ’15). Association for Computing Machinery, New York, NY, USA. Article 7, 6 pages. isbn:9781450340052 https://doi.org/10.1145/2833157.2833162
[47]
Dmitry Lavrov. 2020. Reading 65^ 3 Iridas 3D LUT is incredibly slow. · Issue #573 · colour-science/colour. https://github.com/colour-science/colour/issues/573
[48]
Changjun Li, Zhiqiang Li, Zhifeng Wang, Yang Xu, Ming Ronnier Luo, Guihua Cui, Manuel Melgosa, Michael H Brill, and Michael Pointer. 2017. Comprehensive color solutions: CAM16, CAT16, and CAM16-UCS. Color Research & Application, 42, 6 (2017), 703–718. https://doi.org/10.1002/col.22131
[49]
Bruce Lindbloom. 2017. XYZ to LAB. http://www.brucelindbloom.com/index.html?Eqn%5FXYZ%5Fto%5FLab.html
[50]
Amanda Liu, Gilbert Louis Bernstein, Adam Chlipala, and Jonathan Ragan-Kelley. 2022. Verified tensor-program optimization via high-level scheduling rewrites. Proceedings of the ACM on Programming Languages, 6, Popl (2022), 1–28. https://doi.org/10.1145/3498717
[51]
Steve Marschner and Peter Shirley. 2021. Chapter 14.6.1 Spectral Energy. In Fundamentals of Computer Graphics. AK Peters/CRC Press, 357–382. https://doi.org/10.1201/9781439865521
[52]
Francisco Massa. 2021. [feature request] rgb2lab / rgb2hsv / rgb2gray and other color space conversions (maybe upstream from kornia? or colorsys python core module?) · Issue #4029 · pytorch/vision. https://github.com/pytorch/vision/issues/4029
[53]
Don McCurdy. 2022. Color management. https://threejs.org/docs/##manual/en/introduction/Color-management
[54]
Michael E Miller and Spicer. 2019. Color in Electronic Display Systems. Springer. https://doi.org/10.1007/978-3-030-02834-3
[55]
Andrew C. Myers. 1999. JFlow: practical mostly-static information flow control. In Proceedings of the 26th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (Popl ’99). Association for Computing Machinery, New York, NY, USA. 228–241. isbn:1581130953 https://doi.org/10.1145/292540.292561
[56]
Ryosuke Okuta, Yuya Unno, Daisuke Nishino, Shohei Hido, and Crissman Loomis. 2017. CuPy: A NumPy-Compatible Library for NVIDIA GPU Calculations. In Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS).
[57]
Onnx. 2018. Open Neural Network Exchange. https://github.com/onnx/onnx
[58]
Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2023. Physically based rendering: From theory to implementation. MIT Press.
[59]
Charles Poynton. 2012. Digital video and HD: Algorithms and Interfaces. Elsevier. https://doi.org/10.1016/B978-0-12-391926-7.50059-X
[60]
Andy Rowlands. 2017. Physics of digital photography. IOP Publishing. https://doi.org/10.1088/978-0-7503-2558-5
[61]
D Andrew Rowlands. 2020. Color conversion matrices in digital cameras: a tutorial. Optical Engineering, 59, 11 (2020), 110801. https://doi.org/10.1117/1.oe.59.11.110801
[62]
A. Sabelfeld and A.C. Myers. 2003. Language-based information-flow security. IEEE Journal on Selected Areas in Communications, 21, 1 (2003), 5–19. https://doi.org/10.1109/jsac.2002.806121
[63]
Adrian Sampson, Werner Dietl, Emily Fortuna, Danushen Gnanapragasam, Luis Ceze, and Dan Grossman. 2011. EnerJ: approximate data types for safe and general low-power computation. SIGPLAN Not., 46, 6 (2011), jun, 164–174. issn:0362-1340 https://doi.org/10.1145/1993316.1993518
[64]
Gaurav Sharma. 2017. Color fundamentals for digital imaging. In Digital color imaging handbook. CRC press, 1–114.
[65]
Gaurav Sharma and Raja Bala. 2017. Digital color imaging handbook. CRC press. isbn:0-8493-0900-x
[66]
Justin Slepak, Olin Shivers, and Panagiotis Manolios. 2014. An array-oriented language with static rank polymorphism. In Programming Languages and Systems: 23rd European Symposium on Programming, ESOP 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014, Proceedings 23. 27–46. https://doi.org/10.1007/978-3-642-54833-8_3
[67]
Šárka Sochorová and Ondřej Jamriška. 2021. Practical pigment mixing for digital painting. ACM Transactions on Graphics (TOG), 40, 6 (2021), 1–11. https://doi.org/10.1145/3478513.3480549
[68]
van der Walt Stefan. 2017. Finding color space information about image · Issue #2175 · scikit-image/scikit-image. https://github.com/scikit-image/scikit-image/issues/2175
[69]
Andrew Stockman and David H Brainard. 2010. Color vision mechanisms. The Optical Society of America Handbook of Optics, 3 (2010), 11–1.
[70]
Adilla Susungi, Norman A Rink, Albert Cohen, Jeronimo Castrillon, and Claude Tadonki. 2018. Meta-programming for cross-domain tensor optimizations. ACM SIGPLAN Notices, 53, 9 (2018), 79–92. https://doi.org/10.1145/3278122.3278131
[71]
Ross Tate, Michael Stepp, Zachary Tatlock, and Sorin Lerner. 2009. Equality saturation: a new approach to optimization. In Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 264–276. https://doi.org/10.1145/1480881.1480915
[72]
Varkor. 2018. Types for units of measure. https://varkor.github.io/blog/2018/07/30/types-for-units-of-measure.html
[73]
Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018. Tensor comprehensions: Framework-agnostic high-performance machine learning abstractions. arXiv preprint arXiv:1802.04730, https://doi.org/10.48550/arXiv.1802.04730
[74]
Françoise Viénot, Hans Brettel, and John D Mollon. 1999. Digital video colourmaps for checking the legibility of displays by dichromats. Color Research & Application, 24, 4 (1999), 243–252. https://doi.org/10.1002/(SICI)1520-6378(199908)24:4<243::AID-COL5>3.0.CO;2-3
[75]
Brian A Wandell. 1995. Foundations of vision. sinauer Associates.
[76]
Max Willsey, Chandrakana Nandi, Yisu Remy Wang, Oliver Flatt, Zachary Tatlock, and Pavel Panchekha. 2021. Egg: Fast and extensible equality saturation. Proceedings of the ACM on Programming Languages, 5, Popl (2021), 1–29. https://doi.org/10.1145/3434304
[77]
Günther Wyszecki and Walter Stanley Stiles. 2000. Color science: concepts and methods, quantitative data and formulae. 40, John wiley & sons.
[78]
Yichen Yang, Phitchaya Phothilimthana, Yisu Wang, Max Willsey, Sudip Roy, and Jacques Pienaar. 2021. Equality saturation for tensor graph superoptimization. Proceedings of Machine Learning and Systems, 3 (2021), 255–268.
Index Terms
- CoolerSpace: A Language for Physically Correct and Computationally Efficient Color Programming
Recommendations
Precision requirements for digital color reproduction
An environment was established to perform device-independent color reproduction of full-color pictorial images. In order to determine the required precision for this environment, an experiment was performed to psychophysically measure colorimetric ...
Perceptual color spaces for computer graphics
Perceptually uniform color spaces can be a useful tool for solving computer graphics color selection problems. However, before they can be used effectively some basic principles of tristimulus colorimetry must be understood and the color reproduction ...
Perceptual color spaces for computer graphics
SIGGRAPH '80: Proceedings of the 7th annual conference on Computer graphics and interactive techniquesPerceptually uniform color spaces can be a useful tool for solving computer graphics color selection problems. However, before they can be used effectively some basic principles of tristimulus colorimetry must be understood and the color reproduction ...
Comments
Information & Contributors
Information
Published In
October 2024
2691 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 October 2024
Published in PACMPL Volume 8, Issue OOPSLA2
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 219Total Downloads
- Downloads (Last 12 months)219
- Downloads (Last 6 weeks)88
Reflects downloads up to 29 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in