Cited By
View all- Ge JZhou MBao WXu HFu C(2024)Creating LEGO Figurines from Single ImagesACM Transactions on Graphics10.1145/365816743:4(1-16)Online publication date: 19-Jul-2024
Computational Design of LEGO® Sketch Art
Authors: 
Mingjun Zhou, Jiahao Ge, Hao Xu, Chi-Wing FuAuthors Info & Claims
Mingjun Zhou, Jiahao Ge, Hao Xu, Chi-Wing FuAuthors Info & ClaimsArticle No.: 201, Pages 1 - 15
Abstract
This paper presents computational methods to aid the creation of LEGO®1 sketch models from simple input images. Beyond conventional LEGO® mosaics, we aim to improve the expressiveness of LEGO® models by utilizing LEGO® tiles with sloping and rounding edges, together with rectangular bricks, to reproduce smooth curves and sharp features in the input. This is a challenging task, as we have limited brick shapes to use and limited space to place bricks. Also, the search space is immense and combinatorial in nature. We approach the task by decoupling the LEGO® construction into two steps: first approximate the shape with a LEGO®-buildable contour then filling the contour polygon with LEGO® bricks. Further, we formulate this contour approximation into a graph optimization with our objective and constraints and effectively solve for the contour polygon that best approximates the input shape. Further, we extend our optimization model to handle multi-color and multi-layer regions, and formulate a grid alignment process and various perceptual constraints to refine the results. We employ our method to create a large variety of LEGO® models and compare it with humans and baseline methods to manifest its compelling quality and speed.
Supplementary Material
References
[1]
2023. BrickLink - Studio. https://www.bricklink.com/v2/build/studio.page
[2]
Nafiz Arica and Fatos T. Yarman Vural. 2003. BAS: A Perceptual Shape Descriptor Based on the Beam Angle Statistics. Pattern Recognition Letters 24, 9 (2003), 1627--1639.
[3]
Ilaruo Asada and Michael Brady. 1986. The Curvature Primal Sketch. IEEE Transactions on Pattern Analysis and Machine Intelligence 8 (1986), 2--14.
[4]
Xiang Bai, Cong Rao, and Xinggang Wang. 2014. Shape Vocabulary: A Robust and Efficient Shape Representation for Shape Matching. IEEE Transactions on Image Processing 23, 9 (2014), 3935--3949.
[5]
Serge Belongie, Jitendra Malik, and Jan Puzicha. 2002. Shape Matching and Object Recognition Using Shape Contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 24 (2002), 14.
[6]
Gary Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal: Software Tools for the Professional Programmer 25, 11 (2000).
[7]
Weikai Chen, Yuexin Ma, Sylvain Lefebvre, Shiqing Xin, Jonàs Martínez, and wenping wang. 2017. Fabricable Tile Decors. ACM Transactions on Graphics (SIGGRAPH Asia) 36, 6 (2017), 175:1--175:15.
[8]
Xuelin Chen, Honghua Li, Chi-Wing Fu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2018. 3D Fabrication with Universal Building Blocks and Pyramidal Shells. ACM Transactions on Graphics (SIGGRAPH Asia) 37, 6 (2018), 189:1--189:15.
[9]
Gene C.-H. Chuang and C.-C. Jay Kuo. 1996. Wavelet Descriptor of Planar Curves: Theory and Applications. IEEE Transactions on Image Processing 5, 1 (1996), 56--70.
[10]
Mario Deuss, Daniele Panozzo, Emily Whiting, Yang Liu, Philippe Block, Olga Sorkine-Hornung, and Mark Pauly. 2014. Assembling Self-Supporting Structures. ACM Transactions on Graphics (SIGGRAPH Asia) 33, 6 (2014), 214:1--214:10.
[11]
Michael Eigensatz, Martin Kilian, Alexander Schiftner, Niloy J. Mitra, Helmut Pottmann, and Mark Pauly. 2010. Paneling Architectural Freeform Surfaces. ACM Transactions on Graphics (SIGGRAPH) 29, 4 (2010), 45:1--45:10.
[12]
Chi-Wing Fu, Chi-Fu Lai, Ying He, and Daniel Cohen-Or. 2010. K-Set Tilable Surfaces. ACM Transactions on Graphics (SIGGRAPH) 29, 4 (2010), 44:1--44:6.
[13]
Jiasi Gao, Jiangtao Gong, Guyue Zhou, Haole Guo, and Tong Qi. 2022. Learning with Yourself: A Tangible Twin Robot System to Promote STEM Education. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 4981--4988.
[14]
Konstantinos Gavriil, Ruslan Guseinov, Jesús Pérez, Davide Pellis, Paul Henderson, Florian Rist, Helmut Pottmann, and Bernd Bickel. 2020. Computational Design of Cold Bent Glass Facades. ACM Transactions on Graphics (SIGGRAPH Asia) 39, 6 (2020), 208:1--208:16.
[15]
Sean Gillies, Casper van der Wel, Joris Van den Bossche, Mike W. Taves, Joshua Arnott, Brendan C. Ward, et al. 2023. Shapely. https://github.com/shapely/shapely
[16]
Rebecca A. H. Gower, Agnes E. Heydtmann, and Henrik G. Petersen. 1998. LEGO: Automated Model Construction. 32nd European Study Group with Industry - Final Report (1998), 81--94.
[17]
Branko Grünbaum and G. C. (Geoffrey Colin) Shephard. 1987. Tilings and Patterns. W.H.Freeman and Company.
[18]
Gurobi Optimization, LLC. 2023. Gurobi Optimizer Reference Manual. https://www.gurobi.com
[19]
Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature 585, 7825 (2020), 357--362.
[20]
Alejo Hausner. 2001. Simulating Decorative Mosaics. In Proceedings of SIGGRAPH. 573--580.
[21]
Jhen-Yao Hong, Der-Lor Way, Zen-Chung Shih, Wen-Kai Tai, and Chin-Chen Chang. 2016. Inner Engraving for the Creation of a Balanced LEGO Sculpture. The Visual Computer 32, 5 (2016), 569--578.
[22]
Julian Iseringhausen, Michael Weinmann, Weizhen Huang, and Matthias B. Hullin. 2020. Computational Parquetry: Fabricated Style Transfer with Wood Pixels. ACM Transactions on Graphics (SIGGRAPH) 39, 2 (2020), 12:1--12:14.
[23]
Caigui Jiang, Hui Wang, Victor Ceballos Inza, Felix Dellinger, Florian Rist, Johannes Wallner, and Helmut Pottmann. 2021. Using Isometries for Computational Design and Fabrication. ACM Transactions on Graphics (SIGGRAPH) 40, 4 (2021), 42:1--42:12.
[24]
Craig S. Kaplan and David H. Salesin. 2000. Escherization. In Proceedings of SIGGRAPH. 499--510.
[25]
Jae Woo Kim, Kyung Kyu Kang, and Ji Hyoung Lee. 2014. Survey on Automated LEGO Assembly Construction. In Proc. WSCG. 89--96.
[26]
Mikiya Kohama, Chiharu Sugimoto, Ojiro Nakano, and Yusuke Maeda. 2021. Robotic Additive Manufacturing with Toy Blocks. IISE Transactions 53, 3 (2021), 273--284.
[27]
Torkil Kollsker. 2020. Mathematical Models and Algorithms for Optimisation of the LEGO Construction Problem. Ph.D. Dissertation. Technical University of Denmark.
[28]
Torkil Kollsker and Enrico Malaguti. 2021. Models and Algorithms for Optimising Two-Dimensional LEGO Constructions. European Journal of Operational Research 289, 1 (2021), 270--284.
[29]
Torkil Kollsker and Thomas J. R. Stidsen. 2021. Optimisation and Static Equilibrium of Three-Dimensional LEGO Constructions. Operations Research Forum 2, 21 (2021).
[30]
Ming-Hsun Kuo, You-En Lin, Hung-Kuo Chu, Ruen-Rone Lee, and Yong-Liang Yang. 2015. Pixel2Brick: Constructing Brick Sculptures from Pixel Art. Computer Graphics Forum 34, 7 (2015), 339--348.
[31]
Kin Chung Kwan, Lok Tsun Sinn, Chu Han, Tien-Tsin Wong, and Chi-Wing Fu. 2016. Pyramid of Arclength Descriptor for Generating Collage of Shapes. ACM Transactions on Graphics (SIGGRAPH Asia) 35, 6 (2016), 229:1--229:12.
[32]
Sangyeop Lee, Jinhyun Kim, Jae Woo Kim, and Byung-Ro Moon. 2015. Finding an Optimal LEGO Brick Layout of Voxelized 3D Object Using a Genetic Algorithm. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (GECCO '15). 1215--1222.
[33]
Zhenyuan Liu, Jingyu Hu, Hao Xu, Peng Song, Ran Zhang, Bernd Bickel, and Chi-Wing Fu. 2022. Worst-Case Rigidity Analysis and Optimization for Assemblies with Mechanical Joints. Computer Graphics Forum 41, 2 (2022), 507--519.
[34]
Zhong-Yuan Liu, Zhan Zhang, Di Zhang, Chunyang Ye, Ligang Liu, and Xiao-Ming Fu. 2021. Modeling and Fabrication with Specified Discrete Equivalence Classes. ACM Transactions on Graphics (SIGGRAPH) 40, 4 (2021), 41:1--41:12.
[35]
Sheng-Jie Luo, Yonghao Yue, Chun-Kai Huang, Yu-Huan Chung, Sei Imai, Tomoyuki Nishita, and Bing-Yu Chen. 2015. Legolization: Optimizing LEGO Designs. ACM Transactions on Graphics (SIGGRAPH Asia) 34, 6 (2015), 222:1--222:12.
[36]
Brick me. 2022. Brick.me. https://brick.me/ [Online; accessed 18-May-2023].
[37]
Yuichi Nagata and Shinji Imahori. 2021. Escherization with Large Deformations Based on As-Rigid-As-Possible Shape Modeling. ACM Transactions on Graphics (SIGGRAPH) 41, 2 (2021), 11:1--11:16.
[38]
Chi-Han Peng, Caigui Jiang, Peter Wonka, and Helmut Pottmann. 2019. Checkerboard Patterns with Black Rectangles. ACM Transactions on Graphics (SIGGRAPH Asia) 38, 6 (2019), 171:1--171:13.
[39]
Eric Persoon and King-Sun Fu. 1977. Shape Discrimination Using Fourier Descriptors. IEEE Transactions on Systems, Man, and Cybernetics 7, 3 (1977), 170--179.
[40]
Maxim Peysakhov, Vlada Galinskaya, and William C. Regli. 2000. Representation and Evolution of Lego-Based Assemblies. In Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence. 1089.
[41]
Eugene Smal. 2008. Automated Brick Sculpture Construction. Thesis. Stellenbosch : Stellenbosch University.
[42]
Kaleigh Smith, Yunjun Liu, and Allison Klein. 2005. Animosaics. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '05). 201--208.
[43]
Alexander Toshev, Ben Taskar, and Kostas Daniilidis. 2012. Shape-Based Object Detection via Boundary Structure. International Journal of Computer Vision 99 (2012), 123--146.
[44]
Ruocheng Wang, Yunzhi Zhang, Jiayuan Mao, Chin-Yi Cheng, and Jiajun Wu. 2022. Translating a Visual LEGO Manual to a Machine-Executable Plan. In Computer Vision - ECCV 2022 (Lecture Notes in Computer Science). 677--694.
[45]
Xplicator. 2020. Mosaic Art. https://brickmosaicdesigner.com/ [Online; accessed 18-May-2023].
[46]
Hao Xu, Ka-Hei Hui, Chi-Wing Fu, and Hao Zhang. 2019. Computational LEGO Technic Design. ACM Transactions on Graphics (SIGGRAPH Asia) 38, 6 (2019), 196.
[47]
Hao Xu, Ka-Hei Hui, Chi-Wing Fu, and Hao Zhang. 2020. TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network. ACM Transactions on Graphics (SIGGRAPH) 39, 4 (2020), 129:1--129:16.
[48]
Xuemiao Xu, Linling Zhang, and Tien-Tsin Wong. 2010. Structure-Based ASCII Art. In Proceedings of SIGGRAPH. 52:1--52:10.
[49]
Grim Yun, Cheolseong Park, Heekyung Yang, and Kyungha Min. 2017. Legorization with Multi-Height Bricks from Silhouette-Fitted Voxelization. In Proceedings of the Computer Graphics International Conference (CGI '17). 40:1--40:6.
[50]
Jie Zhou, Xuejin Chen, and Ying-qing Xu. 2019. Automatic Generation of Vivid LEGO Architectural Sculptures. Computer Graphics Forum 38, 6 (2019), 31--42.
Index Terms
- Computational Design of LEGO® Sketch Art
Recommendations
Creating LEGO Figurines from Single Images
This paper presents a computational pipeline for creating personalized, physical LEGO®1 figurines from user-input portrait photos. The generated figurine is an assembly of coherently-connected LEGO® bricks detailed with uv-printed decals, capturing ...
LEGO® for Professional Development: A Systematic Literature Review
HCI in GamesAbstractThe purpose of this systematic literature review is to map how LEGO® is currently utilised within formal and non-formal education to support professional development of adults. The review process encapsulates a wide range of evidence related to ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 December 2023
Published in TOG Volume 42, Issue 6
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- the Research Grants Council of the Hong Kong Special Administrative Region
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 682Total Downloads
- Downloads (Last 12 months)682
- Downloads (Last 6 weeks)84
Reflects downloads up to 01 Nov 2024
Other Metrics
Citations
Cited By
View all- Ge JZhou MBao WXu HFu C(2024)Creating LEGO Figurines from Single ImagesACM Transactions on Graphics10.1145/365816743:4(1-16)Online publication date: 19-Jul-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in