short-paper

Visual sketching: from image sketches to code

Authors:

Marcelo d'Amorim,

Carlos MelloAuthors Info & Claims

ICSE-NIER '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results

Pages 101 - 104

https://doi.org/10.1145/3377816.3381745

Published: 18 September 2020 Publication History

Abstract

Writing code is difficult and time consuming. This vision paper proposes Visual Sketching, a synthesis technique that produces code implementing the likely intent associated with an image. We describe potential applications of Visual Sketching, how to realize it, and implications of the technology.

References

[1]

[n.d.]. The Asimov Institute website. https://www.asimovinstitute.org.

[2]

[n.d.]. Brainscape. https://www.brainscape.com/subjects/computer-programming-flashcards.

[3]

[n.d.]. The Low-Code/No-Code Movement: More Disruptive Than You Realize. https://tinyurl.com/yynmkxu5.

[4]

[n.d.]. The Neural Net Zoo. https://www.asimovinstitute.org/neural-network-zoo/.

[5]

[n.d.]. Play Cards. Learns How to Code. http://codecards.io/.

[6]

[n.d.]. PROSE SDK. https://microsoft.github.io/prose/.

[7]

[n.d.]. Stateflow website. https://www.mathworks.com/products/stateflow.html.

[8]

[n.d.]. Your First Deep Learning Project in Python with Keras. https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/.

[9]

A. Baker, E. O. Navarro, and A. van der Hoek. 2003. Problems and Programmers: an educational software engineering card game. In ICSE. 614--619.

[10]

Carlos Alexandre Barros De Mello, Adriano Lorena Inacio de Oliveira, and Wellington Pinheiro Dos Santos. 2012. Digital document analysis and processing. Nova Science Publishers.

[11]

E.R. Dougherty. 1992. An introduction to morphological image processing. SPIE Optical Engineering Press.

[12]

Tim Furche, Georg Gottlob, Leonid Libkin, Giorgio Orsi, and Norman W. Paton. 2016. Data Wrangling for Big Data: Challenges and Opportunities. In Proceedings of the 19th International Conference on Extending Database Technology (EDBT). 473--478.

[13]

Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. 2017. Program Synthesis. Foundations and Trends in Programming Languages 4, 1-2 (2017), 1--119.

[14]

Nicholas R Howe. 2013. Document binarization with automatic parameter tuning. International Journal on Document Analysis and Recognition (IJDAR) 16, 3 (2013).

[15]

R Reeve Ingle, Yasuhisa Fujii, Thomas Deselaers, Jonathan Baccash, and Ashok C Popat. 2019. A Scalable Handwritten Text Recognition System. arXiv preprint arXiv:1904.09150 (2019).

[16]

Kapser, Cory. 2009. Toward an Understanding of Software Code Cloning as a Development Practice. http://hdl.handle.net/10012/4753

[17]

Shijian Lu, Bolan Su, and Chew Lim Tan. 2010. Document image binarization using background estimation and stroke edges. International Journal on Document Analysis and Recognition (IJDAR) 13, 4 (2010), 303--314.

Digital Library

[18]

I. Patrikakis, B. Gatos, and K. Ntirogiannis. [n.d.]. ICDAR 2013 Document Image Binarization Contest (DIBCO 2013). 2013 12th International Conference on Document Analysis and Recognition (2013).

[19]

Robert Sedgewick. 1900. Algorithms in C++, 3/e. Pearson Education India.

[20]

Armando Solar-Lezama. 2008. Program Synthesis by Sketching. Ph.D. Dissertation. Berkeley, CA, USA. Advisor(s) Bodik, Rastislav. AAI3353225.

Digital Library

[21]

C. Tensmeyer and C. Wigington. [n.d.]. Training Full-Page Handwritten Text Recognition Models without Annotated Line Breaks. ICDAR 2019. https://arxiv.org/abs/1909.02576.

[22]

Shane Torbert. 2016. Applied computer science. Springer.

[23]

S. Xiao, L. Peng, R. Yan, and Wang. S. [n.d.]. Deep Network with Pixel-Level Rectification and Robust Training for Handwriting Recognition. ICDAR 2019. https://arxiv.org/abs/1909.02576.

Cited By

Gomes LVasconcelos V(2023)Transforming Ideas into Code: Visual Sketching for ML DevelopmentCompanion Proceedings of the 2023 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity10.1145/3618305.3623588(10-12)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3618305.3623588
Costa DMello C(2023)Using YOLO Network for Automatic Processing of Finite Automata Images with Application to Bit-Strings RecognitionProceedings of the ACM Symposium on Document Engineering 202310.1145/3573128.3604898(1-9)Online publication date: 22-Aug-2023
https://dl.acm.org/doi/10.1145/3573128.3604898
Lai P(2023)Reimagining Online Coding Assessment Throughout the Pandemic: The Role of Auto-Grading and Enquiry Features2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)10.1109/TALE56641.2023.10398329(1-6)Online publication date: 28-Nov-2023
https://doi.org/10.1109/TALE56641.2023.10398329
Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE-NIER '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results

June 2020

128 pages

ISBN:9781450371261

DOI:10.1145/3377816

General Chairs:
Gregg Rothermel
North Carolina State University
,
Doo-Hwan Bae
KAIST, South Korea

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

KIISE: Korean Institute of Information Scientists and Engineers
IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

CAPES
Fundação para a Ciência e a Tecnologia (FCT)
CNPq
Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco

Conference

ICSE '20

Sponsor:

SIGSOFT

ICSE '20: 42nd International Conference on Software Engineering

June 27 - July 19, 2020

Seoul, South Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
90
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gomes LVasconcelos V(2023)Transforming Ideas into Code: Visual Sketching for ML DevelopmentCompanion Proceedings of the 2023 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity10.1145/3618305.3623588(10-12)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3618305.3623588
Costa DMello C(2023)Using YOLO Network for Automatic Processing of Finite Automata Images with Application to Bit-Strings RecognitionProceedings of the ACM Symposium on Document Engineering 202310.1145/3573128.3604898(1-9)Online publication date: 22-Aug-2023
https://dl.acm.org/doi/10.1145/3573128.3604898
Lai P(2023)Reimagining Online Coding Assessment Throughout the Pandemic: The Role of Auto-Grading and Enquiry Features2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)10.1109/TALE56641.2023.10398329(1-6)Online publication date: 28-Nov-2023
https://doi.org/10.1109/TALE56641.2023.10398329
Costa DMello Cd'Amorim MHealy PBilauca MBonnici A(2021)A comparative study on methods and tools for handwritten mathematical expression recognitionProceedings of the 21st ACM Symposium on Document Engineering10.1145/3469096.3474936(1-4)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3469096.3474936

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents