short-paper

A comparative study on methods and tools for handwritten mathematical expression recognition

Authors:

Daniela S. Costa,

Carlos A. B. Mello,

Marcelo d'AmorimAuthors Info & Claims

DocEng '21: Proceedings of the 21st ACM Symposium on Document Engineering

Article No.: 26, Pages 1 - 4

https://doi.org/10.1145/3469096.3474936

Published: 16 August 2021 Publication History

Get Access

Abstract

Handwritten mathematical expression recognition (HMER) is a challenging task due to factors such as ambiguity, variety of writing styles, and complexity of two-dimensional writing. In this paper, we identify challenges in HMER applications through experiments that simulate real scenarios that go far beyond the usual cases found in literature: variations on luminance; different stroke width, inclination and color; different background pattern; and partially shaded images. The results of state-of-the-art methods (as TAP and Dense-WAP) and a commercial tool (MathPix) are analyzed, using the CROHME 2016 database. We proved that, although the area has had a lot of improvement in recent years, there are still issues to overcome.

References

[1]

Aguilar, F.D.J.J., and Hirata, N.S.T. ExpressMatch: A System for Creating Ground-Truthed Datasets of Online Mathematical Expressions. In: DAS 2012, pp.155--159 (2012)

Google Scholar

[2]

D'Amorim, M., Abreu, R., and Mello, C.A.B. Visual sketching. In: ICSE '20: 42nd International Conference on Software Engineering, pp.101--104 (2020)

Digital Library

Google Scholar

[3]

Chan, C. Stroke extraction for offline handwritten mathematical expression recognition. In: IEEE Access, v. 8, pp.61565--61575 (2020)

Crossref

Google Scholar

[4]

Hong, Z., You, N., Tan, J., and Bi, N. Residual BiRNN based Seq2Seq Model with Transition Probability Matrix for Online Handwritten Mathematical Expression Recognition. In: ICDAR, pp.635--640 (2019)

Crossref

Google Scholar

[5]

Labahn, G., Lank, E., MacLean, S., Marzouk, M., and Tausky, D. MathBrush: a system for doing math on pen-based devices. In: DAS 2008 (2008)

Google Scholar

[6]

Mouchère, H. et al. ICFHR 2014 competition on recognition of on-line handwritten mathematical expressions. In: 2014 ICFHR. IEEE, pp.791--796. (2014)

Google Scholar

[7]

Mouchère, H., Viard-Gaudin, C., Zanibbi, R., and Garain, U. ICFHR2016 CROHME: Competition on Recognition of Online Handwritten Mathematical Expressions. In: 15th ICFHR, pp.607--612, Shenzhen (2016)

Google Scholar

[8]

Zhang, T., Mouchere, H., and Viard-Gaudin, C. Tree-based BLSTM for mathematical expression recognition. In: ICDAR, pp.914--919 (2017)

Crossref

Google Scholar

[9]

Zhang, J., et al. Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition. In: Pattern Recognition 71, pp.196--206 (2017)

Crossref

Google Scholar

[10]

Zhang, J., Du, J., and Dai, L. Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition. In: ICPR, pp.2245--2250 (2018)

Crossref

Google Scholar

[11]

Zhang, J., Du, J., and Dai, L. Track, Attend, and Parse (TAP): An End-to-End Framework for Online Handwritten Mathematical Expression Recognition. In: IEEE Transactions on Multimedia 21(1), pp.221--233 (2019)

Digital Library

Google Scholar

[12]

MathPix Snip: https://mathpix.com/

Google Scholar

[13]

Wolfram Alpha: https://rhttps://www.wolframalpha.com/

Google Scholar

Cited By

View all

Pitts AFowler C(2024)Comparison of open-source software for producing directed acyclic graphsJournal of Causal Inference10.1515/jci-2023-003112:1Online publication date: 10-Jan-2024
https://doi.org/10.1515/jci-2023-0031
Kwon BKim JLee HSuh HMun D(2024)Construction of design requirements knowledgebase from unstructured design guidelines using natural language processingComputers in Industry10.1016/j.compind.2024.104100159-160(104100)Online publication date: Aug-2024
https://doi.org/10.1016/j.compind.2024.104100
Costa DMello C(2023)Using YOLO Network for Automatic Processing of Finite Automata Images with Application to Bit-Strings RecognitionProceedings of the ACM Symposium on Document Engineering 202310.1145/3573128.3604898(1-9)Online publication date: 22-Aug-2023
https://dl.acm.org/doi/10.1145/3573128.3604898
Show More Cited By

Index Terms

A comparative study on methods and tools for handwritten mathematical expression recognition
1. Computing methodologies

Recommendations

Handwritten Mathematical Expression Recognition: A Survey
Pattern Recognition and Artificial Intelligence
Abstract
While the handwritten character recognition has reached a point of maturity, the recognition of handwritten mathematics is still a challenging problem. The problem usually consists of three major parts: strokes segmentation, single symbol ...
Online Recognition System for Handwritten ARABIC Mathematical Symbols
ACSAT '13: Proceedings of the 2013 International Conference on Advanced Computer Science Applications and Technologies

Same as Arabic scripts, Arabic mathematical symbols have some differences with Latin mathematical symbols shape. Although, Arabic Mathematical symbols follow Latin mathematical from the structure of writing the symbols, Arabic symbols have special shape ...
Stroke constrained attention network for online handwritten mathematical expression recognition
Highlights
- A novel stroke constrained attention network for online HMER and online HCCR is proposed.
Abstract
In this paper, we propose a novel stroke constrained attention network (SCAN) which treats stroke as the basic unit for encoder-decoder based online handwritten mathematical expression recognition (HMER). Unlike previous methods which ...

Comments

Information & Contributors

Information

Published In

DocEng '21: Proceedings of the 21st ACM Symposium on Document Engineering

August 2021

178 pages

ISBN:9781450385961

DOI:10.1145/3469096

General Chairs:
Patrick Healy
University of Limerick, Ireland
,
Mihai Bilauca
University of Limerick, Ireland
,
Program Chair:
Alexandra Bonnici
University of Malta, Malta

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco - FACEPE

Conference

DocEng '21

Sponsor:

SIGWEB

DocEng '21: ACM Symposium on Document Engineering 2021

August 24 - 27, 2021

Limerick, Ireland

Acceptance Rates

Overall Acceptance Rate 194 of 564 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
136
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)6

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pitts AFowler C(2024)Comparison of open-source software for producing directed acyclic graphsJournal of Causal Inference10.1515/jci-2023-003112:1Online publication date: 10-Jan-2024
https://doi.org/10.1515/jci-2023-0031
Kwon BKim JLee HSuh HMun D(2024)Construction of design requirements knowledgebase from unstructured design guidelines using natural language processingComputers in Industry10.1016/j.compind.2024.104100159-160(104100)Online publication date: Aug-2024
https://doi.org/10.1016/j.compind.2024.104100
Costa DMello C(2023)Using YOLO Network for Automatic Processing of Finite Automata Images with Application to Bit-Strings RecognitionProceedings of the ACM Symposium on Document Engineering 202310.1145/3573128.3604898(1-9)Online publication date: 22-Aug-2023
https://dl.acm.org/doi/10.1145/3573128.3604898
Zhang XZhou ZMing CSun Y(2023)GPT-Assisted Learning of Structure–Property Relationships by Graph Neural Networks: Application to Rare-Earth-Doped PhosphorsThe Journal of Physical Chemistry Letters10.1021/acs.jpclett.3c0284814:50(11342-11349)Online publication date: 8-Dec-2023
https://doi.org/10.1021/acs.jpclett.3c02848

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Handwritten Mathematical Expression Recognition: A Survey

Online Recognition System for Handwritten ARABIC Mathematical Symbols

Stroke constrained attention network for online handwritten mathematical expression recognition