research-article

Public Access

Transformer: a database-driven approach to generating forms for constrained interaction

Authors:

Protiva Rahman,

Arnab NandiAuthors Info & Claims

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Pages 485 - 496

https://doi.org/10.1145/3301275.3302269

Published: 17 March 2019 Publication History

Abstract

Form-based data insertion or querying is often one of the most time-consuming steps in data-driven workflows. The small screen and lack of physical keyboard in devices such as smartphones and smartwatches introduce imprecision during user input. This can lead to data quality issues such as incomplete responses and errors, increasing user input time. We present Transformer, a system that leverages the contents of the database to automatically optimize forms for constrained input settings. Our cost function models the user input effort based on the schema and data distribution. This is used by Transformer to find the user interface (UI) widget and layout with ideal input cost for each form field. We demonstrate through user studies that Transformer provides a significantly improved user experience, with up to 50% and 57% reduction in form completion time for smartphones and smartwatches respectively.

References

[1]

AllMusic Advanced Music Search. https://www.allmusic.com/advanced-search/.

[2]

Bootstrap. http://getbootstrap.com/.

[3]

Boston University Roommate Matching Form. https://www.bu.edu/sth/admissions/enroll/finding-housing-in-boston/roommate-matching-form/.

[4]

Facilities Maintenance Request Form. https://s2f.osu.edu/.

[5]

Hilton Room Reservation. https://www3.hilton.com/en/index.html/.

[6]

Library Advanced Search. https://osu.worldcat.org/advancedsearch.

[7]

Room Reservation Request. https://ohiounion.osu.edu/meetings_events/space_requests/classroom_requests.

[8]

United Airlines Flight Search. https://www.united.com/en/us/.

[9]

U.S. Passport Application Form. https://pptform.state.gov/.

[10]

S. Al-Megren. A predictive fingerstroke-level model for smartwatch interaction. Multimodal Technologies and Interaction, 2(3):38, 2018.

[11]

S. Al-Megren, W. Altamimi, and H. S. Al-Khalifa. Blind flm: An enhanced keystroke-level model for visually impaired smartphone interaction. In IFIP Conference on Human-Computer Interaction, pages 155--172. Springer, 2017.

[12]

G. J. Badros et al. The cassowary linear arithmetic constraint solving algorithm. TOCHI, 2001.

Digital Library

[13]

G. Bailly, A. Oulasvirta, T. Kötzing, and S. Hoppe. Menuoptimizer: Interactive optimization of menu systems. In Proceedings of the 26th annual ACM symposium on User interface software and technology, pages 331--342. ACM, 2013.

Digital Library

[14]

G. Bhatia, Y. Fu, K. Kowalczykowski, K. W. Ong, K. K. Zhao, A. Deutsch, and Y. Papakonstantinou. Forward: Design specification techniques for do-it-yourself application platforms. In WebDB, 2009.

[15]

X. Bi, Y. Li, and S. Zhai. Ffitts law: modeling finger touch with fitts' law. In SIGCHI, 2013.

Digital Library

[16]

F. Bodart and J. Vanderdonckt. On the problem of selecting interaction objects. In BCSHCI, pages 163--178, 1994.

Digital Library

[17]

S. K. Card. The psychology of human-computer interaction. CRC Press, 2017.

[18]

B. S. Chaparro et al. Is touch-based text input practical for a smartwatch? In HCI International 2015-Posters. Springer, 2015.

[19]

K. Chen, H. Chen, N. Conway, J. M. Hellerstein, and T. S. Parikh. Usher: Improving data quality with dynamic forms. IEEE Transactions on Knowledge and Data Engineering, 23(8):1138--1153, 2011.

Digital Library

[20]

K. Chen et al. Shreddr: pipelined paper digitization for low-resource organizations. In ACM Symposium on Computing for Development, 2012.

Digital Library

[21]

X. A. Chen et al. Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In UIST, 2014.

Digital Library

[22]

Y. Chen et al. Adapting web pages for small-screen devices. Internet Computing, IEEE, 2005.

Digital Library

[23]

M. P. Couper and G. J. Peterson. Why do web surveys take longer on smartphones? Social Science Computer Review, 2016.

Digital Library

[24]

B. Deka, Z. Huang, C. Franzen, J. Hibschman, D. Afergan, Y. Li, J. Nichols, and R. Kumar. Rico: A mobile app dataset for building data-driven design applications. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pages 845--854. ACM, 2017.

Digital Library

[25]

K. El Batran and M. D. Dunlop. Enhancing klm (keystroke-level model) to fit touch screen mobile devices. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, pages 283--286. ACM, 2014.

Digital Library

[26]

J. Falb, S. Kavaldjian, R. Popp, D. Raneburger, E. Arnautovic, and H. Kaindl. Fully automatic user interface generation from discourse models. In Proceedings of the 14th international conference on Intelligent user interfaces, pages 475--476. ACM, 2009.

Digital Library

[27]

U. Feige. A threshold of ln n for approximating set cover. JACM, 1998.

Digital Library

[28]

P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology, 47(6):381, 1954.

[29]

M. Fleshman, I. Argueta, C. Austin, H. Lee, E. Moyer, and G. Gerling. Facilitating the collection and dissemination of patient care information for emergency medical personnel. In Systems and Information Engineering Design Symposium (SIEDS), 2016 IEEE, pages 239--244. IEEE, 2016.

[30]

K. Gajos and D. S. Weld. Supple: automatically generating user interfaces. In Proceedings of the 9th international conference on Intelligent user interfaces, pages 93--100. ACM, 2004.

Digital Library

[31]

K. Gajos and D. S. Weld. Preference elicitation for interface optimization. In Proceedings of the 18th annual ACM symposium on User interface software and technology, pages 173--182. ACM, 2005.

Digital Library

[32]

K. Z. Gajos, D. S. Weld, and J. O. Wobbrock. Automatically generating personalized user interfaces with supple. Artificial Intelligence, 174(12--13):910--950, 2010.

Digital Library

[33]

K. Z. Gajos, J. O. Wobbrock, and D. S. Weld. Automatically generating user interfaces adapted to users' motor and vision capabilities. In Proceedings of the 20th annual ACM symposium on User interface software and technology, pages 231--240. ACM, 2007.

Digital Library

[34]

K. Z. Gajos, J. O. Wobbrock, and D. S. Weld. Improving the performance of motor-impaired users with automatically-generated, ability-based interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 1257--1266. ACM, 2008.

Digital Library

[35]

M. Gordon, T. Ouyang, and S. Zhai. Watchwriter: Tap and gesture typing on a smartwatch miniature keyboard with statistical decoding. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 3817--3821. ACM, 2016.

Digital Library

[36]

J. He et al. A flexible content adaptation system using a rule-based approach. TKDE, 2007.

Digital Library

[37]

G. G. Hendrix et al. Developing a natural language interface to complex data. TODS, 1978.

Digital Library

[38]

P. Holleis, F. Otto, H. Hussmann, and A. Schmidt. Keystroke-level model for advanced mobile phone interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1505--1514. ACM, 2007.

Digital Library

[39]

M. Jayapandian and H. Jagadish. Automated generation of forms-based database query interface. VLDB, 2008.

Digital Library

[40]

L. Jiang, P. Rahman, and A. Nandi. Evaluating interactive data systems: Workloads, metrics, and guidelines. In Proceedings of the 2018 International Conference on Management of Data, pages 1637--1644. ACM, 2018.

Digital Library

[41]

B. E. John and D. E. Kieras. Using goms for user interface design and evaluation: Which technique? TOCHI, 1996.

Digital Library

[42]

J. Katriel. On a generalized recurrence for bell numbers. Journal of Integer Sequences, 2008.

[43]

K. Kowalzcykowski, A. Deutsch, K. W. Ong, Y. Papakonstantinou, K. K. Zhao, and M. Petropoulos. Do-it-yourself database-driven web applications. In Proceedings of the 4th Biennial Conference on Innovative Data Systems Research (CIDR?09). Citeseer, 2009.

[44]

C. Kulkarni and S. Kiemmer. Automatically adapting web pages to heterogeneous devices. CHI'11, 2011.

Digital Library

[45]

A. Lee, K. Song, H. B. Ryu, J. Kim, and G. Kwon. Fingerstroke time estimates for touchscreen-based mobile gaming interaction. Human movement science, 44:211--224, 2015.

[46]

L. A. Leiva et al. Text entry on tiny qwerty soft keyboards. In CHI. ACM, 2015.

Digital Library

[47]

F. Li and H. Jagadish. Constructing an interactive natural language interface for relational databases. VLDB, 2014.

Digital Library

[48]

G. Li et al. Efficient type-ahead search on relational data: a tastier approach. In SIGMOD, 2009.

Digital Library

[49]

G. Li et al. Efficient fuzzy type-ahead search in tastier. In ICDE, 2010.

[50]

Y. Li and T.-H. Chang. Auto-completion for user interface design, Aug. 16 2016. US Patent 9,417,760.

[51]

Y. Li et al. Nalix: an interactive natural language interface for querying xml. In SIGMOD, 2005.

Digital Library

[52]

MacKenzie et al. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction, 2002.

[53]

M. Nebeling et al. Wearwrite: Crowd-assisted writing from smartwatches. In CHI, 2016.

Digital Library

[54]

B. Niamir. Attribute partitioning in a self-adaptive relational data base system. 1978.

[55]

S. Oney et al. Zoomboard: a diminutive qwerty soft keyboard using iterative zooming for ultra-small devices. SIGCHI, 2013.

Digital Library

[56]

A. Peytchev et al. Web survey design paging versus scrolling. Public opinion quarterly, 2006.

[57]

A. Peytchev and C. A. Hill. Experiments in mobile web survey design. Social Science Computer Review, 2010.

Digital Library

[58]

S. E. Poltrock and J. Grudin. Organizational obstacles to interface design and development: two participant-observer studies. ACM Transactions on Computer-Human Interaction (TOCHI), 1(1):52--80, 1994.

Digital Library

[59]

R. Popp, D. Raneburger, and H. Kaindl. Tool support for automated multi-device gui generation from discourse-based communication models. In Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems, pages 145--150. ACM, 2013.

Digital Library

[60]

A. Quezada, R. Juárez-Ramírez, S. Jiménez, A. Ramírez-Noriega, S. Inzunza, and R. Munoz. Assessing the target?size and drag distance in mobile applications for users with autism. In World Conference on Information Systems and Technologies, pages 1219--1228. Springer, 2018.

[61]

D. Raneburger, H. Kaindl, and R. Popp. Strategies for automated gui tailoring for multiple devices. In System Sciences (HICSS), 2015 48th Hawaii International Conference on, pages 507--516. IEEE, 2015.

Digital Library

[62]

D. Raneburger, R. Popp, and J. Vanderdonckt. An automated layout approach for model-driven wimp-ui generation. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems, pages 91--100. ACM, 2012.

Digital Library

[63]

D. Ritchie et al. D.tour: Style-based exploration of design example galleries. In UIST. ACM, 2011.

Digital Library

[64]

V. Roto et al. Minimap: a web page visualization method for mobile phones. In SIGCHI, 2006.

Digital Library

[65]

H. H. Sad and F. Poirier. Modeling word selection in predictive text entry. In International Conference on Human-Computer Interaction, pages 725--734. Springer, 2009.

Digital Library

[66]

C. A. Sanchez and J. Wiley. To scroll or not to scroll: Scrolling, working memory capacity, and comprehending complex texts. The Journal of the Human Factors and Ergonomics Society, 2009.

[67]

A. Sears. Aide: A step toward metric-based interface development tools. In Proceedings of the 8th annual ACM symposium on User interface and software technology, pages 101--110. ACM, 1995.

Digital Library

[68]

R. St Amant, T. E. Horton, and F. E. Ritter. Model-based evaluation of cell phone menu interaction. In CHI, 2004.

Digital Library

[69]

P. Szekely, P. Sukaviriya, P. Castells, J. Muthukumarasamy, and E. Salcher. Declarative interface models for user interface construction tools: The mastermind approach. In Engineering for Human-Computer Interaction, pages 120--150. Springer, 1996.

Digital Library

[70]

L. Tang, T. Li, Y. Jiang, and Z. Chen. Dynamic query forms for database queries. TKDE, 2014.

[71]

V. Tran, J. Vanderdonckt, M. Kolp, and S. Faulkner. Generating user interface from task, user and domain models. In Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services, 2009. CENTRIC'09. Second International Conference on, pages 19--26. IEEE, 2009.

Digital Library

[72]

S. Vairamuthu, A. Anthoniraj, S. M. Anouncia, and U. K. Wiil. User interface design recommendations through multi-criteria decision analysis. In Knowledge Computing and Its Applications, pages 269--293. Springer, 2018.

[73]

C. Vassilakis et al. Exploiting form semantics and validation checks to improve e-form layout. International journal of Web engineering and technology, 2005.

Digital Library

[74]

Wired Magazine. In Less Than Two Years, A Smartphone Could Be Your Only Computer. http://www.wired.com/2015/02/smartphone-only-computer//.

[75]

H. Wu et al. Seaform: Search-as-you-type in forms. VLDB, 2010.

Digital Library

Cited By

Wolf KWinecki DNandi A(2023)Camera-First Form FillingProceedings of the Workshop on Human-In-the-Loop Data Analytics10.1145/3597465.3605218(1-7)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3597465.3605218
Rahman PNandi AHebert C(2020)Amplifying Domain Expertise in Clinical Data PipelinesJMIR Medical Informatics10.2196/196128:11(e19612)Online publication date: 5-Nov-2020
https://doi.org/10.2196/19612
Rahman PJiang LNandi A(2019)Evaluating interactive data systemsThe VLDB Journal10.1007/s00778-019-00589-2Online publication date: 13-Nov-2019
https://doi.org/10.1007/s00778-019-00589-2

Index Terms

Transformer: a database-driven approach to generating forms for constrained interaction
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interactive systems and tools
      1. User interface programming

Recommendations

B2B-Swipe: Swipe Gesture for Rectangular Smartwatches from a Bezel to a Bezel
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We present B2B-Swipe, a single-finger swipe gesture for a rectangular smartwatch that starts at a bezel and ends at a bezel to enrich input vocabulary. There are 16 possible B2B-Swipes because a rectangular smartwatch has four bezels. Moreover, B2B-...
TapBoard: making a touch screen keyboard more touchable
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

A physical keyboard key has three states, whereas a touch screen usually has only two. Due to this difference, the state corresponding to the touched state of a physical key is missing in a touch screen keyboard. This touched state is an important ...
Transformer: A New Paradigm for Building Data-Parallel Programming Models

Cloud computing drives the design and development of diverse programming models for massive data processing. The Transformer programming framework aims to facilitate the building of diverse data-parallel programming models. Transformer has two layers: a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

March 2019

713 pages

ISBN:9781450362726

DOI:10.1145/3301275

General Chairs:
Wai-Tat Fu,
Shimei Pan,
Program Chairs:
Oliver Brdiczka,
Polo Chau,
Gaelle Calvary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

IUI '19

Sponsor:

IUI '19: 24th International Conference on Intelligent User Interfaces

March 17 - 20, 2019

California, Marina del Ray

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
432
Total Downloads

Downloads (Last 12 months)96
Downloads (Last 6 weeks)12

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wolf KWinecki DNandi A(2023)Camera-First Form FillingProceedings of the Workshop on Human-In-the-Loop Data Analytics10.1145/3597465.3605218(1-7)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3597465.3605218
Rahman PNandi AHebert C(2020)Amplifying Domain Expertise in Clinical Data PipelinesJMIR Medical Informatics10.2196/196128:11(e19612)Online publication date: 5-Nov-2020
https://doi.org/10.2196/19612
Rahman PJiang LNandi A(2019)Evaluating interactive data systemsThe VLDB Journal10.1007/s00778-019-00589-2Online publication date: 13-Nov-2019
https://doi.org/10.1007/s00778-019-00589-2

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents