research-article

Open access

Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog

Authors:

Cathal GurrinAuthors Info & Claims

LSC '22: Proceedings of the 5th Annual on Lifelog Search Challenge

Pages 43 - 47

https://doi.org/10.1145/3512729.3533009

Published: 27 June 2022 Publication History

Abstract

Voxento is an interactive voice-based retrieval system for lifelogs which has been redeveloped and optimised to participate in the fifth Lifelog Search Challenge LSC'22, at ACM ICMR'22. Based on the previous experience in the LSC competition and ranked in the top 4 in the last LSC'21 competition among 17 participants, we present a revised version of Voxento to address the critical points to improve the efficiency of retrieval tasks in lifelog datasets. Basically, Voxento provides a spoken interface to the lifelog data, which facilitates an expert and novice user to interact with a personal lifelog using a range of vocal commands and interactions. Briefly, we made some important improvements to support both the retrieval of content and system interaction. This latest version has been enhanced with the addition of a text-based search feature, new filters based on new metadata provided in lifelog data, rich visual information and features and enhanced speech query. Also, the data preparation tasks comprised a new function to reduce the number of non-relevant images and the latest CLIP model version used to derive features from images. The long term development of Voxento includes a lifelog retrieval that supports speech and conversation interaction with less physical actions required by users such as using a mouse. The system presented here uses a desktop computer in order to participate in the LSC'22 competition with the option to use voice interaction or standard text-based retrieval.

References

[1]

Naushad Alam, Yvette Graham, and Cathal Gurrin. 2021. Memento: A Prototype Lifelog Search Engine for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 53--58. https://doi.org/10.1145/3463948.3469069

Digital Library

[2]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2020. Voxento: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC'20) (Dublin, Ireland). Association for Computing Machinery, New York, NY, USA, 77--81. https://doi.org/10.1145/3379172.3391728

Digital Library

[3]

Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2021. Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 65--70. https://doi.org/10.1145/3463948.3469071

Digital Library

[4]

Aaron Duane and Bjorn THORNór Jónsson. 2021. ViRMA: Virtual Reality Multimedia Analytics at LSC 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 29--34. https://doi.org/10.1145/3463948.3469067

Digital Library

[5]

Jim Gemmell, Gordon Bell, and Roger Lueder. 2006. MyLifeBits: A personal database for everything. Commun. ACM 49, 1 (2006), 88--95. https://doi.org/10.1145/1107458.1107460

Digital Library

[6]

Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal big data. Vol. 8. Now Publishers. 1--125 pages. https://doi.org/10.1561/1500000033

Digital Library

[7]

Cathal Gurrin, Liting Zhou, Graham Healy, Björn Por Þór Jónsson, Duc-Tien Dang-Nguyen, Jakub Lokoc, Minh-Triet Tran, Wolfgang Hürst, Luca Rossetto, and Klaus Schöffmann. 2022. Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22. In Proc. International Conference on Multimedia Retrieval (ICMR'22). Association for Computing Machinery, New York, NY, USA.

Digital Library

[8]

Omar Shahbaz Khan, Aaron Duane, Björn THORNór Jónsson, Jan Zahálka, Stevan Rudinac, and Marcel Worring. 2021. Exquisitor at the Lifelog Search Challenge 2021: Relationships between Semantic Classifiers. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 3--6. https://doi.org/10.1145/3463948.3469255

Digital Library

[9]

Emil Knudsen, Thomas Holstein Qvortrup, Omar Shahbaz Khan, and Björn THORNór Jónsson. 2021. XQC at the Lifelog Search Challenge 2021: Interactive Learning on a Mobile Device. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 89--93. https://doi.org/10.1145/3463948.3469063

Digital Library

[10]

Andreas Leibetseder and Klaus Schoeffmann. 2021. LifeXplore at the Lifelog Search Challenge 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 23--28. https://doi.org/10.1145/3463948.3469060

Digital Library

[11]

Jakub Lokoc, Frantiek Mejzlik, Patrik Veselý, and Tomá Soucek. 2021. Enhanced SOMHunter for Known-item Search in Lifelog Data. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 71--73. https://doi.org/10.1145/3463948.3469074

Digital Library

[12]

Thao Nhu Nguyen, Tu Khiem Le, Van Tu Ninh, Minh Triet Tran, Nguyen Thanh Binh, Graham Healy, Annalina Caputo, and Cathal Gurrin. 2021. LifeSeeker 3.0: An Interactive Lifelog Search Engine for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 41--46. https://doi.org/10.1145/3463948.3469065

Digital Library

[13]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.00020 http://arxiv.org/abs/2103.00020

[14]

Jihye Shin, Alexandra Waldau, Aaron Duane, and Björn THORNór Jónsson. 2021. PhotoCube at the Lifelog Search Challenge 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 59--63. https://doi.org/10.1145/3463948.3469073

Digital Library

[15]

Florian Spiess, Ralph Gasser, Silvan Heller, Luca Rossetto, Loris Sauter, Milan Van Zanten, and Heiko Schuldt. 2021. Exploring Intuitive Lifelog Retrieval and Interaction Modes in Virtual Reality with vitrivr-VR. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 17--22. https://doi.org/10.1145/3463948.3469061

Digital Library

[16]

Ly Duyen Tran, Manh Duy Nguyen, Nguyen Thanh Binh, Hyowon Lee, and Cathal Gurrin. 2021. Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 11--16. https://doi.org/10.1145/3463948.3469064

Digital Library

Cited By

Alateeq ARoantree MGurrin C(2024)Voxento-Pro: An Advanced Voice Lifelog Retrieval Interaction for Multimodal LifelogsProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661130(105-110)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661130
Gago AKaluza BBartolomeu EPandeirada JRibeiro RNeves A(2024)MEMORIA: A Memory Enhancement and MOment RetrIeval Application at the LSC2024Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661129(99-104)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661129
Vuong GHo VNguyen Dang TThai XNguyen-Ho TPham MLe TNinh VHealy GGurrin CTran M(2024)LifeInsight2.0: An Enhanced Approach for Automated Lifelog Retrieval in LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661112(1-6)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661112
Show More Cited By

Index Terms

Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog

Recommendations

LifeSeeker 4.0: An Interactive Lifelog Search Engine for LSC'22
LSC '22: Proceedings of the 5th Annual on Lifelog Search Challenge

In this paper, we introduce LifeSeeker 4.0 - an interactive lifelog retrieval system developed for the fifth annual Lifelog Search Challenge (LSC'22). In LifeSeeker 4.0, we focus on enhancing our previous system to allow users who have little to no ...
Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs
LSC '21: Proceedings of the 4th Annual on Lifelog Search Challenge

In this paper, we describe an extended version of Voxento which is an interactive voice-based retrieval system for lifelogs that has been developed to participate in the fourth Lifelog Search Challenge LSC'21, at ACM ICMR'21. Voxento provides a spoken ...
Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search Challenge

In this paper, we introduce Voxento 4.0 – an interactive voice-based retrieval system for lifelogs which has been developed to participate in the sixth Lifelog Search Challenge LSC’23, at ACM ICMR’23. Voxento has participated three times in the LSC ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LSC '22: Proceedings of the 5th Annual on Lifelog Search Challenge

June 2022

59 pages

ISBN:9781450392396

DOI:10.1145/3512729

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Graham Healy
Dublin City University, Ireland
,
Liting Zhou
Dublin City University, Ireland
,
Björn Þór Jónsson
IT University of Copenhagen, Denmark
,
Duc-Tien Dang-Nguyen
University of Bergen, Norway
,
Jakub Lokoč
Charles University, Czech Republic
,
Minh-Triet Tran
HCM University of Science, Vietnam
,
Wolfgang Hürst
Utrecht University, Netherlands
,
Luca Rossetto
University of Zurich, Switzerland
,
Klaus Schoeffmann
Klagenfurt University, Austria

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 June 2022

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

The Insight Centre for Data Analytics
The Ministry of Education in Saudi Arabia

Conference

ICMR '22

Sponsor:

SIGMM

ICMR '22: International Conference on Multimedia Retrieval

June 27 - 30, 2022

NJ, Newark, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)95
Downloads (Last 6 weeks)16

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Alateeq ARoantree MGurrin C(2024)Voxento-Pro: An Advanced Voice Lifelog Retrieval Interaction for Multimodal LifelogsProceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661130(105-110)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661130
Gago AKaluza BBartolomeu EPandeirada JRibeiro RNeves A(2024)MEMORIA: A Memory Enhancement and MOment RetrIeval Application at the LSC2024Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661129(99-104)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661129
Vuong GHo VNguyen Dang TThai XNguyen-Ho TPham MLe TNinh VHealy GGurrin CTran M(2024)LifeInsight2.0: An Enhanced Approach for Automated Lifelog Retrieval in LSC'24Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge10.1145/3643489.3661112(1-6)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3643489.3661112
Vuong GHo VNguyen-Dang TThai XNinh VPham MLe THealy GTran M(2023)NewsInsight: A Comprehensive Video Event Retrieval System with Spatial Insights and Query AssistanceProceedings of the 12th International Symposium on Information and Communication Technology10.1145/3628797.3628805(893-900)Online publication date: 7-Dec-2023
https://dl.acm.org/doi/10.1145/3628797.3628805
Nguyen-Dang TThai XVuong GHo VTran MNinh VPham MLe THealy G(2023)LifeInsight: An Interactive Lifelog Retrieval System with Comprehensive Spatial Insights and Query AssistanceProceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593106(59-64)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593106
Alam NGraham YGurrin C(2023)Memento 3.0: An Enhanced Lifelog Search Engine for LSC’23Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593103(41-46)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593103
Tran QTran LNguyen BGurrin C(2023)MemoriEase: An Interactive Lifelog Retrieval System for LSC’23Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593101(30-35)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593101
Ribeiro RAmaral LYe WTrifan ANeves AIglésias P(2023)MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593099(18-23)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593099
Nguyen TLe TNinh VGurrin CTran MNguyen THealy GCaputo ASmyth S(2023)E-LifeSeeker: An Interactive Lifelog Search Engine for LSC’23Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593098(13-17)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593098
Alateeq ARoantree MGurrin C(2023)Voxento 4.0: A More Flexible Visualisation and Control for LifelogsProceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593097(7-12)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3592573.3593097

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents