research-article

Multiple Feature Fusion Based Hand-held Object Recognition with RGB-D data

Authors:

Shuang Liu,

Shuang Wang,

Lifang Wu,

Shuqiang JiangAuthors Info & Claims

ICIMCS '14: Proceedings of International Conference on Internet Multimedia Computing and Service

Pages 303 - 306

https://doi.org/10.1145/2632856.2632947

Published: 10 July 2014 Publication History

Get Access

Abstract

With the advance of computer technology and smart device, many technologies and applications have been developed to enhance the efficiency of human-computer interaction (HCI). For human, the hand is a natural and direct way in communication. Hand-held Object Recognition (HHOR), which is to predict the label for the object people hold in hand, can help machines in understanding the environment and people's intentions. However, it has not been well studied in the community. So, in this paper, we proposed a novel feature fusion based method for hand-held object recognition with RGB-D data. First, the skeleton information is used to initially locate the object and with depth map we extract object region in a region-growing manner. Then on the corresponding object point cloud, we use Multiple Kernel Learning (MKL) to fuse the shape feature with color feature to obtain the advantages of them. Specially, we collected a dataset, which contains 12800 video frames of 16 categories and each frame captures the visual image, depth map and user skeleton data. The experiment shows promising results in both segmentation and recognition.

References

[1]

Kanezaki, A., Suzuki, T., Harada, T., & Kuniyoshi, Y. (2011, May). Fast object detection for robots in a cluttered indoor environment using integral 3D feature table. ICRA, 2011 IEEE International Conference on (pp. 4026--4033). IEEE.

Google Scholar

[2]

Marton, Z. C., Pangercic, D., Rusu, R. B., Holzbach, A., & Beetz, M. (2010, December). Hierarchical object geometric categorization and appearance classification for mobile manipulation. In Humanoid Robots (Humanoids), 2010 10th IEEE-RAS International Conference on (pp. 365--370). IEEE.

Crossref

Google Scholar

[3]

Knopp, J., Prasad, M., Willems, G., Timofte, R., & Van Gool, L. (2010). Hough transform and 3D SURF for robust three dimensional classification. In Computer Vision--ECCV 2010 (pp. 589--602). Springer Berlin Heidelberg.

Digital Library

Google Scholar

[4]

Guo, Y., Sohel, F. A., Bennamoun, M., Wan, J., & Lu, M. (2013, February). RoPS: A local feature descriptor for 3D rigid objects based on rotational projection statistics. In ICCSPA, 2013 1st International Conference on (pp. 1--6). IEEE.

Google Scholar

[5]

Kanezaki, A., Marton, Z. C., Pangercic, D., Harada, T., Kuniyoshi, Y., & Beetz, M. (2011, September). Voxelized shape and color histograms for RGB-D. IROS, Workshop on Active Semantic Perception and Object Search in the Real World, San Francisco, CA, USA.

Google Scholar

[6]

do Nascimento, E. R., Oliveira, G. L., Vieira, A. W., & Campos, M. F. (2013). On the development of a robust, fast and lightweight keypoint descriptor. Neurocomputing, 120, 141--155.

Crossref

Google Scholar

[7]

Wohlkinger, W., & Vincze, M. (2011, December). Ensemble of shape functions for 3D object classification. In Robotics and Biomimetics (ROBIO), (pp. 2987--2992). IEEE

Google Scholar

[8]

Ip, C. Y., Lapadat, D., Sieger, L., & Regli, W. C. (2002, June). Using shape distributions to compare solid models. In Proceedings of the seventh ACM symposium on Solid modeling and applications (pp. 273--280). ACM.

Digital Library

Google Scholar

[9]

Sonnenburg, S., Rätsch, G., Schäfer, C., & Schöölkopf, B. (2006). Large scale multiple kernel learning. The Journal of Machine Learning Research, 7, 1531--1565

Digital Library

Google Scholar

Cited By

View all

Gao MJiang JZou GJohn VLiu Z(2019)RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A SurveyIEEE Access10.1109/ACCESS.2019.29070717(43110-43136)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2907071
Pandey RPidlypenskyi PYang SKaeser-Chen C(2018)Efficient 6-DoF Tracking of Handheld Objects from an Egocentric ViewpointComputer Vision – ECCV 201810.1007/978-3-030-01216-8_26(426-441)Online publication date: 9-Oct-2018
https://doi.org/10.1007/978-3-030-01216-8_26
Lv XLiu XLi XLi XJiang SHe Z(2017)Modality-specific and hierarchical feature learning for RGB-D hand-held object recognitionMultimedia Tools and Applications10.1007/s11042-016-3375-576:3(4273-4290)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s11042-016-3375-5
Show More Cited By

Index Terms

Multiple Feature Fusion Based Hand-held Object Recognition with RGB-D data
1. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

RGB-D Object Recognition from Hand-Held Object Teaching
ICIMCS'16: Proceedings of the International Conference on Internet Multimedia Computing and Service

For RGB-D object recognition, conventional methods only focus on classification, which neglects the importance of humans for object segmentation and object concept learning in the interaction and has limitations when transferring the learned knowledge ...
Semi-supervised learning and feature evaluation for RGB-D object recognition

We propose a semi-supervised learning method for RGB-D object recognition.We propose CNN-SPM-RNN to extract powerful RGB-D features.An unbiased feature evaluation for recent RGB-D features are introduced. With new depth sensing technology such as Kinect ...
Hand-Object Sense: A Hand-held Object Recognition System Based on RGB-D Information
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Hand-held objects play an important role in human-human and human-machine interaction. It can be used as a reference for understanding user intentions or user requirements. In this technical demonstration, we introduce an object recognition system ...

Comments

Information & Contributors

Information

Published In

ICIMCS '14: Proceedings of International Conference on Internet Multimedia Computing and Service

July 2014

430 pages

ISBN:9781450328104

DOI:10.1145/2632856

General Chairs:
Hanzi Wang
Xiamen University, China
,
Larry Davis
University of Maryland, USA
,
Program Chairs:
Wenwu Zhu
Tsinghua University, China
,
Stephan Kopf
University of Mannheim, Germany
,
Yanyun Qu
Xiamen University, China
,
Publications Chairs:
Jun Yu
Xiamen University, China
,
Jitao Sang
Institute of Automation, CAS, China
,
Tao Mei
Microsoft Research Asia, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

NSF of China: National Natural Science Foundation of China
Beijing ACM SIGMM Chapter

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIMCS '14

ICIMCS '14: International Conference on Internet Multimedia Computing and Service

July 10 - 12, 2014

Xiamen, China

Acceptance Rates

Overall Acceptance Rate 163 of 456 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Gao MJiang JZou GJohn VLiu Z(2019)RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A SurveyIEEE Access10.1109/ACCESS.2019.29070717(43110-43136)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2907071
Pandey RPidlypenskyi PYang SKaeser-Chen C(2018)Efficient 6-DoF Tracking of Handheld Objects from an Egocentric ViewpointComputer Vision – ECCV 201810.1007/978-3-030-01216-8_26(426-441)Online publication date: 9-Oct-2018
https://doi.org/10.1007/978-3-030-01216-8_26
Lv XLiu XLi XLi XJiang SHe Z(2017)Modality-specific and hierarchical feature learning for RGB-D hand-held object recognitionMultimedia Tools and Applications10.1007/s11042-016-3375-576:3(4273-4290)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s11042-016-3375-5
Wan SAggarwal J(2015)Robust object recognition in RGB-D egocentric videos based on Sparse Affine Hull Kernel2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2015.7301302(97-104)Online publication date: Jun-2015
https://doi.org/10.1109/CVPRW.2015.7301302
Lv XJiang SHerranz LWang S(2015)RGB-D Hand-Held Object Recognition Based on Heterogeneous Feature FusionJournal of Computer Science and Technology10.1007/s11390-015-1527-030:2(340-352)Online publication date: 13-Mar-2015
https://doi.org/10.1007/s11390-015-1527-0

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

RGB-D Object Recognition from Hand-Held Object Teaching

Semi-supervised learning and feature evaluation for RGB-D object recognition

Hand-Object Sense: A Hand-held Object Recognition System Based on RGB-D Information