research-article

Tactile mesh saliency

Authors:

Holly RushmeierAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 35, Issue 4

Article No.: 52, Pages 1 - 11

https://doi.org/10.1145/2897824.2925927

Published: 11 July 2016 Publication History

Abstract

While the concept of visual saliency has been previously explored in the areas of mesh and image processing, saliency detection also applies to other sensory stimuli. In this paper, we explore the problem of tactile mesh saliency, where we define salient points on a virtual mesh as those that a human is more likely to grasp, press, or touch if the mesh were a real-world object. We solve the problem of taking as input a 3D mesh and computing the relative tactile saliency of every mesh vertex. Since it is difficult to manually define a tactile saliency measure, we introduce a crowdsourcing and learning framework. It is typically easy for humans to provide relative rankings of saliency between vertices rather than absolute values. We thereby collect crowdsourced data of such relative rankings and take a learning-to-rank approach. We develop a new formulation to combine deep learning and learning-to-rank methods to compute a tactile saliency measure. We demonstrate our framework with a variety of 3D meshes and various applications including material suggestion for rendering and fabrication.

Supplementary Material

ZIP File (a52-lau-supp.zip)

Supplemental files.

Download
47.78 MB

MP4 File (a52.mp4)

Download
326.41 MB

References

[1]

Bächer, M., Bickel, B., James, D. L., and Pfister, H. 2012. Fabricating Articulated Characters from Skinned Meshes. ACM Trans. Graph. 31, 4 (July), 47:1--47:9.

Digital Library

[2]

Bohg, J., Morales, A., Asfour, T., and Kragic, D. 2014. Data-Driven Grasp Synthesis -- A Survey. IEEE Transactions on Robotics 30, 2, 289--309.

Digital Library

[3]

Borji, A., Sihite, D. N., and Itti, L. 2012. Salient Object Detection: A Benchmark. ECCV, 414--429.

[4]

Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva, A., and Torralba, A., 2015. MIT Saliency Benchmark. http://saliency.mit.edu/.

[5]

Chapelle, O., and Keerthi, S. S. 2010. Efficient Algorithms for Ranking with SVMs. Information Retrieval 13, 3, 201--215.

Digital Library

[6]

Chen, D. Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum 22, 3, 223--232.

[7]

Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graph. 28, 3 (July), 73:1--73:12.

Digital Library

[8]

Chen, X., Saparov, A., Pang, B., and Funkhouser, T. 2012. Schelling Points on 3D Surface Meshes. ACM Trans. Graph. 31, 4 (July), 29:1--29:12.

Digital Library

[9]

Cignoni, P., Pietroni, N., Malomo, L., and Scopigno, R. 2014. Field-aligned Mesh Joinery. ACM Trans. Graph. 33, 1 (Feb.), 11:1--11:12.

Digital Library

[10]

Dorsey, J., and Hanrahan, P. 1996. Modeling and Rendering of Metallic Patinas. In Proceedings of SIGGRAPH 96, Annual Conference Series, 387--396.

Digital Library

[11]

Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1 (Jan.), 130--150.

Digital Library

[12]

Garces, E., Agarwala, A., Gutierrez, D., and Hertzmann, A. 2014. A Similarity Measure for Illustration Style. ACM Trans. Graph. 33, 4 (July), 93:1--93:9.

Digital Library

[13]

Gingold, Y., Shamir, A., and Cohen-Or, D. 2012. Micro Perceptual Human Computation for Visual Tasks. ACM Trans. Graph. 31, 5 (Sept.), 119:1--119:12.

Digital Library

[14]

Gingold, Y., Vouga, E., Grinspun, E., and Hirsh, H. 2012. Diamonds from the Rough: Improving Drawing, Painting, and Singing via Crowdsourcing. Proceedings of the AAAI Workshop on Human Computation (HCOMP).

[15]

Goferman, S., Zelnik-Manor, L., and Tal, A. 2012. Context-Aware Saliency Detection. PAMI 34, 10, 1915--1926.

Digital Library

[16]

Goldfeder, C., Ciocarlie, M., Dang, H., and Allen, P. K. 2009. The Columbia Grasp Database. ICRA, 3343--3349.

Digital Library

[17]

Hildebrand, K., Bickel, B., and Alexa, M. 2013. Orthogonal Slicing for Additive Manufacturing. SMI 37, 6, 669--675.

Digital Library

[18]

Hisada, M., Belyaev, A. G., and Kunii, T. L. 2002. A Skeleton-based Approach for Detection of Perceptually Salient Features on Polygonal Surfaces. CGF 21, 4, 689--700.

[19]

Horn, B. 1984. Extended Gaussian Images. Proceedings of the IEEE 72, 12, 1671--1686.

[20]

Howlett, S., Hamill, J., and O'Sullivan, C. 2005. Predicting and Evaluating Saliency for Simplified Polygonal Models. ACM Trans. on Applied Perception 2, 3 (July), 286--308.

Digital Library

[21]

Hu, J., Lu, J., and Tan, Y. P. 2014. Discriminative Deep Metric Learning for Face Verification in the Wild. CVPR, 1875--1882.

Digital Library

[22]

Hu, J., Lu, J., and Tan, Y. P. 2015. Deep Transfer Metric Learning. CVPR, 325--333.

[23]

Itti, L., Koch, C., and Niebur, E. 1998. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. PAMI 20, 11, 1254--1259.

Digital Library

[24]

Itti, L., 2000. Models of Bottom-Up and Top-Down Visual Attention. PhD Thesis, California Institute of Technology Pasadena.

Digital Library

[25]

Jain, A., Thormählen, T., Ritschel, T., and Seidel, H.-P. 2012. Material Memex: Automatic Material Suggestions for 3D Objects. ACM Trans. Graph. 31, 6 (Nov.), 143:1--143:8.

Digital Library

[26]

Järvelin, K., and Kekälänen, J. 2002. Cumulated Gain-based Evaluation of IR Techniques. ACM Trans. on Information Systems 20, 4, 422--446.

Digital Library

[27]

Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D Mesh Segmentation and Labeling. ACM Trans. Graph. 29, 4 (July), 102:1--102:12.

Digital Library

[28]

Kim, Y., Varshney, A., Jacobs, D. W., and Guimbretière, F. 2010. Mesh Saliency and Human Eye Fixations. ACM Trans. on Applied Perception 7, 2 (Feb.), 12:1--12:13.

Digital Library

[29]

Klank, U., Pangercic, D., Rusu, R., and Beetz, M. 2009. Real-time CAD Model Matching for Mobile Manipulation and Grasping. IEEE-RAS Int'l Conf. on Humanoid Robots, 290--296.

[30]

Lau, M., Ohgawara, A., Mitani, J., and Igarashi, T. 2011. Converting 3D Furniture Models to Fabricatable Parts and Connectors. ACM Trans. Graph. 30, 4 (July), 85:1--85:6.

Digital Library

[31]

Lee, C. H., Varshney, A., and Jacobs, D. W. 2005. Mesh Saliency. ACM Trans. Graph. 24, 3 (July), 659--666.

Digital Library

[32]

Liu, Y., Zhu, H., Liu, X., and Wu, E. 2005. Real-time Simulation of Physically based On-Surface Flow. The Visual Computer 21, 8-10, 727--734.

[33]

Liu, T., Hertzmann, A., Li, W., and Funkhouser, T. 2015. Style Compatibility for 3D Furniture Models. ACM Trans. Graph. 34, 4 (July), 85:1--85:9.

Digital Library

[34]

Liu, Z., Wang, X., and Bu, S. 2015. Human-Centered Saliency Detection. IEEE Trans. on Neural Networks and Learning Systems.

[35]

Mérillou, S., and Ghazanfarpour, D. 2008. A Survey of Aging and Weathering Phenomena in Computer Graphics. Computers & Graphics 32, 2, 159--174.

Digital Library

[36]

O'Donovan, P., Libeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory Font Selection Using Crowd-sourced Attributes. ACM Trans. Graph. 33, 4 (July), 92:1--92:9.

Digital Library

[37]

Osada, R., Funkhouser, T., Chazelle, B., and Dobkin, D. 2001. Matching 3D Models with Shape Distributions. Shape Modeling International, 154--166.

Digital Library

[38]

Parikh, D., and Grauman, K. 2011. Relative Attributes. International Conference on Computer Vision (ICCV), 503--510.

Digital Library

[39]

Plaisier, M. A., Tiest, W. M. B., and Kappers, A. M. L. 2009. Salient Features in 3-D Haptic Shape Perception. Attention, Perception, & Psychophysics 71, 2, 421--430.

[40]

Prévost, R., Whiting, E., Lefebvre, S., and Sorkine-Hornung, O. 2013. Make It Stand: Balancing Shapes for 3D Fabrication. ACM Trans. Graph. 32, 4 (July), 81:1--81:10.

Digital Library

[41]

Sahbani, A., and El-Khoury, S. 2009. A Hybrid Approach for Grasping 3D Objects. IROS, 1272--1277.

Digital Library

[42]

Sahbani, A., El-khoury, S., and Bidaud, P. 2012. An Overview of 3D Object Grasp Synthesis Algorithms. Robotics and Autonomous Systems 60, 3, 326--336.

Digital Library

[43]

Saxena, A., Driemeyer, J., Kearns, J., and Ng, A. Y. 2007. Robotic Grasping of Novel Objects. NIPS, 1209--1216.

[44]

Schwartzburg, Y., and Pauly, M. 2013. Fabrication-aware Design with Intersecting Planar Pieces. CGF 32, 2, 317--326.

[45]

Shilane, P., and Funkhouser, T. 2007. Distinctive Regions of 3D Surfaces. ACM Trans. Graph. 26, 2 (June), 7.

Digital Library

[46]

Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. SMI, 167--178.

Digital Library

[47]

Shtrom, E., Leifman, G., and Tal, A. 2013. Saliency Detection in Large Point Sets. ICCV, 3591--3598.

Digital Library

[48]

Song, R., Liu, Y., Martin, R. R., and Rosin, P. L. 2014. Mesh Saliency via Spectral Processing. ACM Trans. Graph. 33, 1 (Feb.), 6:1--6:17.

Digital Library

[49]

Su, H., Maji, S., Kalogerakis, E., and Learned-Miller, E. 2015. Multi-view Convolutional Neural Networks for 3D Shape Recognition. ICCV.

Digital Library

[50]

Surazhsky, T., Magid, E., Soldea, O., Elber, G., and Rivlin, E. 2003. A Comparison of Gaussian and Mean Curvatures Estimation Methods on Triangular Meshes. International Conference on Robotics and Automation, 1021--1026.

[51]

Tao, P., Cao, J., Li, S., Liu, X., and Liu, L. 2015. Mesh Saliency via Ranking Unsalient Patches in a Descriptor Space. Computers and Graphics 46, C, 264--274.

Digital Library

[52]

Varadarajan, K., Potapova, E., and Vincze, M. 2012. Attention driven Grasping for Clearing a Heap of Objects. IROS, 2035--2042.

[53]

Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., and Wu, Y. 2014. Learning Fine-Grained Image Similarity with Deep Ranking. CVPR, 1386--1393.

Digital Library

[54]

Watanabe, K., and Belyaev, A. G. 2001. Detection of Salient Curvature Features on Polygonal Surfaces. CGF 20, 3, 385--392.

[55]

Wei, L., Huang, Q., Ceylan, D., Vouga, E., and Li, H. 2015. Dense Human Body Correspondences Using Convolutional Networks. CVPR.

[56]

Xu, M., Ni, B., Dong, J., Huang, Z., Wang, M., and Yan, S. 2012. Touch Saliency. ACM International Conference on Multimedia, 1041--1044.

Digital Library

[57]

Zagoruyko, S., and Komodakis, N. 2015. Learning to Compare Image Patches via Convolutional Neural Networks. CVPR.

[58]

Zhou, Q., Panetta, J., and Zorin, D. 2013. Worst-case Structural Analysis. ACM Trans. Graph. 32, 4 (July), 137:1--137:12.

Digital Library

[59]

Zimmer, H., Lafarge, F., Alliez, P., and Kobbelt, L. 2014. Zometool Shape Approximation. Graphical Models 76, 5, 390--401.

Digital Library

Cited By

Wang SLiu XWang CLiu J(2024)Physics-aware iterative learning and prediction of saliency map for bimanual grasp planningComputer Aided Geometric Design10.1016/j.cagd.2024.102298111(102298)Online publication date: Jun-2024
https://doi.org/10.1016/j.cagd.2024.102298
Luo HZhai WZhang JCao YTao D(2024)Grounded Affordance from Exocentric ViewInternational Journal of Computer Vision10.1007/s11263-023-01962-z132:6(1945-1969)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11263-023-01962-z
Martin DFandos AMasia BSerrano A(2024)SAL3D: a model for saliency prediction in 3D meshesThe Visual Computer10.1007/s00371-023-03206-040:11(7761-7771)Online publication date: 4-Jan-2024
https://doi.org/10.1007/s00371-023-03206-0
Show More Cited By

Index Terms

Tactile mesh saliency
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling

Recommendations

Mesh saliency

Research over the last decade has built a solid mathematical foundation for representation and analysis of 3D meshes in graphics and geometric modeling. Much of this work however does not explicitly incorporate models of low-level human visual ...
Mesh saliency
SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

Research over the last decade has built a solid mathematical foundation for representation and analysis of 3D meshes in graphics and geometric modeling. Much of this work however does not explicitly incorporate models of low-level human visual ...
A Perceptual Metric Based on Salient Information Entropy for 3D Mesh Distortion
IIH-MSP '10: Proceedings of the 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing

Visual degradation and distortion may be introduced during the 3D mesh simplification. The critical issue in all mesh simplification methods is how to evaluate it. Some geometric metrics without the consideration of human visual perception and perceptual ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 35, Issue 4

July 2016

1396 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2897824

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2016

Published in TOG Volume 35, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF grants
Microsoft Research PhD program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
949
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)8

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang SLiu XWang CLiu J(2024)Physics-aware iterative learning and prediction of saliency map for bimanual grasp planningComputer Aided Geometric Design10.1016/j.cagd.2024.102298111(102298)Online publication date: Jun-2024
https://doi.org/10.1016/j.cagd.2024.102298
Luo HZhai WZhang JCao YTao D(2024)Grounded Affordance from Exocentric ViewInternational Journal of Computer Vision10.1007/s11263-023-01962-z132:6(1945-1969)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s11263-023-01962-z
Martin DFandos AMasia BSerrano A(2024)SAL3D: a model for saliency prediction in 3D meshesThe Visual Computer10.1007/s00371-023-03206-040:11(7761-7771)Online publication date: 4-Jan-2024
https://doi.org/10.1007/s00371-023-03206-0
Shu ZGao LYi SWu FDing XWan TXin S(2023)Context-Aware 3D Points of Interest Detection via Spatial Attention MechanismACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359702619:6(1-19)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3597026
Chen MLau M(2023)Learning 3D Shape Aesthetics Globally and LocallyComputer Graphics Forum10.1111/cgf.1470241:7(579-588)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14702
Chen GDai HZhou TShen JShao L(2023)Automatic Schelling Point Detection From MeshesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314414329:6(2926-2939)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TVCG.2022.3144143
Song RZhang WZhao YLiu YRosin P(2023)3D Visual Saliency: An Independent Perceptual Measure or A Derivative of 2D Image Saliency?IEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3287356(1-17)Online publication date: 2023
https://doi.org/10.1109/TPAMI.2023.3287356
Ujitoko YBan Y(2023)Toward Designing Haptic Displays for Desired Touch Targets: A Study of User Expectation for Haptic Properties via CrowdsourcingIEEE Transactions on Haptics10.1109/TOH.2023.331066216:4(726-735)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TOH.2023.3310662
Dev KPalumbo LLau M(2023)Comparing Shape Representations for the Aesthetic Judgment of 3-D Shape PairsIEEE Computer Graphics and Applications10.1109/MCG.2023.328604743:6(33-45)Online publication date: Nov-2023
https://doi.org/10.1109/MCG.2023.3286047
Lin YComi MChurch AZhang DLepora N(2023)Attention for Robot Touch: Tactile Saliency Prediction for Robust Sim-to-Real Tactile Control2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS55552.2023.10341888(10806-10812)Online publication date: 1-Oct-2023
https://doi.org/10.1109/IROS55552.2023.10341888
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents