TorchSpatial: A Python Package for Spatial Representation Learning and Geo-Aware Model Development
Authors: 
Qian Cao, Nemin Wu, Zhangyu Wang, Zeping Liu, Yanlin Qi, Jielu Zhang, Joshua Ni, Xiaobai Yao, Hongxu Ma, Lan Mu, Stefano Ermon, Tanuja Ganu, Akshay Nambi, Ni Lao, Gengchen MaiAuthors Info & Claims
Qian Cao, Nemin Wu, Zhangyu Wang, Zeping Liu, Yanlin Qi, Jielu Zhang, Joshua Ni, Xiaobai Yao, Hongxu Ma, Lan Mu, Stefano Ermon, Tanuja Ganu, Akshay Nambi, Ni Lao, Gengchen MaiAuthors Info & ClaimsPages 39 - 42
Abstract
Spatial representation learning (SRL) focuses on developing spatial embeddings from various forms of spatial data, such as points, polylines, polygons, graphs, networks, and images without any additional feature engineering or data conversion step. Effective spatial representation is fundamental for a wide range of downstream geospatial applications, including species distribution modeling, satellite image classification, point cloud classification and segementation, trajectory synthesis, building footprint extraction, and cartographic generalization. Despite the widespread use of SRL as a cornerstone for many spatially-aware AI models, there is still no comprehensive package shared across the community that provides ready-made code to support the implementation and reproduction of SRL model development. To fill this void, we present TorchSpatial, a Python package designed to support the encoding of spatial data, starting with location (point) encoding, a fundamental data type in SRL. TorchSpatial includes two key components: 1) We present TorchSpatial, an SRL framework supporting the development of location encoders. TorchSpatial now integrates 15 widely-used encoders and essential encoder components, ensuring scalability and reproducibility for future developments; 2) We establish a ready-to-use workflow that takes the input hyperparameters and outputs the model inference results and evaluation across geo-aware image classification and regression tasks with access to 17 datasets. We believe TorchSpatial will foster future advancement of SRL and spatial fairness in GeoAI research. The TorchSpatial SRL framework and inference models are available at https://github.com/seai-lab/TorchSpatial.
References
[1]
Benjamin Adams et al. 2015. Frankenplace: interactive thematic mapping for ad hoc exploratory search. In Proceedings of the 24th international conference on world wide web. 12--22.
[2]
Alexandre Alahi et al. 2016. Social lstm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition. 961--971.
[3]
Kumar Ayush et al. 2021. Geography-aware self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10181--10190.
[4]
Azavea/Element 84, Robert Cheetham. [n.d.]. Raster Vision: An open source library and framework for deep learning on satellite and aerial imagery (2017-2023). https://doi.org/10.5281/zenodo.8018177
[5]
Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35, 8 (2013), 1798--1828.
[6]
Thomas Berg, Jiongxin Liu, et al. 2014. Birdsnap: Large-scale fine-grained visual categorization of birds. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2011--2018.
[7]
Vicente Vivanco Cepeda et al. 2023. GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization. In NeurIPS 2023.
[8]
Gordon Christie et al. 2018. Functional map of the world. In CVPR 2018. 6172--6180.
[9]
Elijah Cole et al. 2023. Spatial implicit neural representations for global-scale species mapping. In ICML 2023. PMLR, 6320--6342.
[10]
Matthias Fey et al. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv.1903.02428 (2019).
[11]
Danhuai Guo et al. 2024. SpatialScene2Vec: A self-supervised contrastive representation learning method for spatial scene similarity evaluation. International Journal of Applied Earth Observation and Geoinformation 128 (2024), 103743.
[12]
Weiming Huang et al. 2023. Learning urban region representations with POIs and hierarchical graph infomax. ISPRS Journal of Photogrammetry and Remote Sensing 196 (2023), 134--145.
[13]
Oisin Mac Aodha et al. 2019. Presence-only geographical priors for fine-grained image classification. In ICCV2019. 9596--9606.
[14]
Gengchen Mai et al. 2020. Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells. In ICLR 2020. openreview.
[15]
Gengchen Mai et al. 2020. SE-KGE: A location-aware knowledge graph embedding model for geographic question answering and spatial semantic lifting. Transactions in GIS 24, 3 (2020), 623--655.
[16]
Gengchen Mai et al. 2022. A review of location encoding for GeoAI: methods and applications. International Journal of Geographical Information Science 36, 4 (2022), 639--673.
[17]
Gengchen Mai et al. 2022. Towards general-purpose representation learning of polygonal geometries. GeoInformatica (2022), 1--52.
[18]
Gengchen Mai et al. 2023. Csp: Self-supervised contrastive spatial pre-training for geospatial-visual representations. In ICML 2023. PMLR, 23498--23515.
[19]
Gengchen Mai et al. 2023. Spatial Representation Learning in GeoAI. In Handbook of Geospatial Artificial Intelligence (1st edition ed.), Author's Editor (Ed.). CRC Press, 22.
[20]
Gengchen Mai et al. 2023. Sphere2Vec: A general-purpose location representation learning over a spherical surface for large-scale geospatial predictions. ISPRS Journal of Photogrammetry and Remote Sensing 202 (2023), 439--462.
[21]
Gengchen Mai et al. 2024. SRL: Towards a General-Purpose Framework for Spatial Representation Learning. In ACM SIGSPATIAL 2024.
[22]
Sébastien Marcel and Yann Rodriguez. 2010. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM international conference on Multimedia. 1485--1488.
[23]
Ben Mildenhall et al. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99--106.
[24]
Ali Rahimi, Benjamin Recht, et al. 2007. Random Features for Large-Scale Kernel Machines. In NIPS, Vol. 3. Citeseer, 5.
[25]
Jinmeng Rao et al. 2020. LSTM-TrajGAN: A Deep Learning Approach to Trajectory Privacy Protection. In GIScience 2020. 12:1--12:17.
[26]
Jinmeng Rao et al. 2023. CATS: Conditional Adversarial Trajectory Synthesis for privacy-preserving trajectory data publication using deep learning approaches. International Journal of Geographical Information Science 37, 12 (2023), 2538--2574.
[27]
Esther Rolf et al. 2021. A generalizable and accessible approach to machine learning with global satellite imagery. Nature communications 12, 1 (2021), 4392.
[28]
Marc Rußwurm et al. 2024. Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks. In ICLR 2024.
[29]
Adam J Stewart et al. 2022. Torchgeo: deep learning with geospatial data. In ACM SIGSPATIAL 2022. 1--12.
[30]
Kevin Tang et al. 2015. Improving image classification with location context. In ICCV 2015. 1008--1016.
[31]
Grant Van Horn et al. 2018. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8769--8778.
[32]
Rein van't Veer et al. 2018. Deep learning for classification tasks on geospatial vector polygons. (2018).
[33]
Minjie Wang et al. 2019. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv.1909.01315 (2019).
[34]
Xiaofeng Wang et al. 2019. Molecule property prediction based on spatial graph embedding. Journal of chemical information and modeling 59, 9 (2019), 3817--3828.
[35]
Nemin Wu et al. 2024. TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning. arXiv preprint arXiv:2406.15658 (2024).
[36]
Xiaoling Xia et al. 2017. Inception-v3 for flower classification. In 2017 2nd international conference on image, vision and computing (ICIVC). IEEE, 783--787.
[37]
Christopher Yeh et al. 2021. SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning. In NeurIPS 2021 Datasets and Benchmarks Track. https://openreview.net/forum?id=5HR3vCylqD
[38]
Dazhou Yu et al. 2024. PolygonGNN: Representation Learning for Polygonal Geometries with Heterogeneous Visibility Graph. In KDD 2024. 4012--4022.
Recommendations
SRL: Towards a General-Purpose Framework for Spatial Representation Learning
SIGSPATIAL '24: Proceedings of the 32nd ACM International Conference on Advances in Geographic Information SystemsRepresentation learning (RL) techniques are widely adopted in areas such as natural language processing and computer vision, with prominent examples such as attention and ConvNet architectures. In comparison, many GeoAI works still rely on feature ...
Building of geo-spatial data model for tea agricultural crop-lands compliance with LPIS Core Model (LCM) based land administration domain standards
It is searched on building geo-spatial data model for tea agricultural croplands.It is searched tea agricultural crops integrated LCM/LADM collaboration model.We suggested building geo-spatial data model for tea agricultural croplands.We figured out ...
The spectralrao-monitoring Python package: A RAO's Q diversity index-based application for land-cover/land-use change detection in multifunctional agricultural areas
Highlights- The Rao's Q diversity index is used to detect LCLU changes in multifunctional agricultural areas.
AbstractMonitoring multifunctional agricultural areas is paramount to ensure their cost-effective management. The remote sensing-based detection of land-cover/land-use (LCLU) changes and analysis of vegetation dynamics constitute a relevant ...
Comments
Information & Contributors
Information
Published In
October 2024
47 pages
ISBN:9798400711459
DOI:10.1145/3681766
- Editors:
- Jinmeng Rao,
- Emre Eftelioglu,
- Heba Aly,
- Yiqun Xie,
- Song Gao
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 November 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
Conference
SIGSPATIAL '24
Sponsor:
SIGSPATIAL '24: The 32nd ACM International Conference on Advances in Geographic Information Systems
October 29 - November 1, 2024
GA, Atlanta, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 353Total Downloads
- Downloads (Last 12 months)353
- Downloads (Last 6 weeks)96
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in