Understanding SyncMap: Analyzing the components of Its Dynamical Equation
Pages 246 - 254
Abstract
SycnMap has been recently proposed as an unsupervised approach to perform chunking. This model, which falls under the paradigm of self-organizing dynamical equations, can achieve learning merely using the principle of self-organization without any objective function. However, it is still poorly understood due to its novelty. Here, we provide a comprehensive analysis of the underlying dynamical equation that governed the learning of SyncMap. We first introduce several components of the dynamical equation: (1) Learning rate, (2) Dynamic noise, and (3) Coefficient of attraction force; As well as model-specific variables: (4) Input signal noise and (5) Dimension of weight space. With that, we examine their effect on the performance of SyncMap. Our study shows that the dynamic noise and dimension of weight space play an important role in the dynamical equation; By solely tuning them, the enhanced model can outperform the baseline methods as well as the original SyncMap in 6 out of 7 environments.
References
[1]
David L Barack and John W Krakauer. 2021. Two views on the cognitive brain. Nature Reviews Neuroscience 22, 6 (2021), 359–371.
[2]
Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35, 8(2013), 1798–1828.
[3]
Zhijian Chen and Nelson Cowan. 2005. Chunk limits and length limits in immediate recall: a reconciliation.Journal of Experimental Psychology: Learning, Memory, and Cognition 31, 6(2005), 1235.
[4]
Nelson Cowan. 2001. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and brain sciences 24, 1 (2001), 87–114.
[5]
Fernand Gobet, Peter CR Lane, Steve Croker, Peter CH Cheng, Gary Jones, Iain Oliver, and Julian M Pine. 2001. Chunking mechanisms in human learning. Trends in cognitive sciences 5, 6 (2001), 236–243.
[6]
Alexander N Gorban and Andrei Zinovyev. 2010. Principal manifolds and graphs in practice: from molecular biology to dynamical systems. International journal of neural systems 20, 03 (2010), 219–232.
[7]
Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013).
[8]
Teuvo Kohonen. 1990. The self-organizing map. Proc. IEEE 78, 9 (1990), 1464–1480.
[9]
Nikolaus Kriegeskorte and Jörn Diedrichsen. 2019. Peeling the onion of brain representations. Annual review of neuroscience 42 (2019), 407–432.
[10]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444.
[11]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781(2013).
[12]
Joaquin Quinonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. 2008. Dataset shift in machine learning. Mit Press.
[13]
Anna C Schapiro, Timothy T Rogers, Natalia I Cordova, Nicholas B Turk-Browne, and Matthew M Botvinick. 2013. Neural representations of events arise from temporal community structure. Nature neuroscience 16, 4 (2013), 486–492.
[14]
Erich Schubert, Jörg Sander, Martin Ester, Hans Peter Kriegel, and Xiaowei Xu. 2017. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Transactions on Database Systems (TODS) 42, 3 (2017), 1–21.
[15]
Hamed Shah-Hosseini and Reza Safabakhsh. 2003. TASOM: a new time adaptive self-organizing map. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 33, 2(2003), 271–282.
[16]
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. 2015. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems 28 (2015).
[17]
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23, 5(2019), 828–841.
[18]
Danilo Vasconcellos Vargas and Toshitake Asabuki. 2021. Continual general chunking problem and syncmap. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 10006–10014.
Index Terms
- Understanding SyncMap: Analyzing the components of Its Dynamical Equation
Recommendations
Dynamics in spectral solutions of Burgers equation
For low values of the viscosity coefficient, Burgers equation can develop sharp discontinuities, which are difficult to simulate in a computer. Oscillations can occur by discretization through spectral collocation methods, due to Gibbs phenomena. Under ...
Dynamics and synchronization of numerical solutions of the Burgers equation
For Chebyshev spectral solutions of the forced Burgers equation with low values of the viscosity coefficient, several bifurcations and stable attractors can be observed. Periodic orbits, quasiperiodic and strange ones may arise. Bistability can also be ...
Analysing Dynamical Systems
SIMULTECH 2017: Proceedings of the 7th International Conference on Simulation and Modeling Methodologies, Technologies and ApplicationsOrdinary differential equations arise in a variety of applications, including e.g. climate systems, and can exhibit complicated dynamical behaviour. Complete Lyapunov functions can capture this behaviour by dividing the phase space into the chain-...
Comments
Information & Contributors
Information
Published In
December 2022
302 pages
ISBN:9781450398749
DOI:10.1145/3582099
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 20 April 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- JST SPRING
- JST, ACT-I
- JSPS KAKENHI
Conference
AICCC 2022
AICCC 2022: 2022 5th Artificial Intelligence and Cloud Computing Conference
December 17 - 19, 2022
Osaka, Japan
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 29Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format