short-paper

Public Access

LANMC: LSTM-Assisted Non-Rigid Motion Correction on FPGA for Calcium Image Stabilization

Authors:

Jason CongAuthors Info & Claims

FPGA '19: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

Pages 104 - 109

https://doi.org/10.1145/3289602.3293919

Published: 20 February 2019 Publication History

Abstract

Calcium imaging is an emerging technique for visualizing and recording neural population activity at large scale in vivo. Non-rigid motion correction is a critical step in the calcium image analysis pipeline due to non-uniform deformations of the brain tissue during the data collection. However, existing non-rigid motion correction algorithms are costly in computation time and energy, and it is hard to implement such algorithm in real time on an embedded device. In this paper, we propose LANMC, an LSTM-assisted non-rigid motion correction method for real-time calcium image stabilization. This method reduces the computational cost by using the LSTM inference to predict the non-rigid motion. Based on this method, we demonstrate a non-rigid motion correction implementation for real-time calcium image stabilization on FPGA. Experimental results show that the non-rigid motion correction can be accomplished within 80 µs on the Ultra96 under 300 MHz frequency, and the latency outperforms that on a 12-thread CPU by 82x.

References

[1]

Denise J. Cai, Daniel Aharoni, Tristan Shuman, and et al. 2016. A shared neural ensemble links distinct contextual memories encoded close in time . Nature, Vol. 534 (2016), 115--118.

[2]

T.N. Chandrapala, L.M.A.P. Cabral, S. Ahangama, and et al. 2012. Hardware implementation of motion blur removal . In Int. Conf. Field Program. Log. Appl. 243--248.

[3]

Zhe Chen, Andrew Howe, Hugh T. Blair, and et al. 2018. CLINK: Compact LSTM inference kernel for energy efficient neurofeedback devices. In Proc. Int. Symp. Low Power Electron. Des. 2:1--2:6.

Digital Library

[4]

Ye Han Cong Shi, Jie Yang and et al. 2014. A 1000fps vision chip based on a dynamically reconfigurable hybrid architecture comprising a PE array and self-organizing map neural network. In IEEE Int. Solid-State Circuits Conf. Dig. Tech. Pap. 128--129.

[5]

Gunnar Farneb"a ck. 2003. Two-frame motion estimation based on polynomial expansion. In Image Anal. Springer Berlin Heidelberg, 363--370.

Digital Library

[6]

Kunal K Ghosh, Laurie D Burns, Eric D Cocker, and et al. 2011. Miniaturized integration of a fluorescence microscope . Nat. Methods, Vol. 8 (2011), 871.

[7]

Andrea Giovannucci, Johannes Friedrich, Matthew Kaufman, and et al. 2017. OnACID: Online analysis of calcium imaging data in real time . In Adv. Neural Inf. Process. Syst. 2378--2388.

Digital Library

[8]

Diego Gonzalez, Guillermo Botella, Soumak Mokheerje, and et al. 2011. FPGA-Based acceleration of block matching motion estimation techniques. In Int. Conf. Field Program. Log. Appl. 389--392.

Digital Library

[9]

Xin He, Liu Ke, Wenyan Lu, and et al. 2018. AxTrain: Hardware-oriented neural network training for approximate inference. In Proc. Int. Symp. Low Power Electron. Des. 20:1--20:6.

Digital Library

[10]

Sepp Hochreiter and Jü rgen Schmidhuber. 1997. Long short-term memory . Neural Comput., Vol. 9, 8 (1997), 1735--1780.

Digital Library

[11]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, and et al. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proc. 22nd ACM Int. Conf. Multimed. New York, NY, USA, 675--678.

Digital Library

[12]

Jianan Li, Tingfa Xu, and Kun Zhang. 2017. Real-time feature-based video stabilization on FPGA . IEEE Trans. Circuits Syst. Video Technol., Vol. 27, 4 (2017), 907--919.

Digital Library

[13]

Jinghao Lu, Chunyuan Li, Jonnathan Singh-Alvarado, and et al. 2018. MIN1PIPE: A miniscope 1-photon-based calcium imaging signal extraction pipeline. Cell Rep., Vol. 23, 12 (2018), 3673--3684.

[14]

Eftychios A. Pnevmatikakis and Andrea Giovannucci. 2017. NoRMCorre: An online algorithm for piecewise rigid motion correction of calcium imaging data . J. Neurosci. Methods, Vol. 291 (2017), 83--94.

[15]

Christoph Stosiek, Olga Garaschuk, Knut Holthoff, and et al. 2003. In vivo two-photon calcium imaging of neuronal networks . Proc. Natl. Acad. Sci. U. S. A., Vol. 100, 12 (2003), 7319--7324.

Cited By

Chen ZBlair GCao CZhou JAharoni DGolshani PBlair HCong J(2023)FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback ApplicationsIEEE Transactions on Biomedical Circuits and Systems10.1109/TBCAS.2023.326813017:2(169-179)Online publication date: Apr-2023
https://doi.org/10.1109/TBCAS.2023.3268130
Kim M(2023)Software Engineering for Data Intensive Scalable Computing and Heterogeneous Computing2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE)10.1109/ICSE-FoSE59343.2023.00006(54-68)Online publication date: 14-May-2023
https://doi.org/10.1109/ICSE-FoSE59343.2023.00006
Kundu ANguyen U(2023)PyTPU: Migration of Python Code for Heterogenous Acceleration with Automated Test Generation2023 IEEE/ACIS 8th International Conference on Big Data, Cloud Computing, and Data Science (BCD)10.1109/BCD57833.2023.10466329(208-215)Online publication date: 14-Dec-2023
https://doi.org/10.1109/BCD57833.2023.10466329
Show More Cited By

Index Terms

LANMC: LSTM-Assisted Non-Rigid Motion Correction on FPGA for Calcium Image Stabilization
1. Computing methodologies
  1. Machine learning
2. Hardware
  1. Communication hardware, interfaces and storage
    1. Signal processing systems
      1. Digital signal processing
  2. Integrated circuits
    1. Reconfigurable logic and FPGAs

Recommendations

Motion correction based on an appropriate system matrix for statistical reconstruction of respiratory-correlated PET acquisitions

Respiratory motion correction in positron emission tomography (PET) seeks to incorporate motion information into an image reconstruction algorithm by using the full counting statistics of an acquisition to generate a single, motion-free volume. Here, we ...
Spatio-temporal image registration for respiratory motion correction in pet imaging
ISBI'09: Proceedings of the Sixth IEEE international conference on Symposium on Biomedical Imaging: From Nano to Macro

Positron emission tomography (PET) is a molecular imaging technique which is now widely established as a powerful tool for diagnosing a variety of cancers. However, PET images are substantially degraded by respiratory motion to the extent that this may ...
Motion correction in positron emission tomography considering partial volume effects in optical flow estimation
ISBI'10: Proceedings of the 2010 IEEE international conference on Biomedical imaging: from nano to Macro

Motion correction in Positron Emission Tomography (PET) using optical flow estimation can lead to image artifacts due to Partial Volume Effects (PVE). These artifacts appear especially in cardiac gated PET images and cause blurred edges in the averaged ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

FPGA '19: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

February 2019

360 pages

ISBN:9781450361378

DOI:10.1145/3289602

General Chair:
Kia Bazargan
Univ. of Minnesota, USA
,
Program Chair:
Stephen Neuendorffer
Xilinx, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 February 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Science Foundation

Conference

FPGA '19

Sponsor:

SIGDA

FPGA '19: The 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays

February 24 - 26, 2019

CA, Seaside, USA

Acceptance Rates

Overall Acceptance Rate 125 of 627 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
610
Total Downloads

Downloads (Last 12 months)146
Downloads (Last 6 weeks)14

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen ZBlair GCao CZhou JAharoni DGolshani PBlair HCong J(2023)FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback ApplicationsIEEE Transactions on Biomedical Circuits and Systems10.1109/TBCAS.2023.326813017:2(169-179)Online publication date: Apr-2023
https://doi.org/10.1109/TBCAS.2023.3268130
Kim M(2023)Software Engineering for Data Intensive Scalable Computing and Heterogeneous Computing2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE)10.1109/ICSE-FoSE59343.2023.00006(54-68)Online publication date: 14-May-2023
https://doi.org/10.1109/ICSE-FoSE59343.2023.00006
Kundu ANguyen U(2023)PyTPU: Migration of Python Code for Heterogenous Acceleration with Automated Test Generation2023 IEEE/ACIS 8th International Conference on Big Data, Cloud Computing, and Data Science (BCD)10.1109/BCD57833.2023.10466329(208-215)Online publication date: 14-Dec-2023
https://doi.org/10.1109/BCD57833.2023.10466329
Abbas WMasip D(2022)Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A SurveyApplied Sciences10.3390/app1214687612:14(6876)Online publication date: 7-Jul-2022
https://doi.org/10.3390/app12146876
Song LChi YSohrabizadeh AChoi YLau JCong JAdler MIenne P(2022)Sextans: A Streaming Accelerator for General-Purpose Sparse-Matrix Dense-Matrix MultiplicationProceedings of the 2022 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays10.1145/3490422.3502357(65-77)Online publication date: 13-Feb-2022
https://dl.acm.org/doi/10.1145/3490422.3502357
Zhang QWang JKim MSpinellis DGousios GChechik MDi Penta M(2021)HeteroFuzz: fuzz testing to detect platform dependent divergence for heterogeneous applicationsProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3468610(242-254)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3468610
Chen ZBlair GBlair HCong J(2021)Fast Calcium Trace Extraction for Large-Field-of-View Miniscope2021 IEEE Biomedical Circuits and Systems Conference (BioCAS)10.1109/BioCAS49922.2021.9644936(1-5)Online publication date: 7-Oct-2021
https://doi.org/10.1109/BioCAS49922.2021.9644936
Chi YCong JLi Z(2020)Exploiting computation reuse for stencil acceleratorsProceedings of the 57th ACM/EDAC/IEEE Design Automation Conference10.5555/3437539.3437723(1-6)Online publication date: 20-Jul-2020
https://dl.acm.org/doi/10.5555/3437539.3437723
Lau JSivaraman AZhang QGulzar MCong JKim MRothermel GBae D(2020)HeteroRefactorProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380340(493-505)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380340
Chen ZBlair GBlair HCong JAlonso DQiu QReda SChen Y(2020)BLINKProceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design10.1145/3370748.3406552(217-222)Online publication date: 10-Aug-2020
https://dl.acm.org/doi/10.1145/3370748.3406552
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents