research-article

Open access

FusionTrack: Towards Accurate Device-free Acoustic Motion Tracking with Signal Fusion

Authors:

Jiarui Zhang,

Jiliang WangAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 20, Issue 3

Article No.: 71, Pages 1 - 30

https://doi.org/10.1145/3654666

Published: 06 May 2024 Publication History

PDF eReader

Abstract

Acoustic motion tracking is rapidly evolving with various applications. However, existing approaches still have some limitations. Tracking based on single-frequency continuous wave (CW) faces cumulative errors in tracking and limited accuracy in tracking the absolute location of the target. Tracking based on frequency-modulated continuous wave (FMCW) faces errors introduced by the Doppler and multipath effects. To overcome these limitations, we propose FusionTrack, a novel device-free motion-tracking approach that leverages the fusion of CW and FMCW signals. We eliminate the absolute tracking errors of FMCW-based tracking by compensating for Doppler frequency offsets with the results of CW-based relative tracking. Furthermore, we address the static multipath with down-sampling and filtering and mitigate the dynamic multipath with chirp aggregation. We employ a Kalman filter-based fusion of relative and absolute tracking to enhance accuracy further. We implement FusionTrack on Android smartphones for real-time tracking and perform extensive experiments. The results show that FusionTrack achieves real-time 1D tracking with an accuracy of 1.5 mm, which is 46% better than the existing approaches and extends the 1D tracking range to 2.2 m, which is 3.1× of the existing approaches. FusionTrack also achieves a 2D tracking accuracy of 4.5 mm.

1 Introduction

Acoustic motion tracking uses acoustic signals to accurately detect and track objects or human movements in diverse environments. This non-invasive and versatile approach has gained significant attention for its potential applications in domains such as human-computer interaction [15, 23, 33], health monitoring [22, 26, 28, 32, 34, 47], and mobile gaming [3, 45, 46].

However, existing device-free acoustic motion tracking approaches face challenges in achieving accurate tracking. Two prominent approaches are commonly used: based on single-frequency continuous wave (CW) and frequency-modulated continuous wave (FMCW). CW-based approaches estimate the relative distance change with motion-induced signal changes, such as Doppler frequency shifts [6, 25, 27] and phase changes [35]. These approaches can provide tracking results that are accurate for a short period, but they suffer from cumulative errors [35], which accumulate over time and lead to a significant tracking error. FMCW-based approaches estimate the absolute distance with the frequency difference between transmitted and received signals. For example, PDF [4] mixes the transmitted FMCW signal and the reflected FMCW signal to extract their frequency difference. It measures the distance from the transmitter to the target by calculating the phase change rate of the mixed signal. However, existing FMCW-based device-free tracking approaches [4, 5, 18] overlook the Doppler frequency offset during motion, leading to significant tracking errors. For example, the error can be 8 cm for motion with a velocity of 1m/s.

In this paper, we propose FusionTrack, a novel device-free motion-tracking approach that addresses these limitations through signal fusion. We design the transmitted signal as a combination of CW and FMCW signals.

When a target reflects the signal, we analyze the signal and develop a fusion scheme to track the target’s motion. Firstly, we mitigate the absolute distance errors due to Doppler frequency offset by enhancing the FMCW signal with the relative distance change derived from the CW signal. Secondly, we build a Kalman filter model to fuse the relative distance change and absolute distance, leveraging the benefits of both FMCW and CW signals, thus achieving accurate tracking over a long period.

FusionTrack addresses the following key challenges in achieving accurate absolute tracking with signal fusion:

(1)

Doppler effect: The target’s motion introduces a frequency offset in the received signal due to the Doppler effect, leading to significant errors in FMCW-based absolute distance estimation, which derives the absolute path length with the frequency difference between transmitted and received signals. We address this challenge by developing a Doppler compensation method to compensate for the Doppler frequency offset with the relative distance change derived from the CW signal. Doppler compensation effectively eliminates frequency offset and improves absolute tracking accuracy.

(2)

Multipath effect: Real-world applications involve reflections from the target and other static and dynamic multipaths. The multipath effect reduces the signal-to-noise ratio(SNR) of the received signal and introduces tracking errors. To address the static multipaths, we introduce a static multipath elimination method, eliminating the static multipath reflection with down-sampling and filtering. To address dynamic multipaths, we propose a time-domain aggregation method to aggregate the strength of multiple chirps with the help of relative tracking. This method amplifies the target reflection and diminishes the dynamic multipaths. These methods address errors introduced by the multipath effect and improve tracking performance under low SNR.

(3)

Weak signals and tracking errors: Despite mitigating errors from the Doppler and multipath effects, there are still errors in both relative and absolute tracking due to a weak reflection signal strength. To further refine the tracking results, we propose a Kalman filter-based method that combines the outputs of relative tracking and absolute tracking. The Kalman filter focuses on uncertainties in both tracking results, allowing for a more robust estimation. This fusion technique effectively addresses relative and absolute tracking errors and combines their strength.

The key contributions of FusionTrack are as follows:

–

We propose FusionTrack, a novel approach for accurate acoustic device-free motion tracking that leverages signal fusion. The main idea of FusionTrack is to combine accurate relative tracking with the CW signal and the ability to find the absolute location with the FMCW signal. FusionTrack successfully overcomes the limitations of CW-based and FMCW-based tracking and achieves accurate absolute tracking.

–

FusionTrack addresses practical challenges in device-free motion tracking. We introduce Doppler compensation to address tracking errors induced by the Doppler effect. Additionally, we address multipath effects by static multipath elimination based on down-sampling and filter and dynamic multipath mitigation with the aggregation of multiple chirps signals. Furthermore, we utilize the Kalman filter to combine the relative distance change and the absolute distance, improving the tracking accuracy with both tracking results.

–

We implement FusionTrack on PC and Android-based smartphones and conduct extensive experiments to evaluate its performance. The results demonstrate that FusionTrack achieves remarkable tracking accuracy of 1.5 mm in 1D motion tracking, which is 46% better than existing approaches. FusionTrack also achieves a tracking range of 2.2 m, 3.1× of existing approaches. We also achieve a 2D tracking accuracy of 4.5 mm. The results highlight the effectiveness and practicality of FusionTrack in real-world scenarios.

The remaining parts of this article are as follows: Section 2 provides an overview of the prior arts in acoustic motion tracking and discusses their limitations. Section 3 presents the design of absolute tracking with signal fusion. Section 4 introduces our approach to addressing static and dynamic multipaths. In Section 5, we propose the design for FusionTrack. Following this, Section 6 covers the implementation and evaluation of FusionTrack, with a comprehensive discussion of the results. Section 7 reviews the related work, and Section 8 discusses our work and provides some future work. Finally, in Section 9, we conclude our work.

2 Prior Arts and Limitations

This section provides an overview of existing approaches for acoustic motion tracking with different acoustic signals and discusses their limitations.

2.1 CW-Based Tracking

Continuous wave (CW) based motion tracking transmits a signal with a constant frequency, and the signal reflected by the target is:

\begin{equation} r(t) = A \cos \left(2\pi f_{CW} \left(t-\frac{d(t)}{c}\right)\right) \end{equation}

(1)

where A denotes the amplitude, $d(t)$ represents the length of the target reflection path, $f_{CW}$ is the signal’s frequency, and c is the speed of sound.

Existing CW-based tracking approaches focus on extracting the length of the reflection path from the received signal with different characteristics, such as Doppler frequency shift [6] and phase change [35]. For example, LLAP [35] proposes a method based on the phase change of the CW signal, achieving high tracking accuracy at the millimeter level. As shown in Figure 1, the received signal consists of components of the target reflection and other multipaths, such as the direct path. LLAP introduces a local extreme value detection method to estimate and remove static components, extracting the target reflection.

Fig. 1.

Limitations. To begin with, CW-based tracking approaches cannot derive the absolute path length. Because of the CW signal’s periodicity, the reflected signal’s phase is folded into a range of $2\pi$. Consequently, CW-based tracking approaches can only obtain relative distance changes by accumulating the phase change over time. Meeting this limitation, CW-based tracking approaches usually choose a fixed initial location or measure the initial location beforehand [45]. However, it adds to the burden on the user in real applications. LLAP proposes to estimate the absolute path length with multiple CW signals with different frequencies. However, the inaudible acoustic signal has limited tracking range and accuracy due to its weak strength and narrow bandwidth.

What’s more, CW-based tracking faces the issue of cumulative tracking errors caused by frequency offset, ambient noise, and multipath reflection in the received signal. These errors accumulate over time during relative tracking, which significantly affects the accuracy of CW-based tracking.

2.2 FMCW-Based Tracking

The FMCW signal is composed of periodic chirps with a linearly increasing frequency. The blue lines in Figure 2 illustrate the frequency-time distribution of an FMCW signal. The transmitted FMCW signal is:

\begin{equation} \left\lbrace \begin{aligned}& s(t) = \cos \left(2\pi \left(f_{\min } t+\frac{B}{2T}t^2\right)\right) ,& 0 \le t \lt T \\ & s(t+T) = s(t) \end{aligned} \right. \end{equation}

(2)

where $B = f_{\max } - f_{\min }$ is the bandwidth of one FMCW chirp and T is the length of a chirp duration. The frequency of $s(t)$ is $f_s(t) = f_{\min } + \frac{B}{T}t$.

Fig. 2.

The signal reflected by the target is a time-delay version of the transmitted signal, which is:

\begin{equation} r(t) = A s\left(t-\frac{d(t)}{c}\right) = A\cos \left(2\pi \left(f_{\min } \left(t-\frac{d(t)}{c}\right)+\frac{B}{2T}\left(t-\frac{d(t)}{c}\right)^2\right)\right) \end{equation}

(3)

where A is the amplitude, $d(t)$ is the length of the reflection path, and c is the speed of sound.

Existing work on FMCW-based tracking generally assumes that the target moves slowly [35] or the chirp duration is short enough [4]. Consequently, the target is approximately static within a chirp duration. Figure 2(a) shows the tracking model. Existing FMCW-based approaches analyze the frequency difference between the transmitted and the received signals [5], which is denoted as $\Delta f$, and derive the absolute path length with:

\begin{equation} d = \frac{\Delta f c T}{B} \end{equation}

(4)

Specifically, PDF [5] analyzes this frequency difference by multiplying the received signal by the transmitted signal and passing it through a low-pass filter, obtaining a mixed signal with a constant frequency of $f = \frac{Bd}{cT}$. The distance of the reflected path is derived by analyzing the frequency of the mixed signal based on examining its phase change rate. This approach has demonstrated accuracy at the millimeter level.

Limitations. The tracking model of existing FMCW-based approaches is designed for approximately static targets. The assumption of slow movement or short chirp duration is not valid for tracking a target that moves quickly. Under this condition, we need to consider the change in path length $d(t)$. The frequency of the reflected signal can be derived by taking the derivative of its phase, which is:

\begin{equation} f_r(t) = \frac{1}{2\pi } \frac{d \phi (r(t))}{d t} = \left(f_{\min }+\frac{B}{T}t\right) - f_0(t) \frac{v(t)}{c} - \frac{B}{T}\frac{d(t)}{c} \end{equation}

(5)

where $f_0(t) = f_{\min }+\frac{B}{T}(t - \frac{d(t)}{c})$ is the original frequency of the reflected signal, and $v(t)$ is the speed of the path length change.

The frequency difference between the transmitted signal and the reflected signal has two components, which can be expressed as:

\begin{equation} \Delta f = f_r - f_s = \Delta f_{ToF} + \Delta f_{D} \end{equation}

(6)

where $\Delta f_{ToF} = \frac{Bd(t)}{cT}$, which is introduced by the time of flight (ToF) of the target’s reflection, and $\Delta f_D = f_0(t) \frac{v(t)}{c}$, which is introduced by the Doppler effect during the target’s motion. Figure 2(b) visually illustrates the real frequency-time distribution of the received FMCW signal under the target’s motion, revealing the Doppler frequency offset $\Delta f_{D}$.

If we directly derive the path length d with Equation (4), the ignored Doppler effect will introduce distance error in motion tracking, which is:

\begin{equation} \varepsilon _d = \frac{\Delta f_{D} c T}{B} = \frac{f_0(t) v(t) T}{B} \end{equation}

(7)

The transmitted signal should be in the inaudible frequency band in real-world application scenarios to avoid disturbing users. Thus, the signal frequency is limited to 16-24 kHz, and the bandwidth B should not exceed 8 kHz. Additionally, the chirp duration should be long enough to prevent range ambiguity. For an indoor environment, the chirp duration should not be less than 0.02 s for a maximum path length of 6.8 m. Consequently, we set $f_{\min }$ = 16 kHz, B = 8 kHz, and T = 0.02 s to derive the minimum error in path length caused by the Doppler effect. Figure 3 shows the relationship between path length error $\varepsilon _d$ and the speed of path length change v. Because of the constraints in real applications, the tracking error introduced by the Doppler effect is significant. For example, if v > 1 m/s, the tracking error will be more significant than 8 cm, which is not affordable for real-world motion-tracking applications.

Fig. 3.

Additionally, we do experiments to investigate the tracking error caused by the Doppler effect. To achieve this, we moved the target back and forth over a distance of 10 cm and employed FMCW-based tracking with the model for static targets. For ground truth path length, we utilized an ORBBEC Astra Pro Plus[10] depth camera. Figure 4 shows the tracking result, which demonstrates that the tracking result is greater than the ground truth when the target moves away but smaller when the target approaches. The tracking error is at the centimeter level.

Fig. 4.

3 Accurate Absolute Tracking with Signal Fusion

FusionTrack focuses on combating the limitations of both CW-based and FMCW-based tracking, including the limited absolute tracking capacity of CW signals and FMCW-based tracking errors caused by the Doppler effect. The main idea is to combine CW signals and FMCW signals. To begin with, we accurately calculate the relative distance change from the CW signal. We then design a Doppler compensation technique with the help of the relative distance change to precisely eliminate the Doppler frequency offset introduced by the target’s motion, thus eliminating the tracking errors. Finally, we achieve accurate absolute distance estimation with frequency domain analysis of the received signal after Doppler compensation. In this chapter, we will introduce the methods in detail.

3.1 CW-Based Relative Tracking

We analyze the received CW signal’s phase to track the relative distance change. The received CW signal follows Equation (1). We extract the phase of it by I-Q demodulation. Specifically, we multiply the received CW signal by $\cos (2\pi f_{CW} t)$ and $-\sin (2\pi f_{CW} t)$ and apply a low-pass filter. The results of I-Q demodulation are $I(t) = A \cos (-2\pi f_{CW} \frac{d(t)}{c})$ and $Q(t) = A \sin (-2\pi f_{CW} \frac{d(t)}{c})$.

We construct a complex signal:

\begin{equation} Z(t) = I(t) + jQ(t) = Ae^{-j\left(2\pi f_{CW} \frac{d(t)}{c}\right)} \end{equation}

(8)

where $d(t)$ is the path length of the target reflection, c is the speed of sound, and $f_{CW}$ is the frequency of the CW signal. The phase of $Z(t)$ changes with the variation of path length $d(t)$.

In real-world application scenarios, the multipath effect should be considered. Figure 5 illustrates the complex signal $Z(t)$ in the I-Q plane. Static multipaths introduce static phase components, while dynamic multipaths introduce dynamic phase components. Accurate elimination of static multipath components is crucial for precise phase estimation and effective relative tracking.

Fig. 5.

LLAP proposes a local extreme value detection (LEVD) method, estimating the static phase components with the local maxima and minima of $I(t)$ and $Q(t)$. The static component is updated when the local maxima or minima are detected. In real-world applications, the noise in the received signal leads to errors in distance change estimation. The noise will introduce false maxima and minima in $I(t)$ and $Q(t)$, leading to errors in estimating static components. Moreover, LEVD analyses $I(t)$ and $Q(t)$ separately, introducing errors when their variation does not follow the model. These errors are cumulative over time, leading to a more significant error in relative tracking.

In FusionTrack, we propose a state machine model to accurately estimate the phase change with local maxima and minima of $I(t)$ and $Q(t)$. Figure 6 shows the state machine, a computational model representing the system’s behavior based on different states and transitions [7]. The states represent different conditions in which the system can be, and the transitions denote the conditions under which the state machine transits from one state to another. This state machine provides an integrated analysis of the phase change with $I(t)$ and $Q(t)$.

Fig. 6.

Our state machine model has five states, including an initial state and four states corresponding to the maximum or minimum of $I(t)$ or $Q(t)$. The state transitions occur when the maximum or minimum of $I(t)$ or $Q(t)$ is detected. Due to the phase continuity, each non-initial state has only two possible state transitions, corresponding to a phase change of $\frac{\pi }{2}$ or $-\frac{\pi }{2}$. This state machine model considers phase continuity and mitigates phase errors introduced by noise in $I(t)$ or $Q(t)$. It has two functions: one is to estimate the static component accurately, and the other is to calculate the dynamic component’s phase roughly. When the state transition occurs, the estimated static component $Z_{static}$ and phase change $\phi$ are updated. We further derive the precise phase change $\hat{\phi }(t)$ with the phase of $Z_{target} = Z - Z_{static}$. The roughly estimated phase change also solves the phase ambiguity of $Z_{target}$.

Algorithm 1 demonstrates the relative tracking algorithm with pseudo-code. There are two functions in the Algorithm. state_machine_transition is the state transition of our state machine model. The inputs are the current state and the transition condition. The outputs are the next state and the phase change. phase_refinement is the function of the refinement of phase estimation. The inputs are the dynamic component $Z_{dynamic}$ and the roughly estimated phase $\phi$. The output is the precise phase change $\hat{\phi }$.

This state machine model effectively mitigates phase errors caused by noise in $I(t)$ and $Q(t)$ because noise-induced local maxima and minima do not trigger state transitions. Therefore, we can achieve accurate relative tracking with a linear time complexity with the state machine model.

3.2 Doppler Compensation

Section 2.2 highlights the issue of absolute distance errors in FMCW-based tracking caused by the Doppler frequency offset. The error limits the ability to accurately calculate the absolute path length of the moving target’s reflection. To address this problem, we propose compensating for the Doppler frequency offset with the help of relative tracking and derive the absolute path length accurately.

We decompose the absolute path length, $d(t)$, into two components, which is:

\begin{equation} d(t) = d_0 + \Delta d(t) \end{equation}

(9)

where $d_0$ represents the absolute distance at time $t_0$, i.e., $d_0 = d(t_0)$, and $\Delta d(t)$ indicates the relative distance change comparing with $d_0$.

With the relative tracking we discuss in Section 3.1, we can accurately estimate $\Delta d(t)$ from the received single-frequency CW signal. Since the target’s motion introduces the Doppler effect, we construct a pseudo-transmitted signal with the help of relative tracking to compensate for the Doppler frequency offset. Specifically, we add a time delay to the transmitted signal according to the relative change in path length $\Delta d(t)$. The pseudo-transmitted signal is:

\begin{equation} s_c(t) = s\left(t - \frac{\Delta d(t)}{c}\right) = \cos \left(2\pi \left(f_{\min } \left(t-\frac{\Delta d(t)}{c}\right)+\frac{B}{2T}\left(t-\frac{\Delta d(t)}{c}\right)^2\right)\right) \end{equation}

(10)

The frequency of $s_c$ is:

\begin{equation} f_{s_c}(t) = \left(f_{\min } + \frac{B}{T}\left(t-\frac{\Delta d(t)}{c}\right)\right) \left(1-\frac{v(t)}{c}\right) \end{equation}

(11)

We compare the frequency difference between the target-reflected signal and the pseudo-transmitted signal to verify the effectiveness of Doppler compensation. The frequency difference is:

\begin{equation} f_{s_c}(t) - f_r(t) = \frac{Bd_0}{cT} \left(1-\frac{v(t)}{c}\right) \approx \frac{Bd_0}{cT} = \Delta f_{ToF} \end{equation}

(12)

where $v(t)$ is the speed of the path length change.

In motion tracking scenarios, the target motion speed is much slower than the speed of sound (typically slower than 4 m/s [6]). Therefore, the frequency difference approximately equals $\frac{Bd_0}{cT}$. Consequently, with the relative tracking and the construction of the pseudo-transmitted signal, we can successfully eliminate the frequency offset introduced by the Doppler effect and accurately derive the absolute path length from the frequency difference.

Figure 7 gives an example of Doppler compensation and reveals the frequency-time distribution of $s(t)$, $r(t)$, and $s_c(t)$. This figure shows that the frequency of the target’s reflected signal does not linearly increase with time because of the Doppler effect. Moreover, our Doppler compensation approach effectively compensates for the Doppler frequency offset, and the pseudo-transmitted signal keeps a constant frequency difference with the target reflected signal.

Fig. 7.

3.3 Absolute Tracking

The Doppler compensation method successfully eliminates the Doppler frequency offset and makes it possible to calculate the path length accurately. In this section, we propose a frequency-domain cross-correlation approach to calculate the frequency difference between the reflected signal r and the pseudo-reflected signal $s_c$ accurately and efficiently, thus achieving accurate absolute tracking.

Figure 8 depicts the frequency-domain cross-correlation process. To begin with, we perform the Fast Fourier Transform (FFT) to $r(t)$ and $s_c(t)$ to derive the frequency spectrum of these signals, which are denoted as $R(f)$ and $S_c(f)$, respectively.

Fig. 8.

From Equation (12), we know that there is a constant frequency difference between r and $s_c$. Therefore, $R(f)$ and $S_c(f)$ meets:

\begin{equation} R(f) = S_c(f+\Delta f_{ToF}) \end{equation}

(13)

We utilize cross-correlation of the frequency spectrum of R and $S_c$ to extract the constant frequency difference. The cross-correlation function is defined as:

\begin{equation} Cor(\Delta f) = \sum _{f = f_{lb}}^{f_{ub}} R^*(f) \cdot S_c(f+\Delta f) \end{equation}

(14)

where $f_{lb}$ and $f_{ub}$ are the correlation’s lower and upper bound, respectively, and $R^*$ is the complex conjugate of R.

According to Equation (13) and Equation (14). When $\Delta f = \Delta f_{ToF}$ is satisfied, the magnitude of $Cor(\Delta f)$ has a maximum value, i.e.,:

\begin{equation} |Cor(\Delta f_{ToF})| = \sum _{f = f_{lb}}^{f_{ub}}|R(f)|^2 = \max (|Cor(\Delta f)|) \end{equation}

(15)

Consequently, we can find the frequency difference introduced by the time of flight (ToF) with the highest magnitude of $Cor(\Delta f)$, i.e.,:

\begin{equation} \Delta f_{ToF} = \mathop {\mathrm{argmax}}\limits _{\Delta f} |Cor(\Delta f)| \end{equation}

(16)

In this way, we calculate the absolute path length with:

\begin{equation} d_0 = \Delta f_{ToF} \frac{cT}{B} \end{equation}

(17)

where B is the bandwidth, T is the frequency of one chirp duration, and c is the speed of sound.

Time complexity and distance resolution. We propose some designs to reduce the time complexity and improve the distance resolution. To begin with, we adopt FFT and IFFT to reduce the time complexity of the cross-correlation, which is:

\begin{equation} Cor(\Delta f) = \text{IFFT}(\text{FFT}(R(f))^* \cdot \text{FFT}(S_c(f))) \end{equation}

(18)

If the input of the frequency-domain cross-correlation has N frequency bins, the time complexity of this implementation of cross-correlation is $O(N \log N)$. Because of the limited frequency band of the transmitted FMCW signal, we can truncate the frequency spectrum and only keep the valid portion of the frequency spectrum. Considering the frequency offset, we set the upper and lower bounds of the cross-correlation to $f_{lb} = f_{\min }-0.5B$ and $f_{ub} = f_{\min }+1.5B$, respectively. Therefore, we only perform a correlation in a bandwidth of $2B$. In contrast, the traditional signal mixing and dechirp methods require spectrum analysis in the entire frequency band. Therefore, our method can reduce the complexity of absolute tracking and eliminate noise interference in other frequency bands.

Moreover, to improve the resolution of absolute tracking, we adopt zero-padding during the IFFT process, which is equivalent to performing interpolation to the frequency spectrum. The zero-padding also adds to the time complexity of absolute tracking. Therefore, there is a trade-off between distance resolution and calculation time. Meeting with this problem, we set the bandwidth B = 3 kHz and the zero-padding coefficient of N = 64. The resolution of the path length is $\frac{c}{NB}$ = 1.77 mm. This setting makes FusionTrack support absolute tracking at the millimeter level while maintaining real-time tracking.

4 Combating Multipath Effect

In real motion tracking scenarios, the received signal contains the signal reflected by the target and signals from static and dynamic multipaths. The presence of multipath signals complicates the extraction of the target reflection and introduces errors in motion tracking. To overcome these challenges of the multipath effect, we propose static multipath elimination and dynamic path mitigation approaches in this section.

4.1 Static Multipath Elimination

The static multipaths include a direct path and the reflection of static reflectors in the environment. The signal from the direct path is usually strong, overshadowing the target reflection signal because of its high power. The other static multipaths also introduce errors in absolute tracking.

Existing work has proposed various methods to eliminate static multipath signals based on time-domain subtraction. The first method is background multipath subtraction [15], which measures the environmental signal without target reflection in advance and subtracts the environmental signal during tracking. However, this method lacks stability. First, it is necessary to measure the background signal again after the location of the system changes. Second, although the system’s location remains unchanged, the external environment also changes over time, limiting the performance of eliminating the static multipath.

The second method is to directly subtract the signals of the two adjacent chirp cycles [5], which is:

\begin{equation} y(t) = r(t) - r(t-T) \end{equation}

(19)

where T is the length of the duration of one chirp in the FMCW signal. This method can mitigate static multipath signals. However, it retains the difference between dynamic multipath signals from two cycles, reducing the strength of the signals from the dynamic paths and affecting absolute tracking.

In FusionTrack, we propose a novel method to eliminate the static multipath by down-sampling and filtering. This results in better performance in suppressing static multipath signals while retaining dynamic signals.

To distinguish static and dynamic multipath reflections, we utilize the periodicity of the transmitted signal and down-sample the received signal at a sampling rate of $1/T$. This down-sampling process results in $N = f_s T$ signals, where the i-th signal is represented by the equation:

\begin{equation} r^{(i)}(k) = r(kT+t_i) = As\left(kT+t_i-\frac{d(kT+t_i)}{c}\right) = As\left(t_i-\frac{d(kT+t_i)}{c}\right) \end{equation}

(20)

where k is an integer and $r^{(i)}(k)$ is a down-sampled version of $r(t)$, with a start time of $t_i$ and a sampling rate of $1/T$.

Static multipaths have constant reflection path lengths, which make the down-sampled signals constant. On the other hand, dynamic multipaths derive non-constant down-sampled signals. Therefore, we can differentiate between static and dynamic multipaths with the frequencies of reflected signals.

After signal down-sampling, we apply a high-pass filter to each down-sampled signal to extract the dynamic components. The filter’s design is crucial for eliminating the static components. We aim to design a filter that can eliminate the static components with a frequency of approximately zero and retain the dynamic components. The high-pass IIR filter meets our needs. IIR filters are linear time-invariant systems whose outputs are formed from the inputs of the system and the previous outputs of the system. They can be described as:

\begin{equation} y(t) = \sum _{n=1}^{N} a_n y(t-nT) + \sum _{m=0}^{M} b_m r(t-mT) \end{equation}

(21)

where two sets of filter coefficients, the feedback coefficients $a_n$ and the feedforward coefficients $b_m$, decide the performance of the filter. The order of the filter is M. Specifically, we utilize a second-order high-pass Butterworth filter for static path elimination. We set the cutoff frequency to 4 Hz under a down-sampling rate of 50Hz.

Figure 9(a) shows the frequency response of our design and direct subtraction. We can see that although direct subtraction can eliminate static components with a frequency of approximately zero, it also reduces the energy of dynamic components with higher frequencies, which limits the performance of motion tracking. In contrast, our design performs better in keeping the dynamic components because the dynamic components whose frequency is higher than the cutoff frequency are perfectly preserved. Note that because of the down-sampling, the strength of dynamic components whose frequency is an integer multiple of the sampling rate is also attenuated.

Fig. 9.

Figure 9(b) reveals the results of absolute tracking using different static path elimination methods. Without elimination, the peak of the direct path, whose length is nearly zero, is much higher than the target reflection, overshadowing the peak of the target’s reflection. When direct subtraction is applied, the static and the dynamic components are reduced. In contrast, the dynamic components are perfectly preserved after applying our design of static multipath elimination, which improves the performance of absolute tracking.

We further analyze the performance of static path elimination in FMCW-based tracking. To simplify the analysis, we assume that the velocity remains the same during a chirp duration. Hence, the path length of the target reflection is $d(t) = d_0 + vt$. The frequency of the i-th down-sampled signal is given by:

\begin{equation} f_r^{(i)}(v) = \frac{v}{c} \left[\frac{B}{T} \frac{d_0+vt_i}{c} - f_{\min } - \frac{B}{T} t_i\right] \end{equation}

(22)

where v is the speed of path length change, c is the speed of sound, B is the bandwidth of a chirp, and T is the length of a chirp. We can see that $d_0$ and v are relative to the frequency. Moreover, the frequency of different down-sampled signals for the same target differs. We examine the performance of static multipath elimination by calculating the average amplitude of an FMCW chirp.

Figure 10(a) compares the normalized amplitude of the signals after applying different filters. It shows that our filter design performs better than the existing method of direct subtraction. We also calculate the power amplification of our design compared to direct subtraction. As Figure 10(b) shows, our design has a maximum amplification of 15 dB and an average amplification of 3.75 dB. Therefore, our static path elimination design performs better in keeping dynamic components, which is the basis for improving motion tracking range and accuracy.

Fig. 10.

We also examine the performance of our static path elimination in real-world application by experiment. In this experiment, the target moves back and forth for 10 cm. The results of applying different filters are shown in Figure 11. Each column in this figure shows the absolute tracking result in a chirp duration after removing static paths. The color corresponds to the amplitude. The results show that our design performs better in tracking because it keeps the dynamic components better, especially for the target that moves slowly.

Fig. 11.

4.2 Mitigating the Dynamic Multipath

Although our static path elimination performs well, the dynamic components introduced by dynamic multipaths still exist because they also introduce frequency changes in the down-sampled signal. Different path lengths of dynamic multipaths result in multiple peaks in the result of absolute tracking, and some of the peaks of multipaths are higher than the peak of the target reflection, resulting in errors in absolute tracking.

Figure 12(a) shows an example of the absolute tracking result. The figure shows the existence of a dynamic multipath with a length of about 1 m, which results in incorrect absolute tracking results.

Fig. 12.

We find some differences between the target reflection and the dynamic multipaths. First, the peaks of the target reflection appear continuously, whereas the peaks of dynamic multipaths appear occasionally. Second, the path length change of the dynamic multipaths is usually inconsistent with the relative tracking. Therefore, we propose a time-domain aggregation method to mitigate dynamic multipaths with the help of relative tracking. The main idea of time-domain aggregation is aggregating multiple chirps to mitigate the dynamic multipaths and achieve a better performance of absolute tracking.

As in Equation (9), the path length d(t) can be denoted as $d(t) = d_0 + \Delta d(t)$. As long as we know the relative distance change to $d_0$ during any chirp period, we can calculate $d_0$ using absolute distance analysis, regardless of whether t is in the same chirp duration as $t_0$. Therefore, we can utilize signals in the same chirp duration with $t_0$ and other windows to calculate the absolute distance. Therefore, we aggregate the energy of multiple signals by adding the result of the absolute tracking of multiple signals. We name this process time-domain aggregation. The aggregated correlation profile is:

\begin{equation} Cor_{agg}(\Delta f, n) = \sum _{k=0}^{N} Cor(\Delta f, n-k) \end{equation}

(23)

where $Cor(\Delta f, n)$ is the magnitude of the frequency-domain correlation with an input of the n-th chirp.

Figure 13 shows the process of time-domain aggregation. The first figure shows the results of absolute tracking derived from different chirp durations; it shows that the multipath peaks may be higher than that of the target reflection. The time-domain aggregation aligns the peaks of the target reflection and scatters the peaks of dynamic multipath, amplifying the target reflection. The correlation results after aggregation reveal a high peak corresponding to the target reflection. The tracking results after time-domain aggregation are shown in Figure 12(b). It shows that it performs well in mitigating the dynamic multipath and amplifying the target reflection, contributing to high accuracy in absolute tracking.

Fig. 13.

There is a trade-off between performance and time complexity of time-domain aggregation. If too many chirps are aggregated, it results in high timer complexity and time delay, which affects the system’s real-time property. On the other hand, aggregating fewer chirps affects the tracking accuracy. To strike a balance between the two, we have decided to aggregate 16 chirps, which leads to good tracking accuracy and a delay of 32 ms. This delay does not hamper the system’s real-time property.

Although we mitigate the dynamic multipath with time-domain aggregation, the strength of the multipath signal may still be stronger than the target reflection. We further utilize the continuity of path length change to select the peaks corresponding to the target reflection, improving the tracking accuracy. Specifically, when the spectrum has one prominent peak, we adopt the distance corresponding to this peak as the result of absolute tracking. When there is more than one prominent peak, we choose the one that satisfies the relative tracking result as the result of absolute tracking. With time-domain aggregation and multipath selection, we successfully achieve accurate absolute tracking of the target reflection.

5 System Design

5.1 Overview

Figure 14 shows the workflow of FusionTrack, which consists of five main components.

Fig. 14.

(1)

Signal design. We design FusionTrack’s transmitted signal by combining a continuous wave (CW) signal and a frequency-modulated continuous wave (FMCW) signal in different frequency bands. We demonstrate how to design the transmitted signal to reduce frequency leakage and ensure precise alignment.

(2)

Relative tracking. FusionTrack calculates the relative change in path length utilizing the method described in Section 3.1. This relative tracking is achieved by examining the phase shift of the CW signal with a state machine model.

(3)

Static multipath elimination. FusionTrack utilize the static multipath elimination method described in Section 4.1 to eliminate the impact of static multipath reflection. The down-sampling and filtering method eliminates static multipath reflections while preserving the dynamic components of the received signal.

(4)

Absolute tracking. FusionTrack accurately calculates the absolute path length by combining FMCW and CW signals. The absolute tracking module incorporates Doppler compensation and dynamic multipath mitigation to ensure accurate absolute tracking. As described in Section 3.2, a pseudo-transmitted signal is generated to cancel out the Doppler frequency offset with the assistance of the relative tracking. The absolute path length is then calculated by performing frequency-domain cross-correlation between the received and pseudo-transmitted signals. Additionally, as described in Section 4.2, the time-domain aggregation of consecutive chirps amplifies the target reflection and mitigates dynamic multipaths.

(5)

Kalman filter. The results of both relative and absolute tracking have errors, and they are inconsistent with each other. FusionTrack adopt a Kalman filter to combine relative and absolute tracking results and correct for their errors, improving tracking accuracy and robustness.

The signal design and the Kalman filter are discussed in the following parts.

5.2 Signal Design

We design the transmitted signal of FusionTrack to combine CW and FMCW signals. Figure 15 shows the frequency-time distribution of the transmitted signal. Two types of signals are set in different frequency bands to avoid interference. Moreover, the transmitted FMCW signal consists of alternated up-chirps and down-chirps. This signal design provides several advantages over repeated chirps, which are discussed in the following parts of this section.

Fig. 15.

5.2.1 Reduced Spectral Leakage.

As shown in Figure 16(a), repeated chirps cause substantial spectral leakage when the frequency hops from $f_{\text{max}}$ to $f_{\text{min}}$, leading to periodic audible signals that disturb users. The common solution to address spectral leakage is applying windows to the transmitted signal (e.g., Hanning window [18]). But it also results in a power reduction, which reduces the SNR and the tracking accuracy.

Fig. 16.

In contrast, as shown in Figure 16(b), our signal design avoids power leakage, maintaining high signal power while remaining inaudible. This characteristic is crucial for practical applications in acoustic tracking.

5.2.2 Accurate Signal Alignment.

Accurate synchronization is crucial for motion tracking but challenging for mobile devices because the speakers and microphones are not perfectly synchronized. We use the direct path signal for synchronization to address this issue because it has a constant time of flight (ToF) and a high power level.

The imperfect synchronization of speakers and microphones leads to a constant central frequency offset (CFO) that introduces errors in cross-correlation-based synchronization [21]. To address the challenge of CFO, FusionTrack proposes a fine alignment method using alternated up-chirps and down-chirps after a coarse alignment by cross-correlation.

Fine alignment is achieved through a dechirp process, as shown in Figure 17. We multiply the received down-chirp with the transmitted up-chirp and vice versa. We then apply the fast Fourier transform (FFT) to the dechirped signal to analyze its frequency spectrum. For an input down-chirp with CFO and time offset (TO), the peak frequency in the spectrum is given by $f_{p1} = 2f_c + \text{CFO} + \frac{B}{T} \times \text{TO}$. For an input up-chirp, the peak frequency is $f_{p2} = 2f_c + \text{CFO} - \frac{B}{T} \times \text{TO}$. Therefore, we can derive the CFO and TO by:

\begin{align} CFO =&\, \frac{f_{p1}+f_{p2}}{2} - 2f_c \end{align}

(24)

\begin{align} TO =&\, \frac{(f_{p2}-f_{p1})T}{2B} \end{align}

(25)

where $f_c = \frac{f_{\min }+f_{\max }}{2}$, T is the length of a chirp duration, and B is the bandwidth.

Fig. 17.

In real-world applications, the multipaths should be considered. The frequency spectrum may exhibit multiple peaks corresponding to different reflection paths. To identify the peak of the direct path signal, we look for the spectrum peak with the smallest TO.

5.3 Kalman Filter

Relative tracking provides the result of the change in relative path length with high accuracy in a short period, but it has cumulative errors over time. There are also occasional errors in the results of absolute tracking due to dynamic multipaths and noise. The errors in both tracking results make them inconsistent and reduce the accuracy. We introduce a Kalman filter approach to fuse two tracking results to improve tracking accuracy further.

The Kalman filter [14] is an efficient recursive algorithm that estimates the state of a system by combining measurements with predictions based on a mathematical model. It is helpful to deal with noisy measurements, as it can effectively filter out noise and provide accurate estimations of the underlying system state.

Kalman filter operates in two steps: prediction and update. In the prediction step, it predicts the next state of the system based on the previous state using a mathematical model. In FusionTrack, we build a prediction model for the absolute path length. There are three variables in our model. d denotes the absolute path length, $\dot{d}$ denotes the difference of d, i.e., the change in relative path length, and $\ddot{d}$ denotes the second order difference of d, i.e., acceleration. The prediction model is:

\begin{align} \hat{d}_t =&\, {d}_{t-1} + {\dot{d}}_{t-1} \nonumber \nonumber\\ \hat{\dot{d}}_t =&\, {\dot{d}}_{t-1} + {\ddot{d}}_{t-1} \end{align}

(26)

\begin{align} \hat{\ddot{d}}_t =&\, {\ddot{d}}_{t-1} \end{align}

(27)

where $\hat{\cdot }$ denotes the prediction of a variable. This prediction can be represented in matrix form as:

\begin{equation} \begin{aligned}\hat{\mathbf {x}}_t = \mathbf {A}\mathbf {x}_{t-1} \end{aligned} \end{equation}

(28)

where $\mathbf {x}_t = [ d_t,\dot{d}_t,\ddot{d}_t ]^T$, and $\mathbf {A} = \left[ \begin{array}{c c c} 1 & 1 &0 \\ 0 & 1 &1 \\ 0 & 0 &1 \end{array} \right]$.

In the update step, the Kalman filter combines the predicted state with the real measurements to improve the estimation. In our model, the absolute tracking results ($z_t$) and the relative tracking results ($\dot{z}_t$) are considered to be real measurements of d and $\dot{d}$, respectively. Thus, the update step is:

\begin{align} \hat{\mathbf {P}}_t =&\, \mathbf {A}\mathbf {P}_{t-1}\mathbf {A}^T+\mathbf {Q} \nonumber \nonumber\\ \mathbf {K} =&\, \hat{\mathbf {P}}_t\mathbf {H}^T(\mathbf {H}\hat{\mathbf {P}}_t\mathbf {H}^T+\mathbf {R})^{-1} \nonumber \nonumber\\ \mathbf {x}_t =&\, \hat{\mathbf {x}}_t + \mathbf {K}(\mathbf {z}_t - \mathbf {H}\hat{\mathbf {x}}_t)\nonumber \nonumber\\ \mathbf {P}_t =&\, (I-\mathbf {K}\mathbf {H})\hat{\mathbf {P}}_t \end{align}

(29)

where $\mathbf {z}_t = [ z_t, \dot{z}_t ]^T$, $\mathbf {H} = \left[ \begin{array}{c c c} 1 & 0 & 0 \\ 0 & 1 & 0 \\ \end{array} \right]$ describes the relationship between $\mathbf {x}_t$ and $\mathbf {z}_t$. $\mathbf {P}$ represents the uncertainty of the predicted state, $\mathbf {Q}$ represents the uncertainty of the model, and $\mathbf {R}$ represents the uncertainty of the measurements.

Figure 18 compares the motion tracking results of three methods. The first one is relative tracking with the information of the initial absolute distance. The second one is the absolute tracking. The last one is combining relative and absolute tracking with the Kalman filter. The tracking results demonstrate that the Kalman filter can mitigate the cumulative error of relative tracking and improve the accuracy of motion tracking.

Fig. 18.

6 Implementation & Evaluation

6.1 Implementation

We implement two versions of FusionTrack: an offline version with Matlab on PC and an online version on Android smartphones. The online version on the Android platform can support real-time absolute tracking of the user’s motion and draw the 2D trajectory on the screen.

The frequency of the transmitted CW signal is 22 kHz, and the frequency band of the FMCW signal is 18 kHz to 21 kHz. The signals in the ultrasonic band, together with our signal design, ensure the inaudibility of our transmitted signal. The sampling rate of the received signal is 48 kHz. The length of an FMCW chirp is $T =$ 20 ms, which facilitates a maximum path length of $cT_{chirp} =$ 6.8 m and an update rate of 50 Hz.

As Figure 1 shows, we implement FusionTrack with one speaker and two microphones on the smartphone. We find that the two microphones on a smartphone have similar performance, but the two speakers usually have different performances. We utilize the speaker with better performance to transmit the signal for better tracking accuracy.

6.2 1D Tracking

We first evaluate the performance of FusionTrack under 1D motion tracking. Figure 19(a) shows the setup of the evaluation of the 1D tracking. We conduct experiments with an Android phone, using the bottom speaker to send the signal and the bottom microphone to receive the signal reflected by the target. We measure the distance between the smartphone and the target with an ORBBEC Astra Pro Plus [10] depth camera to provide the ground truth of the distance to the target, which has an accuracy of sub-millimeter level. At the same time, we use a coordinate paper whose length of each grid is 1 mm to provide an mm-level baseline of the moving distance. In our evaluation, the target moves back and forth with a fixed distance several times.

Fig. 19.

We evaluate 1D tracking with two metrics:

(1)

Relative distance change error: The relative distance change error concentrates on the relative distance change from one ending point to the next ending point of the 1D motion and ignores the absolute distance of the ending points. We calculate the relative tracking error by comparing the relative distance change of the tracking result and the ground truth moving distance.

(2)

Absolute distance error: Absolute distance error concentrates on the absolute distance at each ending point of the motion. We calculate this error by comparing the absolute distance of each ending point and the benchmark absolute distance from the smartphone to the target.

The results of our 1D motion tracking evaluation are listed below.

a) The relative distance change error of FusionTrackis 1.5 mm. We repeatedly move the target between 30 and 40 cm to the smartphone and conduct motion tracking with FusionTrack. Figure 20(a) shows the CDF of the relative distance change errors. The average error of FusionTrack is 1.5 mm, and the 90th percentile error is 2.7 mm. The results show that FusionTrack can achieve relative tracking with high accuracy.

Fig. 20.

b) The absolute distance error of FusionTrackis 1.3mm. Figure 21(a) shows the CDF of the absolute distance error when the target moves from 30 cm to 40 cm. The average error is 1.3 mm, and the 90th percentile error is 2.3 mm. The results show that FusionTrack also has a good performance in absolute tracking, which is the basis for the localization of the moving target.

Fig. 21.

c) Impact of the target’s distance. We evaluate the impact of the target’s distance by changing the distance of the target’s motion. We keep the length of the target’s motion to 10 cm and change the initial distance of the target. Figure 20(b) shows the average relative tracking error with different initial distance of the target, and Figure 21(b) shows the average absolute tracking error. The relative error is lower than 5 mm when the initial distance is lower than 220 cm, and the absolute tracking error is lower than 4 mm when the initial distance is lower than 220 cm. The results show that FusionTrack performs well in motion tracking. We also observe that the error increases when the target is too close or too far. When the target is too close, the impact of the target’s size increases. When the target is far away, the SNR of the reflected signal decreases, resulting in more significant errors.

d) Impact of the length of motion. We evaluate the impact of the target’s motion length by evaluating the tracking error when the target moves with different lengths. The initial distance is 30 cm, and the moving distance varies from 10 cm to 130 cm at intervals of 10 cm. Figure 22 shows the absolute tracking error under different motion lengths. The results show that the absolute tracking error is less than 2 mm when the length of motion is less than 100 cm. It means that FusionTrack performs well in tracking the target that undergoes a significant motion. The reason for a larger tracking error when the length is more significant than 120 cm is that the reflected signal strength is significantly reduced, which limits the relative and absolute tracking performance.

Fig. 22.

e) Impact of the target type. We evaluate FusionTrack with three different target types, including a 10 mm × 10 mm board, a hand, and a finger. Figure 23 shows the absolute tracking error of the three types of targets. The best average errors with a board, a hand, and a finger are 1.03 mm, 2.18 mm, and 2.11 mm, respectively. Moreover, FusionTrack achieves an absolute tracking error of lower than 4 mm for different target types. The results show that FusionTrack can track different types of targets with different sizes. The reason for lower accuracy and shorter range when tracking a hand or a finger is that the strength of the reflected signal is lower for a smaller target. Despite the weaker reflected signal, we can achieve a tracking range of 70 cm.

Fig. 23.

f) Comparison with other methods. We also implement LLAP [35] and PDF [5] to compare the performance of FusionTrack with them. Figure 20 and Figure 21 also show the relative and absolute tracking errors of PDF and LLAP. The average relative tracking errors of PDF and LLAP are 3.7 and 2.8 mm, while FusionTrack has an error of 1.5 mm. Therefore, FusionTrack has an improvement of 46% compared to existing work. We also examine the absolute tracking of PDF and LLAP. The absolute distance errors of PDF and LLAP are 2.6 mm and 3.5 mm, respectively. The reason for a larger absolute distance error for LLAP is that the limited bandwidth causes a low accuracy in absolute distance estimation. Furthermore, the tracking range of LLAP is 30 cm, which is limited by the decreased signal strength of signals in different frequency bands. The tracking range of PDF is 70 cm, which is limited by the low SNR when the target is far away. FusionTrack achieves a tracking range of 220 cm, 3.1× of existing works. Therefore, with the fusion of CW and FMCW signals, FusionTrack performs better than existing work in tracking accuracy and tracking range.

6.3 Ablation Study

We conduct an ablation study to evaluate the performance of each module of FusionTrack. Specifically, we construct some subsystems by removing one or more modules from the system and verifying their performance in motion tracking. We jointly verify the following subsystems:

(1)

Pure relative tracking.

(2)

Absolute tracking without Doppler compensation.

(3)

Absolute tracking without dynamic multipath mitigation.

(4)

Absolute tracking without Kalman filtering.

(5)

The complete system of FusionTrack.

Table 1 shows the specific settings for the ablation study. The symbols + and - denote whether the system has the corresponding module. Due to the subsequent modules requires the previous module’s output as input, removing one of the modules to create subsystems is impossible. Therefore, we construct the subsystems by keeping the former parts of FusionTrack. The abbreviations in Table 1 represent relative tracking, absolute tracking, static multipath elimination, Doppler compensation, dynamic multipath mitigation, and Kalman filtering, respectively. We evaluate the relative and absolute tracking errors of these subsystems. Figure 24 shows the results of the ablation study.

Table 1.

Index	Description	RT	AT	SE	DC	DM	KF
1	relative tracking	+	–	–	–	–	–
2	without Doppler compensation	–	+	+	–	–	–
3	without dynamic mitigation	+	+	+	+	–	–
4	without Kalman filter	+	+	+	+	+	–
5	running all modules	+	+	+	+	+	+

Table 1. Settings of Ablation Study

Fig. 24.

Figure 24(a) shows the relative and absolute tracking errors under different settings. The relative tracking of FusionTrack performs well, with a relative tracking error of 1.22 mm. However, only the relative tracking cannot derive the absolute distance. The absolute tracking error without Doppler compensation is 2.98 mm. The errors are reduced to 2.29 and 1.50 mm with the help of Doppler compensation and dynamic multipath mitigation. After the Kalman filter, the absolute distance is 1.16 mm. The results reveal that each module of FusionTrack contributes to accurate absolute motion tracking.

Figure 24(b) shows the tracking range of FusionTrack, the maximum range with a tracking error under 5mm. Only taking relative tracking cannot derive the absolute distance. The tracking range without Doppler compensation is 90 cm. The range is improved to 150 cm and 220 cm with the help of Doppler compensation and dynamic multipath mitigation. After the Kalman filter, the tracking range is 230 cm. The results reveal that Doppler compensation and dynamic multipath mitigation contribute to a more extended tracking range.

6.4 Case Study

We design experiments and conduct case studies to verify the impact of different factors on the accuracy of tracking, including different devices, noise conditions, and multipath conditions. The results are as follows.

a) Impact of different devices We implement FusionTrack on three mobile platforms and conduct experiments to verify our system’s performance. These three platforms are Redmi 8A [12], Mi 9 SE [11], and Xiaomi 13 [13]. The prices of them are approximately $100, $300, and $500, respectively. Figure 25 shows the experimental results. These platforms’ average absolute tracking errors are 1.67 mm, 1.81 mm, and 1.16 mm, respectively. The results indicate that FusionTrack can achieve accurate motion tracking on all three platforms.

Fig. 25.

b) Impact of noise We evaluate the performance of our system in environments with different noise conditions, including a quiet environment, a noisy environment with people speaking (about 60dB), and a noisy environment with music playing (about 75dB). Figure 26 shows the results. The average absolute tracking errors under these noise conditions are 1.16 mm, 1.35 mm, and 1.52 mm, respectively. The results indicate that noise in the environment has almost no impact on our motion-tracking results. The reason is that our system operates in the ultrasonic frequency band, and the audible noise is filtered out.

Fig. 26.

c) Impact of multipath effect. We evaluate the impact of the multipath effect on the tracking accuracy in different indoor scenarios, including a meeting room and an office. There are fewer multipaths in the meeting room and a more substantial multipath effect in the office. Figure 27 shows the tracking results. The average absolute tracking errors under these multipath conditions are 1.16 mm and 1.46 mm, respectively. The results indicate that our system can achieve accurate motion tracking in environments with static multipaths. The reason is that our multipath elimination method can achieve good performance in mitigating the multipath effect. We also verify the tracking accuracy in dynamic multipath scenarios. We place an electric fan at another location that is 1 m from the smartphone and perform motion tracking with FusionTrack. The average absolute tracking error is 1.56 mm under this setting. The results indicate that our system can also accurately track targets in the environment with dynamic multipaths because of our dynamic multipath elimination method.

Fig. 27.

6.5 Computational Cost

To evaluate the computational cost of FusionTrack, we analyze the time complexity of each module of FusionTrack and measure the computation time of each module and the overall computation time with the implementation of FusionTrack on Xiaomi 13. Table 2 shows the results. The absolute tracking module in the table includes Doppler compensation and dynamic multipath elimination. To begin with, the overall time complexity of FusionTrack is $O(n \log n)$, with the length of the input signal being n. It means that FusionTrack achieves good efficiency. Furthermore, we measure the computation time of FusionTrack with an input signal whose length is 1 second. The overall computation time is 248.3 ms, which means that FusionTrack can achieve accurate tracking in real time on Android smartphones.

Table 2.

Module	Time complexity	Computation time (ms)
Alignment	$O(n \log n)$	43.6
Relative tracking	$O(n)$	30.2
Static multipath elimination	$O(n)$	14.8
Absolute tracking	$O(n \log n)$	158.4
Kalman filter	$O(n)$	1.3
Overall	$O(n \log n)$	248.3

Table 2. Computation Cost of FusionTrack

6.6 Overall Comparison with Existing Approaches

We compare our system in multiple aspects of the evaluation, including the type of transmitted acoustic signal, the tracking error, and the tracking range. Table 3 shows the results. Signal fusion helps FusionTrack outperform methods that use only one type of signal, including CW, FMCW, and OFDM signals.

Table 3.

Method	Signal type	Absolute accuracy	Tracking range
LLAP[35]	multiple CW	40 mm	0.4 m
PDF[5]	FMCW	3.6 mm	0.7 m
Strata[39]	OFDM	10 mm	0.4 m
FusionTrack	CW + FMCW	1.3 mm	2.2 m

Table 3. Comparison with Existing Approaches

6.7 2D Tracking

For 2D tracking, we utilize one speaker and two microphones to track the 2D trajectory of the target. As Figure 1 shows, each microphone-speaker pair can confine the target’s possible location to an ellipse. An intersection point of the two ellipses is a possible 2D location of the target. We further solve the ambiguity of 2D locations with the trajectory’s continuity.

a) FusionTrack achieves a 2D trajectory error of 4.5 mm. We evaluate 2D tracking with three types of 2D motion templates: a linear motion with a 20 cm length, a circular motion with a 20 cm diameter, and a square motion with a 20 cm edge length. Figure 28(a) shows the CDF of the 2D trajectory errors for tracking with these three templates. The average errors are 4.5 mm, 5.1 mm, and 6.4 mm for linear, square, and circular motions, respectively. Figure 28(b) and Figure 28(c) show the comparison of the 2D tracking result and the trajectory of motion templates. The shapes and trajectories of the tracking result match the template. We also try complicated motions. Figure 28(d) shows the tracking result of the word ”yes”. The results of the 2D tracking show that FusionTrack can achieve 2D motion tracking with high accuracy.

Fig. 28.

7 Related Work

Acoustic sensing with built-in microphones and speakers has become a rapidly evolving research area, driven by the advancement of wireless sensing and the increasing popularity of smart devices. Acoustic sensing captures and analyzes acoustic signals, extracting valuable information about the environment, objects, or individuals. Acoustic sensing facilitates applications in various domains, including environmental sensing [1, 17, 31, 43, 44], health monitoring [16, 22, 26, 34, 41, 47], localization [2, 9, 21], user verification [8, 42, 48], human-computer interaction [23, 25, 27, 37].

Researchers have proposed various signals for acoustic sensing, with two prominent types being CW-based and FMCW-based sensing.

7.1 CW-Based Sensing

Single-frequency continuous wave (CW) signals have a constant frequency and amplitude. CW-based sensing techniques usually rely on frequency shift or phase information. For device-based applications, AAMouse [38] applies Doppler shift to track a mobile device and turns a mobile phone into a computer mouse for device control. The phase change during the motion supports tracking with higher resolution. Swadloon [9] employs a phase lock loop to capture the phase change, achieving precise indoor localization and tracking. Vernier [45] introduces phase analysis in the time domain based on very few samples, achieving millimeter-level tracking accuracy with low computational cost. Moreover, SoundTrak [40] realizes 3D finger tracking on smartwatches with a microphone array on the smartphone, and EarphoneTrack [36] facilitates acoustic tracking on earphones. For device-free applications, SoundWave [6], AudioGest [27], and Multiwave [25] utilize Doppler frequency shift for gesture recognition. Vpad [20] designs a tracking system for laptops without touchscreens, utilizing the energy feature and Doppler shift to track hand movement. LLAP [35] tracks target motion through phase analysis with multiple CW signals of different frequencies. SpiroSonic [28] tracks the motion of the chest wall for respiration detection using phase information.

7.2 FMCW-Based Sensing

Frequency-modulated continuous wave (FMCW) signals consist of repeated signals with linearly increasing or decreasing frequency. BeepBeep [24] develops an approach to measure the distance between two devices without synchronization. CAT [21] constructs a pseudo-transmitted signal for device-based tracking without the requirement of precise synchronization. EchoTrack [4] and EchoSpot [18] propose Time of Flight (ToF) estimation with correlation for indoor localization. FM-Track [15] leverages a microphone array for multi-target tracking. Additionally, PDF [5] and Millisonic [30] utilize phase analysis of the received FMCW signal for device-free tracking. FMCW-based sensing can also sense micro movements, enabling applications such as breath sensing [16, 22, 34] and heartbeat sensing [16, 26, 41].

In addition to CW and FMCW signals, other works employ different types of signals, such as pseudo-noise sequences [19, 29, 39] and OFDM signals [23].

8 Discussion and Future Work

8.1 Discussion

This paper shows that FusionTrack is a promising solution for acoustic motion tracking, which overcomes practical challenges. Although our approach demonstrates significant advancements, there are several aspects to consider in the discussion.

–

Scalability and deployment: The 1D and 2D tracking demonstrates the versatility of FusionTrack. Furthermore, FusionTrack has good scalability. FusionTrack does not have the requirement for synchronizing speakers and microphones, making it available for systems with multiple devices. Thus, FusionTrack can be used to build and deploy a larger tracking system on different devices.

–

Energy efficiency: Energy efficiency is significant for applications on devices with limited resources, such as smartphones. Investigating optimizations and compromises to reduce the energy consumption of FusionTrack without sacrificing tracking accuracy will be essential for its successful implementation on mobile platforms.

8.2 Future Work

We propose some possible future work:

–

Multimodal fusion: Exploring the combination of acoustic signals with other signals can improve the sensing ability, increasing tracking accuracy and dependability. Future studies will focus on multimodal sensing with microphones and other sensors, such as visual or inertial sensors, and calibrating signals from different sources.

–

Tracking of multiple targets: FusionTrack focuses on tracking the movement of a single target with an acoustic signal and achieves good performance. Accurate tracking of multiple moving targets is promising and can facilitate more applications. Future studies will concentrate on maintaining the accuracy of individual tracks in a multi-target setting, which is challenging because of the mutual interference.

–

Extending FusionTrack to support other signals: The methods utilized in FusionTrack are effective for acoustic signals and applicable for other signals, such as RF signals. Therefore, the potential of FusionTrack can be significantly increased by extending FusionTrack to be available for other signals.

9 Conclusion

In this paper, we propose FusionTrack, a novel device-free acoustic motion tracking approach that addresses the limitations of existing methods through innovative signal fusion techniques. By combining single-frequency continuous wave (CW) and frequency-modulated continuous wave (FMCW) signals, FusionTrack achieves accurate absolute tracking for dynamic targets. We present a comprehensive fusion scheme that leverages the strengths of both CW and FMCW signals and addresses the limitations of tracking based on a single type of signal. Moreover, FusionTrack successfully addresses challenges in acoustic motion tracking with signal fusion, including the Doppler effect and interference of static and dynamic multipaths. Specifically, we propose a Doppler compensation method to accurately eliminate errors introduced by the Doppler effect in absolute tracking. We mitigate errors caused by the multipath effect through static multipath elimination and dynamic multipath mitigation. We employ a Kalman filter to fuse relative and absolute tracking results to enhance tracking accuracy further.

We implement FusionTrack on both PC and Android-based smartphones and conduct extensive experiments to demonstrate its effectiveness and superiority over existing approaches. FusionTrack achieves a remarkable accuracy of 1.5 mm in 1D motion tracking, surpassing existing methods by 46%, and extends the tracking range to 2.2 m, demonstrating a 3.1× improvement. Additionally, our approach achieves a 2D tracking accuracy of 4.5 mm, highlighting its applicability in real-world scenarios.

References

[1]

Chao Cai, Zhe Chen, Henglin Pu, Liyuan Ye, Menglan Hu, and Jun Luo. 2020. AcuTe: Acoustic thermometer empowered by a single smartphone. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems (SenSys’20). ACM, New York, NY, USA, 28–41. DOI:

Module	Time complexity	Computation time (ms)
Alignment	\(O(n \log n)\)	43.6
Relative tracking	\(O(n)\)	30.2
Static multipath elimination	\(O(n)\)	14.8
Absolute tracking	\(O(n \log n)\)	158.4
Kalman filter	\(O(n)\)	1.3
Overall	\(O(n \log n)\)	248.3

Abstract

1 Introduction

2 Prior Arts and Limitations

2.1 CW-Based Tracking

2.2 FMCW-Based Tracking

3 Accurate Absolute Tracking with Signal Fusion

3.1 CW-Based Relative Tracking

3.2 Doppler Compensation

3.3 Absolute Tracking

4 Combating Multipath Effect

4.1 Static Multipath Elimination

4.2 Mitigating the Dynamic Multipath

5 System Design

5.1 Overview

5.2 Signal Design

5.2.1 Reduced Spectral Leakage.

5.2.2 Accurate Signal Alignment.

5.3 Kalman Filter

6 Implementation & Evaluation

6.1 Implementation

6.2 1D Tracking

6.3 Ablation Study

6.4 Case Study

6.5 Computational Cost

6.6 Overall Comparison with Existing Approaches

6.7 2D Tracking

7 Related Work

7.1 CW-Based Sensing

7.2 FMCW-Based Sensing

8 Discussion and Future Work

8.1 Discussion

8.2 Future Work

9 Conclusion

References

Cited By

Index Terms

Recommendations

Accurate 3D motion tracking by combining image alignment and feature matching

Target Tracking Using a Joint Acoustic Video System

Motion Tracking with an Active Camera

Comments

Information

Published In

Publisher

Journal Family

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations