US20140066816A1

US20140066816A1 - Method, apparatus, and system for characterizing gait

Info

Publication number: US20140066816A1
Application number: US13/920,032
Authority: US
Inventors: James McNames; Sean Pearson; Lars Holmstrom; Pedro Mateo Riobo Aboy; Andrew Greenberg; Gavin Gallino; Timothy Brandon
Original assignee: APDM Inc
Current assignee: Wearables Ip Holdings Inc; Wearables Ip Holdings LLC
Priority date: 2008-12-07
Filing date: 2013-06-17
Publication date: 2014-03-06
Also published as: US8647287B2; US20110214030A1; US20140122958A1; US10117204B2

Abstract

Disclosed embodiments relate to methods, apparatuses, and systems for characterizing gait. Specifically, disclosed embodiments are related methods, apparatuses, and systems for characterizing gait with wearable and wirelessly synchronized inertial measurement units. These include a method for gait characterization that comprises (a) detecting zero-velocity periods using two or more wearable and wirelessly synchronized movement monitoring devices including a triaxial accelerometer and a triaxial gyroscope and (b) calculating temporal measures of gait during walking by estimating the change in position and orientation during each step.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation-in-Part of U.S. patent application Ser. No. 13/037,310 filed on 2011 Feb. 28 which is a Continuation-In-Part of U.S. patent application Ser. No. 12/632,778 filed on 2009 Dec. 7, which claims the benefit of U.S. Provisional Application No. 61/120,485 filed on 2008 Dec. 7, and are hereby incorporated by reference in their entirety. This application also claims the benefit of U.S. Provisional Application No. 61/1,660,700 filed on 2012 Jun. 16, which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Disclosed embodiments relate to methods, apparatuses, and systems for characterizing gait. Specifically, disclosed embodiments are related methods, apparatuses, and systems for characterizing gait with wearable inertial measurement units.

BACKGROUND

Gait analysis is important in diagnosing and assessing several neurological diseases such as Parkinson's disease (PD) and other conditions. Objective, accurate, and fully automated gait characterization requires novel biomedical signal processing methods and specialized hardware for continuous movement monitoring.

A. Objective Assessment of Movement Disorders

In recent years, large advances have been made in micro-electro-mechanical systems (MEMS) and inertial sensors. It is now possible to record body movements for hours with small, low-power, wearable sensors that include accelerometers, gyroscopes, goniometers, and magnetometers. Despite these advances, clinical practice and clinical trials related to movement disorders are still based on subjective assessment using rating scales. This is due to the fact that there are no commercially available systems to perform objective assessment of movement disorders. One of the main challenges in designing a complete, portable, and easy-to-use system for objective assessment of movement disorders that would be appropriate for clinical practice and clinical trials is the unavailability of movement monitors that can wireless communicate with each other in order to collect synchronized kinematic data from different locations such as the ankles, wrists, waist, and trunk. Currently, there are no movement monitors capable of performing wireless synchronization of the data collected by the different sensors and ensuring that the collected data is never lost during wireless data transmission (i.e. robust wireless data transfer).

A.1. Subjective Assessment of Movement Disorders and Clinical Trials

Subjective assessment of movement disorders using clinical rating scales or poor instruments of mobility result in clinical trials that are inefficient, slow, complicated, and expensive. The primary outcomes are typically self-reported outcomes recorded from patient diaries (falls), clinician rating scales (UPDRS, Berg Balance scale), and/or patient questionnaires (PDQ-39). All of these instruments have limited resolution, are subjective, and are susceptible to bias. To overcome the limitations of these instruments, clinical trials typically require a large number of subjects to detect a clinically significant difference between groups. The data is typically collected on paper versions of the scales and questionnaires. The data is then entered into a database by research assistants, which may result in transcription errors. Finally, the data from each site is then transmitted to a central site, so that a statistician can analyze the data and generate the results of the trial.

A.2. Subjective Assessment of Movement Disorders

Subjective clinical rating scales such as the Unified Parkinson's Disease Rating Scale (UPDRS) are the most widely accepted standard for motor assessment. Presently motor symptoms are diagnosed and assessed during a brief clinical evaluation performed by a primary care physician or neurologist every 3-6 months. Current methods of motor system assessment for PD are inadequate because they are intermittent, coarse, subjective, momentary, stressful to the patient, and insensitive to subtle changes in the patient's motor state. These scales can only be applied in clinical settings by trained clinicians.
Patient diaries and other methods of self reporting are sometimes used to determine patients' motor condition throughout the day, but these are often inaccurate, incomplete, cumbersome, and difficult to interpret. These methods are also susceptible to selection, perceptual, and recall bias. Patients generally have poor consistency and validity at assessing the clinical severity of their impairment. Patients with mild or moderate dyskinesia may be unaware of their impairment and may have poor recall. However, patients may be able to accurately monitor their overall disability.

A.3. Objective Assessment of Balance, Gait, and Fall Risk

Neurological deficits, such as Parkinson's disease, inevitably result in limitations on mobility, a sensitive measure of health and a critical element for independent living and quality of life. However, clinical practice aimed at reducing mobility disability have been limited either by insensitive, descriptive balance rating scales, timed tests of gait speed, fall counts or by complex, expensive, and time-consuming laboratory assessments of balance and gait. For instance, the lack of accurate objective measures of balance and gait greatly impedes the development and testing of new treatments to improve mobility in neurological patients.
As an example, movement disorders such as balance and gait disorders, are the most common cause of falls and reduced quality of life in people with neurological disorders. People with Parkinson's disease (PD) fall more often than any other neurological disease with 43-70% falling each year. Fear of falling leads to activity restriction and declines in mobility. However, no system currently exists that allows clinicians to evaluate fall risk based on objective tests of balance and gait in a clinical environment.
Up to 52% of healthy older adults experience a fall each year. Falls are costly, both financially and in terms of quality of life. Financially, one in four falls necessitates use of health care resources. In addition, fear of falling often leads to self-induced activity restriction and declines in mobility status and emotional well being. Although the cost of falls in patients with all neurological disorders has not been explicitly delineated, people with Parkinson's disease have a 57% higher prevalence of falls and injuries than same age control subjects. This is especially significant given the cost of falls, which in 1996 apparently exceeded $9 billion spread across 225,000 older Americans.

B. Movement Monitors

State of the art movement disorder monitors employ inertial sensors, such as accelerometers and gyroscopes, to measure position, velocity and acceleration of the subject's limbs and trunk. Current monitors fall into two classes, namely activity monitors and inertial monitors, both of which have disadvantages and limitations that make them incapable of continuous monitoring of movement disorders or objective monitoring.
Activity monitors, such as in U.S. Pat. No. 4,353,375, collect low frequency and low resolution samples of the subject's gross activity for days to weeks at a time. These monitors are usually small, unobtrusive devices resembling watches or brooches which are worn by the subject for long periods of time such as days or weeks outside of the clinical setting. They measure movement using low quality inertial sensors at low sampling frequencies, and usually measure only a few degrees of freedom of motion instead of all six possible degrees of freedom of motion. The low quality measurements are stored in data storage on-board the device which is later downloaded and analyzed. While they are useful for recording the gross activity levels of the subject, and they may be comfortable and unobtrusive enough to be worn by the subject for longs periods of time, they are only useful in measuring non-subtle symptoms of movement disorders such as activity versus rest cycles. Subtle symptoms, such as symptom onset and decline, or non-obvious symptoms such as bradykinesia, can not be measured by these devices. These devices, also known as actigraphers, typically measure movement counts per minute which make even simple determinations such as determining the wake-up time challenging. Consequently, actigraphers are inappropriate for continuous ambulatory monitoring of movement disorders such as in Parkinson's disease.
Inertial monitors, such as in U.S. Pat. No. 5,293,879, collect high frequency, high resolution samples of the subject's movements for short periods of time. These devices are larger and more obtrusive, resembling small boxes which are worn by the subject for short periods of time such as hours, or at most, a day, and usually in clinical settings. They measure movement using high quality inertial sensors, and usually include all six degrees of freedom of motion (three linear axes and three rotational axes). Inertial monitors may store the inertial measurements in the device for later analysis, or they may use telemetry radios to wirelessly transmit the measurements in real-time to a nearby computer or recording device. These devices are useful for measuring all symptoms of movement disorders, but because of their larger, obtrusive size and short operational times, they are not useful for measuring symptoms outside of clinical settings or for long periods of time.
Movement disorder monitoring can be enhanced by monitoring multiple locations on a subject at the same time. Current systems either do not synchronize their measurements, or require wires to synchronize sampling. Additionally, current movement disorder monitoring devices also lack aiding sensors, such as absolute measures of position.
Movement monitoring devices and systems that overcome challenges of physical size, power consumption, and wireless synchronization are currently unavailable and have significant potential in numerous applications including clinical practice and research.
Currently, the most common and accurate method of tracking movement is based on optical motion analysis systems. However, these systems are expensive, can only measure movements in a restricted laboratory space, and cannot be used to observe patients at home.
Current inertial monitoring systems can be divided into three categories: computer-tethered, unit-tethered, and untethered. Computer-tethered devices connect the sensor directly to a computer. One of the best systems in this category is MotionNode (GLI Interactive LLC, Seattle). These systems are not practical for home settings. Unit-tethered systems connect the sensors to a central recording unit that is typically worn around the waist. This unit typically houses the memory, batteries, and wireless communications circuits. Currently, these systems are the most widely available and are the most common in previous studies. One of the best systems in this category is the Xbus kit (Xsens, Netherlands). This system includes up to five sensors, each with high-performance, triaxial accelerometers, gyroscopes, and magnetometers. The system can operate continuously and wirelessly stream data via Bluetooth to a laptop for over 3 h at distances up to 100 m. However the system is too cumbersome and difficult to use in a home study due to the wires connecting the sensors and central recording unit, the battery life is too short, and the interconnecting wires may be hazardous during normal daily activities. The typical untethered system combines the batteries, memory, and sensors in single stand-alone units. The only wireless untethered systems reported in the literature are “activity monitors,” which measure the coarse degree of activity at intervals of 1-60 s, typically with a wrist-worn device that contains a single-axis accelerometer. These devices are sometimes called actigraphs or actometers. Most of these devices only report activity counts, which are a measure of how frequently the acceleration exceeds a threshold. Some custom activity monitors directly compute specific metrics of motor impairment, such as tremor. A few studies have shown that activity monitors worn over 5-10 days could detect on/off fluctuations, decreased activity from hypokinesia, and increased activity associated with dyskinesia. However, typical activity monitors cannot distinguish between motor activity caused by voluntary movement, tremor, or dyskinesia. They do not have sufficient bandwidth, memory, or sensors for precise monitoring of motor impairment in PD. They also cannot distinguish between periods of hypokinesia and naps.
Recently, Cleveland Medical Devices (Cleveland, Ohio) introduced two untethered systems, the KinetiSense and Kinesia devices. These systems include triaxial accelerometers and gyroscopes with bandwidths of 0-15 Hz, but lack magnetometers. Although large, the central recording units could to be worn on the wrist. The sensor and recording unit can be connected to form a single unit. This devices can record data continuously and store it on an on-board memory for up to 12 h. However, 1) the due to their size it is difficult for several of these devices to be used at the same time (e.g. wrist, ankle, waits, trunk), 2) the storage capability is limited to a single day and consequently it is difficult to conduct multiple day studies, and 3) the devices are not synchronized.
Movement monitoring devices and systems that overcome the challenges of 1) physical size (volume), 2) power consumption, 3) wireless synchronization, 4) wireless connectivity, 5) automatic calibration, and 6) noise floor; are currently unavailable and have significant potential in numerous applications including clinical practice and research. Finally, the limited solutions currently available are device-centric and do not include a complete platform to perform collection, monitoring, uploading, analysis, and reporting.
C. Movement Monitors with Wireless Synchronization
While there are several commercial movement monitors available capable of wireless data transmission, currently none of these movement monitors is capable of providing wireless synchronization of the sampling instances. The most advanced inertial monitors capable of wireless data transfer such as Xsens' full body motion capture monitor (XSens Technologies) require wires between each of the movement monitors and a central unit in order to synchronize the sampling instances of each of the monitors. Synchronization is critical for applications where more than one movement monitor is needed.
Wireless sensor networks have multiple independent nodes all sensing environmental factors at the same time. In the case of a wearable wireless movement monitor, these environmental factors are the kinetic state of the various limbs of a subject wearing two or more movement monitors. Later, during data analysis, the samples of the two or more movement monitors must correlated in time to make any sense together. For example, two movement monitors on the ankles need to be correlated in time in order to show the difference between a lopsided gallop and a smooth run. The problem is that in order to be correlated in time, the sensors must sample at the same time, and, over time, at the same rate, over a long time period of hours, or even days.
There are many ways to do this correlation, but the challenge with small wireless sensor systems is how to go about providing this synchronization of the sampling time and rate without unduly impacting other system parameters.
One way in which current wireless sensor networks synchronize with each other is to provide a wired sync line between nodes. While simple and effective, this not only requires cumbersome wires running between nodes, but obviously defeats the wireless part of the wireless sensor network.
D. Movement Monitors with Robust Wireless Data Transfer
In small, highly mobile wireless devices, such as wireless movement monitors, it is necessary to robustly stream large amounts of data (100s of bits to 100s of kilobits per second) in near real time (without large latencies in transmission) over a radio frequency communication channel. These continuous, real-time wireless transmissions often suffer from unpredictable data loss due to a variety of environmental factors, including distance between transmitter and receiver, absorption of the signals by local materials (including human bodies), multipath interference due to objects which reflect or refract signals, and even interference from other devices. The challenge with these small embedded systems is how to go about guaranteeing transmission of the signal without unduly impacting other system parameters.
One way in which current wireless movement monitors overcome transmission problems, such as distance and interference, is to increase the radio frequency (RF) signal strength of their transmissions and/or to use receive amplifiers. Either method leads to an large increase in consumed power, which leads to larger battery sizes, which leads to dramatically larger and heavier devices, forcing some systems to even have large, separate wired unit which holds a replaceable battery pack.
None of the current methods to overcome radio communication disruptions allows a wireless sensor to remain small, reduce power consumption, and avoid data loss during long interruptions in communication.

BRIEF DESCRIPTION OF THE DRAWINGS

Disclosed embodiments of example results are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings.

FIG. 1 shows a block diagram of the gait characterization method according to one embodiment.

FIG. 2 shows a block diagram of the gait characterization method according to an alternative embodiment.

FIG. 3 shows a block diagram of the template matching method according to one embodiment.

FIG. 4 illustrates the initial and final location of the foot during a single step with the left side. The straight vertical line shows the forward direction of travel. The two angles θ_iand θ_eshow the toe out angles at the beginning and end of the step. The average of these two angles is reported by the method as the toe out angle for this step.

FIG. 5 shows a sequence of three foot placements. A straight line path from the first to the last foot placement is considered the forward direction of travel. The lateral step deviation is calculated as the maximum lateral distance of the middle step from this path.

FIG. 6 illustrates a table of results produced by the gait characterization method including the output metrics. This includes the number of step sequences used to calculate each gait metric (column n), the average (μ) and standard deviation (σ) of each metric for the left foot, the right foot, and the left-right differences. Each of these six statistics is listed for all sixteen metrics of gait included in this table.

FIG. 7 shows a heatmap of averaged step trajectories versus the time of each detected step. The top row shows the forward position, the middle row shows the amount of lateral swing, and the bottom plot shows the vertical position of the top of the foot during swing.

FIG. 8 shows the average morphology of the accelerometer and gyroscope magnitudes during a step. The shaded region shows the variability. This shows the shape of the four-channel templates used for template matching.

FIG. 9 shows the weighted and scaled template error versus time for an example recording. The lower horizontal lines show the initial thresholds for detecting minima in the error that represent steps. The higher horizontal lines show the thresholds used to add missed steps during the template matching method. The bottom row of plots shows the intervals between steps. The occasional spikes that are roughly 20 s apart are due to the slowing in the gait cycle that occurs when the subject made 180 degree turns.

FIG. 10 shows the initial detection of steps by vertical black lines based on still periods of the other foot as detected by the gyroscopes and accelerometers. The horizontal lines show thresholds. The bottom set of plots shows the intervals between the initial detection of steps. Once the initial detection of steps is completed, an initial template can be created to begin the iterative template matching detection.

FIG. 11 shows the actual templates for a real subject. The left column of plots shows the stance side and the right column of plots shows the swing side. The top row of plots shows the accelerometer magnitudes and the bottom shows the gyroscope magnitudes. The width of the shaded regions shows the standard deviation across the detected steps that were used to create the template. The thick dark lines show the actual templates used for step detection.

FIG. 12 shows another example of the template error for the left and right feet. At the beginning and end of the recording the subject was still and the normalized error was equal to 1, as the method is designed. The horizontal lines show the thresholds. The green dots show the individual detected steps.

FIG. 13 shows the error versus the shift in alignment of the right template relative to the left template. The minimum is shown by the red dot and represents the best shift to align the left and right templates.

FIG. 14-29 shows illustrative examples of the apparatus and overall system for wireless synchronized movement monitoring.

DETAILED DESCRIPTION

1) Wireless Synchronization of Sampling Time Instances in Movement Monitors

The teachings of this disclosure directed to the calculation of temporal measures of gait such as single support time require a particular type of movement monitor (in this disclosure the concepts of movement monitor, movement sensor, and inertial measurement unit are considered synonyms and are used interchangeably). Specifically, it requires wearable movement monitors characterized by being 1) wearable, 2) untethered, 3) capable of wirelessly synchronizing the sampling time instances of two or more monitors (preferable with a synchronization resolution≧1 ms), and 4) having a bandwidth higher than 15 Hz. The details relating to such movement monitors are found in U.S. patent application Ser. No. 13/037,310 filed on 2011 Feb. 28 entitled “Wireless Synchronized Movement Monitor and System” which is hereby incorporated by reference.

2) General Description of Method and Apparatus for Characterizing Gait

According to one embodiment the method for gait characterization comprises: (a) detecting zero-velocity periods using two or more wearable and wirelessly synchronized movement monitoring devices, the movement monitoring devices comprising a triaxial accelerometer and a triaxial gyroscope with a bandwidth of at least 15 Hz; and (b) calculating temporal measures of gait during walking by estimating the change in position and orientation during each step. In a more particular embodiment, the step of calculating temporal measures of gait includes performing template matching based on the magnitude of the accelerometer's and gyroscope's signals from both feet. Furthermore, template matching is characterized by 1) enabling multiple iterations to refine a final template, 2) weighting each template by the standard deviation of the template across detected steps, 3) scaling the template's error to be equal to one when the movement monitoring devices are stationary, 5) using a fast method based on a fast Fourier transform configured for calculating the template's matching error, or combinations thereof. In a particular embodiment, and without limitation, the step of calculating temporal measures of gait further comprises: a) detecting initial steps, b) estimating a gait cycle duration, c) building an initial template, d) calculating a template match, e) detecting initial template steps, f) validating steps, g) adding missed steps, or combinations thereof. In some embodiments, the step of calculating temporal measures of gait further comprises 1) measuring asymmetry using time-proximate steps by combining left and right steps into consecutive left-right pairs, 2) characterizing gait during normal periods of walking by isolating sequences of steps in which the subject is traveling forward on a flat surface based on changes in height, bank angle, elevation angle, and heading angle, 3) generating an indicator of foot drop and fall risk by characterizing the pitch of the foot with wearable sensors at the moments of heel strike and toe-off, 4) characterizing the lateral deviation in a sequence of two steps resulting in three foot placements based on how far a middle foot placement deviates from a straight path from a first to a last foot placement, and 5) measuring the lateral swing of the foot during a single step. In a particular embodiment, the overall method for gait characterization comprises 1) upsampling, 2) estimating biases, 3) calculating magnitudes, 4) finding still periods, 5) calculating positions, 6) detecting steps, 7) finding and validating step sequences, and 8) calculating gait metrics (FIG. 1 and FIG. 2). According to specific embodiments, the disclosed method can be implemented in other hardware besides a digital computer including microcontrollers, processors, DSPs, FPGAs or ASICs, and firmware.

3) Description of Method and Apparatus for Characterizing Gait According to Particular Embodiments

In the following description the term “Subject oriented” describes a reference frame defined as the forward direction in which the subject is traveling (x-axis) projected onto the plane that is orthogonal to gravity, the subject's left side that is orthogonal to gravity and the forward direction (y-axis), and the up direction defined as the opposite direction of gravitational attraction (z-axis). This is sometimes briefly described as forward-left-up. This can be calculated by rotating the position in the Earth frame (north-west-up) about the z-axis (i.e., changing the heading angle). The origin of the subject-oriented frame is the still period when the foot is level on the ground preceding a step or sequence of steps. The forward direction defining this reference frame can be defined based on the final position of the foot after a single step or a sequence of steps. The term “rotational magnitude” describes the norm of the three gyroscope channels, possibly after processing to account for calibration, temperature compensation, upsampling, and bias removal. Specifically this is defined as the square root of the sum of the three squared gyroscope channels. When the wearable device is stationary or still, the magnitude is expected to be close to zero. The term “accelerometer magnitude” describes the norm of the three accelerometer channels, possibly after processing to account for calibration, temperature compensation, upsampling, and bias removal. Specifically, this is defined as the square root of the sum of the three squared accelerometer channels. When the wearable device is stationary or still, the magnitude is expected to be close to the acceleration due to gravity, which is approximately 9.8 m/s². The term “step pair” describes a pair of normal steps consecutive in time without delay or pause of either the left side followed by the right side, or the right side followed by the left side. Paired steps are helpful for statistical comparisons of the gait between the left and right sides. The term inertial measurement unit (IMU) describes a device containing at least triaxial accelerometers and triaxial gyroscopes with a bandwidth of at least 15 Hz.
The following sections describe various embodiments to implement a method, apparatus, and system for gait characterization based on IMUs. According to one particular embodiment, and without limitation, the method for characterizing one or more features of gait during walking from two or more wirelessly synchronized IMUs attached to the feet or shoes based on detected steps comprises detecting zero-velocity periods and estimating the change in position and orientation during each step using inertial navigation with or without aiding. In a particular embodiment, the method further comprises estimating the bias of the accelerometers by detecting still periods, estimating the attitude of the sensor, subtracting the effect of gravity, calculating the residual—which is expected to be bias (slowly varying) and white noise (broad band), and calculating the bias in between still periods with some form of interpolation or smoothing. In a particular embodiment, the method comprises detecting each step by initially detecting still periods based on low rotational magnitude and an accelerometer magnitude that is close to that of gravity, applying prior knowledge of still periods during the gait cycle to eliminate implausible gait periods, estimating an initial template from the initially detected periods based on one or more of the accelerometer or gyroscope signals from either of the feet, calculating a figure of merit by comparing the shifted template to signal segments over the full range of the signal, and detecting minima or maxima in the figure of merit to determine step locations. This embodiment comprises estimating the variability of the template, and weighting the figure of merit based on the variability of the template. In a particular embodiment, the method further comprises calculating the figure of merit for two or more of the four magnitudes (accelerometer magnitude and gyroscope magnitudes for each foot), scaling the figure of merit for each magnitude such that during a still period the error is equal to a constant, and combining the figure of merit for the different magnitudes with a statistic such as the average, median, minimum, or maximum. In a particular embodiment, the method further comprises estimating the template locally in time using either a fixed window or a fixed number of steps that are nearby in time to the time at which the template error is calculated, and computing a subject-oriented reference frame for calculation of the foot position and orientation during gait comprised of estimating the orientation and position of the foot in an inertial reference frame (such as the Earth reference frame of north-west-up or north-east-down), defining the forward direction of gait based on the change in position over one or more steps, translating the inertial reference frame to an origin defined as the starting location of one or more steps, and rotating the inertial reference frame to have a forward axis calculated from the change in position from the starting location to the ending location after one or more steps in the plane orthogonal to the up direction defined by gravitational attraction. In particular embodiments, and without limitation, the method further comprises calculating the following continuously during each step: the lateral (leftward) position of the foot, the height of the foot (defined relative to the location of the wearable device), the forward position, the heading angle (defined relative to forward in the forward-left plane), the elevation angle (defined as extent of upward tilt relative to the forward-left plane), and the bank angle (defined as the remaining Euler angle). Additionally, in particular embodiments, the method further comprises detecting step pairs by detecting candidate steps on each side and pairing steps that meet known normal physiologic criteria such as the period of time between the start of a step on one side and the start of a step on the other side. Certain embodiments, further comprise detecting the time at which the toe leaves the ground (toe off) based on the time of the maximum subject-oriented elevation Euler angle, detecting the time at which the foot is parallel to the ground during the swing phase of a gait cycle based on the time at which the subject-oriented elevation Euler angle is near zero, detecting the time at which the heel strikes the ground based on the minimum subject-oriented elevation Euler angle and a large acceleration magnitude, calculating the standard division of the gait cycle from wirelessly synchronized triaxial IMUs attached to the feet or shoes into relative durations for the initial double support, single support, terminal double support, initial & mid swing, and terminal swing from the detected toe-off, foot flat, and heel strikes for both feet, calculating the orientation of the foot during still periods which may be used to calculate the extent of pronation, and calculating sequences of consecutive steps in the forward direction to characterize normal gait during periods that exclude starts, stops, turns, pauses, and other interruptions to normal forward gait. According to one embodiment, an apparatus comprises a processor configured to perform the method steps above described and hardware to display the results. A system comprises the method, the apparatus, and a plurality of wearable synchronized movement monitors (FIG. 14-29). The following sections describe in more detail the particular method steps involved in the various embodiments of the method, apparatus, and system.
The following sections provide additional detailed information for particular embodiments, and without limitation, of the method disclosed in FIG. 1 and FIG. 2.

3. A. Upsampling

According to one embodiment, the first stage of the gait characterization method upsamples the raw sensor data to an effective sampling rate that is high enough to prevent significant errors in the integration caused by first order approximations (i.e., the Euler method) of integrals. One skilled in the art will know there are other methods that could be used to estimate nonlinear integrals that may require less computation or have other advantages. According to one embodiment, and without limitation, the method uses a bandlimited interpolation methodology to upsample the signals, though many other largely equivalent methods are available. In one embodiment, after upsampling, the sample rate should be 500 Hz or higher, roughly 10× the bandwidth of the signal. Further improvements are possible with resampling to higher rates.

3. B. Zero-Velocity Detection

Detection of periods when the IMUs and feet are still is used in one embodiment of the method in several stages of the signal processing. These still periods are often referred to as zero-velocity periods in the literature and the algorithms for detecting them are called zero-velocity detectors. When the sensors are placed on the feet, these still periods normally occur during gait when the foot is flat on the ground.
According to one embodiment, and without limitation, the method detects these still periods by calculating the Euclidean magnitude, or norms, of the gyroscopes and magnetometers. These magnitudes are expected to be zero for the gyroscopes and equality to the magnitude of gravity for the accelerometers. The magnitudes are convenient to work with because they are independent of the sensor orientation. Three thresholds are specified for the minimum and maximum magnitudes of the accelerometers and the maximum magnitudes of the gyroscopes. If any of the threshold criteria are not met, the IMU is declared as moving. If all of the threshold criteria are met, the IMU is declared as still.
Some stages of processing require detection of periods that are more stationary than others. For example, estimation of the sensor bias requires periods that are very still. Estimation of the IMU attitude by determining the direction of gravity relative to the IMU's body orientation, requires still periods that can be less still.
It is possible to improve performance by smoothing either the signals before magnitude calculation or smoothing the magnitude signals. Smoothing may be implemented with a lowpass filter, kernel smoother, or any of a variety of other methods. The extent of smoothing may vary depending on the requirements of the processing stage. Alternative embodiments make use of these techniques to improve performance.
Using a simple threshold detection can result in detecting still periods in which there is slight movement near the crossing points of the thresholds. Performance may be improved by finding the first minimum in the magnitude signal, with or without smoothing, after crossing the threshold to eliminate these slight periods of movement.
In some cases the still period is expected to be of a certain duration. Performance may be improved by specifying an additional threshold on the still duration required in order for a still period to be considered valid and usable for a given stage of signal processing.

3.C. Bias Estimation

According to one embodiment, the next stage of the method estimates the sensor bias. In one particular embodiment, and without limitation, this processing stage begins by finding very still periods in which tight thresholds are used to detect the still periods. During still periods the gyroscopes are expected to contain a slowly varying bias and broadband, zero-mean noise. The bias can be estimated during still periods with a lowpass filter or equivalent means of estimating the slowly varying component. If the periods are brief enough, as a constant estimated as the mean, median, or some other measure of central tendency.
During still periods the accelerometers are expected to contain a constant component due to gravity, a slowly varying bias, and broadband noise. The component due to gravity is expected to be much larger in magnitude and can be used with techniques to estimate the attitude (elevation and bank angles, but not heading) of the IMU. The attitude is combined with knowledge of the magnitude of gravity (approximately 9.8 m/s²at most locations) to estimate the expected gravitational component of the accelerometers, which can then be subtracted from the accelerometer signals during each of the still periods. The difference is approximately comprised of just the slowly varying bias and broadband, zero-mean noise. As with the gyroscopes, the accelerometer bias can be estimated during still periods with a lowpass filter or equivalent means of estimating the slowly varying component. If the periods are brief enough, as a constant estimated as the mean, median, or some other measure of central tendency.
Once the gyroscope and accelerometer biases are estimated in each of the still periods, any form of smoothing or interpolation, such as piecewise linear interpolation, kernel smoothing, a spline, or quadratic interpolation, can be used to estimate the bias when the IMU is not still. Once the sensor biases are estimated, they can be subtracted from the entire observed signals to produce signal estimates that are largely immune to the effects of bias.

3.D. Step Position Tracking

According to one embodiment, once still periods are detected, inertial navigation methods can be used to estimate the orientation and position of the IMU and foot during each transition between one still period and the next. Note that a transition between still periods may or may not correspond to a normal step. The method for tracking the position of a step requires an initial estimation of the IMU orientation. The methods uses the first still period and the accelerometers to determine the attitude. The heading may be defined arbitrarily. If a magnetometer is available, it may be used to determine the heading and possibly improve the initial attitude estimate. The direction of gravity is used to define the upward direction in the reference frame used for position tracking, often called the inertial reference frame or the Earth reference frame.
In navigation applications it is common to use a reference frame defined as north (x), east (y), down (z), which satisfies the right hand rule. In this application it is more convenient and natural to use an inertial reference frame defined as north (x), west (y), up (z), which also satisfies the right hand rule.
According to one embodiment, the method uses quaternions to represent and track changes in the orientation from one still period to the next. Forward and backward estimates are calculated separately and then combined statistically based on the estimated variance of the two estimates. When state space tracking methods are used, this method of estimation is called smoothing. Once the smoothed orientation estimates are calculated, the position estimates are also computed forward and backward in time by rotating the accelerometer signals into the inertial reference frame, subtracting the gravitational component, and then integrating twice in time to convert acceleration estimates into position estimates. In a particular embodiment, and without limitation, during each still period the IMU attitude is updated by using the gravitational component of the accelerometers to determine where the upward direction is. The heading is unchanged, though it could also be updated if magnetometers or some other absolute reference indicating the IMU heading is available. During the backward phase of orientation estimation, the updated attitude is used as the starting point for the orientation estimate.

3.E. Initial Step Detection

In a particular embodiment, once the bias is estimated and subtracted from the signals, the method detects candidates for steps. During normal forward walking a period of single support is expected in which one foot is swinging forward while the other foot is stationary on the ground. The method detects these periods initially by finding still periods in which one foot is on the ground using the method described previously. The still periods are then checked against a variety of criteria to ensure they correspond to a step. For example if two minima are adjacent in time with a duration less than that expected to be physiologically possible for the duration between two steps, the minimum with a larger magnitude of movement, as measured for example by the gyroscope magnitude, is removed. It is also expected that between still periods for one foot, the other foot will go through a swing phase that will include a certain amount of movement. Thus the maximum accelerometer and gyroscope magnitudes during the swing phase are compared to thresholds to ensure that the foot taking the step undergoes sufficient movement to qualify as a step. This helps eliminate candidate periods in which both feet are still.

3.F. Template Matching

FIG. 3 shows the method steps of the template detection method according to one embodiment. In a particular embodiment, and without limitation, once the initial steps are detected, a three step template matching method is used to more precisely detect all of the steps. During the first step, a template is estimated. This particular embodiment of the method uses a four channel template which includes the accelerometer and gyroscope magnitudes for each of the feet. The template is computed locally in time based on a specified number of detected steps that have occurred before and after the time of interest for the template matching. The duration of the template is user-specified. In this implementation, the template spans from half a gait cycle prior to the center of the template to half a gait cycle after the template. For each channel and each point in time for the template, the mean across all steps and the standard deviation across all steps is estimated. One with ordinary skill in the art will know that other measures of central tendency, such as the median, and measures of variability, such as the inter-quartile range, could be used instead.
In a particular embodiment, during the second step, for each channel and each point in time, a weighted error, or some other figure of merit, is computed that corresponds to the degree of similarity between a signal segment and the template. The error is weighted by the variability of the template. In this embodiment, the method uses a weighted mean squared error, but other similarity measures, weighted or unweighted, could be easily used. The error for each channel is scaled by the error that occurs for constant magnitudes in each channel. In this manner, the error for each channel is normalized so that it has a value of 1 when a constant signal is applied. This makes it easier to select detection thresholds that do not vary with the step morphology. Finally, the template errors from all four channels are combined. In this embodiment, the method combines them by calculating the average, but other statistics such as the median, max, or min, could be used. Finally, a threshold is applied to detect the initial candidate steps based on template matching. During the third step logic based on domain knowledge is used to revise and correct the steps detected by applying a threshold to the detected minima in the threshold error. For example, if two minima or adjacent in time by a duration that is shorter than is physiologically possible for normal gait, the minimum with the smaller template error is retained and the one with the larger error is eliminated.
The template error is expected to increase significantly in between steps. The maximum error between two candidate steps is compared to a threshold. If the transition error is not as large as expected, the candidate step is eliminated.
In a particular embodiment, a forward search and backward search is also used to find steps with template errors larger than the initial threshold. A second, higher threshold, is applied to cases when the separation of two detected steps is larger than would be expected by a normal gait cycle. A search is performed for a template minimum over an interval when the expected next step is expected to occur. If a minimum is found that is lower than the second, higher template error threshold, and the candidate step meets other criteria for the expected transition amplitude and gait cycle duration, then the new step is added. This search is performed forward in time and backward in time to search for steps that were missed during the template matching. The three steps comprising template matching may be repeated for multiple iterations, with the newly detected steps replacing the initial steps during each iteration. This can improve both the accuracy of the times at which the steps are detected as well as the morphology of the templates.
In a particular embodiment, and without limitation, the detection of steps by the left and right feet can be computed separately. However, to compare the symmetry during subsequent processing, it's important that the left and right templates be aligned with one another. Once one side has been processed and the template finalized, it can serve as a reference to align the template of the other foot. The alignment can be performed by computing the template error for a variety of shifted templates and the shift with the minima error can be selected.

3. H. Detection of Step Pairs

In order to characterize normal walking, and particularly gait asymmetry, it is useful to consider pairs of steps for the left and right sides. In a particular embodiment, the method begins with the steps detected on the left side and searches for steps on the right side that most immediately follow. In alternative embodiments, the method could instead or additionally search for pairs of right-left steps. Each candidate pair of steps is then evaluated for a variety of criteria to ensure the step pair is valid. For example prior to each step the still period is compared to thresholds for maximum and minimum durations known to occur during normal walking. Similarly, the transition from one still period to the next for each foot is evaluated to make sure the duration is not shorter or longer than is known to occur during normal walking. The delay from the step on one side to the step on the next side is also compared to the minimal and maximal values that are expected to occur during normal walking. Pairs of steps that pass all of the evaluation criteria are then used for subsequent processing.

3.I. Detection of Step Sequences

Characterization of some aspects of gait, such as walking forward normally, requires processing of sequences of 1 or more consecutive steps. For example, to determine the direction of forward motion and to compute the variability in the lateral (left-right) position of each step, two steps (three still periods) are required so that the forward direction can be defined as the path from the first still period to the last still period and the lateral placement can be determined from the location of the foot during the intermediate (second) still period.
In one embodiment of the method, detection of sequences begins with step pairs as candidates. Pairs that are neighboring in time are considered as members of the sequence. As with the earlier stages of processing, candidate sequences are evaluated initially for a variety of criteria to ensure the sequences comprise normal forward steps. These criteria include maximal and minimal allowed durations between steps. Further criteria acceptance criteria are applied in later stages of processing.

3. J. Transform Inertial Reference Frame to Subject Reference Frame

According to one embodiment, once a candidate sequence of steps is identified, the method defines a new subject oriented reference frame. The origin is defined as the starting location of the foot before the first step is taken. As with the earth reference frame, gravity is used to define the upward direction (z axis). The direction from the origin to the resting location of the foot after the final step in the sequence projected onto the plane that is orthogonal to the z axis is defined as the heading (x axis). The direction orthogonal to the plane defined by the z and x axes that satisfies the right hand rule is defined as the left (y axis), which satisfies the right hand rule. Each step sequence is rotated from the inertial reference frame to the subject reference frame through a rotation about the z axis, which is common to both reference frames.
In one embodiment, the transition of IMU orientations between the starting and final location of the foot during a sequence is computed relative to the starting orientation. This produces changes in orientation that are relative to the starting period in which the foot is flat. These orientations are converted to traditional navigation Euler angles, which can be interpreted in terms of heading, elevation, and bank angles or in terms of angles familiar to those who practice gait analysis.

3. K. Detection of Gait Cycle Components

In one embodiment of the method, the trajectory of orientations and positions during each step is used to detect different points in time during gait that are physiologically meaningful. For example, the time at which the toe leaves the ground at the beginning of a swing period, the time at which the foot is level with the ground during the middle of the swing period, and the time at which the heel strikes the ground at the end of the swing period can be detected. Specifically the time of toe off can be approximated as the time at which the elevation angle (i.e. pitch) is maximal and the time of heel strike can be approximated as the time at which the elevation angle is minimal. Alternatively, the heel strike can be detected from the accelerometer or the rapid deceleration in the Earth reference frame at the time of heel strike. The toe off can also be detected from the change in elevation and knowledge of the location of the IMU relative to the end of the foot. Once the toe off, foot level, and heel strike phases are identified, the periods of the gait cycle can be delineated. Specifically the periods of stance, which include initial double support, single limb support, and terminal double limb support can be estimated. Also the swing phase of gait can be estimated as initial+middle swing as the period from toe off until the foot is horizontal and the terminal swing as the period from foot horizontal to heel strike. These periods can be expressed in units of time or as a percentage of the overall gait cycle, the latter of which is generally preferred. It should be noted that synchronization of the sensor signals is essential to accurately calculate these periods.

3.L. Calculation of Metrics

FIG. 6 shows the statistical summary of gait metrics according to one embodiment. According to one embodiment, once the subject oriented foot trajectories of changes in position and orientation are determined, a variety of metrics can be easily calculated. This embodiment reports the cadence, stride length, foot clearance, pitch angle at the time of toe off, pitch angle at the time of heel strike, the lateral step position, and the percentage of time spent in each of the phases of the gait cycle. Both the average and standard deviation of each metric is reported for each foot and for the differences between the feet. The method also performs a statistical test on the left and right metrics to determine if there is a statistically significant difference. According to one embodiment, and without limitation, the method uses a paired t-test with a 5% level of significance. Alternative embodiments include other parametric and nonparametric tests, including computer intensive methods such as bootstrap. The method also reports how many steps or sequences of steps were used to calculate each of the metrics. Other metrics could be easily computed and added to this list.

3.M. Display of the Results

It is often useful and instructive in many applications to visually display many characteristics of gait. Individual metrics can be plotted versus time. The left and right sides can be plotted separately or together on the same plot. As a visual guide, the system may plot characteristics from the left foot in blue and the right foot in red. Various colors or line types can be used.
Since the gait is divided into discrete events (i.e., steps) or sequences of events, there are many other methods of visual display that can be used to show the characteristics of the population of events. For example heatmaps (e.g, FIG. 7) in which the density of a characteristic versus time or as a percentage of the gait cycle can be displayed as an image with the metric value on one axis, the percentage or time on the horizontal axis, and a color map or grayscale axis for the pixel intensity. Overlapping trajectories can also be displayed with multiple traces of the position or orientation of the foot. Individual characteristics can be plotted as scatter plots.
In plots showing the average characteristics of a metric, a surrounding shading region can be used to show the variability of the metric as measured, for example, by a standard deviation, standard error of the mean, interquartile range, or a confidence interval. In our reports we usually show a 95% confidence interval or a standard deviation. Different embodiments implement a combination of graphical results as shown in the appendix to the specification.

Alternative Embodiments

There are several possible improvements to the methods described in previous sections. For example, the weighted template error used for template matching can be scaled and calculated such that the error during a still period is normalized to 1. This makes it easier to set thresholds that are more tolerant of variations in gait across subjects. Specifically the template error can be calculated as
$\begin{matrix} ε (n) = \frac{1}{N_{s}} \sum_{k = 1}^{N_{s}} ε_{k} (n) & (1) \end{matrix}$
where ε_kis the template error for a particular sensor or device. In one embodiment, the gyroscope and accelerometer magnitudes from the IMUs on both feet are used to determine the total template error resulting in a template error comprised of four components (N_s=4). Each component is calculated as
$\begin{matrix} ε_{k} (n) = \frac{1}{s_{k}} \frac{1}{m_{1} - m_{0} + 1} \sum_{ = m_{0}}^{m_{1}} {w_{}^{2} (x_{n - } - p_{})}^{2} & (2) \end{matrix}$
where l is an index representing the lag from the current time n of the signal segment {x_n}|_n=m ₀ ^m ¹, p_lis the average of the detected templates,
$\begin{matrix} p_{} = \frac{1}{N_{t}} \sum_{k = 1}^{N_{t}} x_{n_{k} - } & (3) \end{matrix}$
where n_kis the time index of the kth detected template, w_lis a weighting factor that can be calculated as the inverse of the standard deviation of the detected templates at lag l,
$\begin{matrix} w_{}^{2} = \frac{1}{\frac{1}{N_{t} - 1} \sum_{k = 1}^{N_{t}} {(x_{n_{k} - } - p_{})}^{2}} & (4) \end{matrix}$
The scaling factor s_kis chosen such that when the sensor is stationary, ε_k(n)=1. For the gyroscopes this is calculated as
$\begin{matrix} s_{k} = \frac{1}{m_{1} - m_{0} + 1} \sum_{ = m_{0}}^{m_{1}} {w_{}^{2} (p_{})}^{2} & (5) \end{matrix}$
and for the accelerometers this is calculated as
$\begin{matrix} s_{k} = \frac{1}{m_{1} - m_{0} + 1} \sum_{ = m_{0}}^{m_{1}} {w_{}^{2} (g - p_{})}^{2} & (6) \end{matrix}$
where g is the acceleration due to gravity (approximately 9.81). One with ordinary skill in the art will know that template error measures like the weighted squared error used above can be used with fast methods based on the fast Fourier transform. This method of weighting could also easily be adapted to other similarity measures such as mean absolute error, median absolute errors, and measures of correlation.
While particular embodiments have been described, it is understood that, after learning the teachings contained in this disclosure, modifications and generalizations will be apparent to those skilled in the art without departing from the spirit of the disclosed embodiments. It is noted that the foregoing embodiments and examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting. While the methods and apparatuses have been described with reference to various embodiments, it is understood that the words used herein are words of description and illustration, rather than words of limitation. Further, although the methods and apparatuses have been described herein with reference to particular means, materials and embodiments, the actual embodiments are not intended to be limited to the particulars disclosed herein; rather, the methods and apparatuses extend to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims. Those skilled in the art, having the benefit of the teachings of this specification, may effect numerous modifications thereto and changes may be made without departing from the scope and spirit of the disclosed embodiments in its aspects.

Claims

What is claimed is:

1. A method for gait characterization comprising:

(a) detecting zero-velocity periods using two or more wearable and wirelessly synchronized movement monitoring devices, said movement monitoring devices comprising a triaxial accelerometer and a triaxial gyroscope with a bandwidth of at least 15 Hz; and

(b) calculating temporal measures of gait during walking by estimating the change in position and orientation during each step.

2. The method of claim 1, wherein said calculating temporal measures of gait includes performing template matching based on the magnitude of said accelerometer's and said gyroscope's signals from both feet.

3. The method of claim 2, wherein said template matching is characterized by 1) enabling multiple iterations to refine a final template, 2) weighting each template by the standard deviation of said template across detected steps, 3) scaling the template's error to be equal to one when said movement monitoring devices are stationary, 5) using a fast method based on a fast Fourier transform configured for calculating the template's matching error, or combinations thereof.

4. The method of claim 3, wherein said calculating temporal measures of gait further comprises: a) detecting initial steps, b) estimating a gait cycle duration, c) building an initial template, d) calculating a template match, e) detecting initial template steps, f) validating steps, g) adding missed steps, or combinations thereof.

5. The method of claim 4, wherein calculating temporal measures of gait further comprises measuring asymmetry using time-proximate steps by combining left and right steps into consecutive left-right pairs.

6. The method of claim 5, wherein calculating temporal measures of gait further comprises characterizing gait during normal periods of walking by isolating sequences of steps in which the subject is traveling forward on a flat surface based on changes in height, bank angle, elevation angle, and heading angle.

7. The method of claim 6, wherein calculating temporal measures of gait further comprises generating an indicator of foot drop and fall risk by characterizing the pitch of the foot with wearable sensors at the moments of heel strike and toe-off.

8. The method of claim 7, wherein calculating temporal measures of gait further comprises characterizing the lateral deviation in a sequence of two steps resulting in three foot placements based on how far a middle foot placement deviates from a straight path from a first to a last foot placement.

9. The method of claim 8, wherein calculating temporal measures of gait further comprises measuring the lateral swing of the foot during a single step.

10. The method of claim 9, wherein said method comprises: 1) upsampling, 2) estimating biases, 3) calculating magnitudes, 4) finding still periods, 5) calculating positions, 6) detecting steps, 7) finding and validating step sequences, and 8) calculating gait metrics.