Falls in the elderly are a common phenomenon in daily life, which causes serious injuries and even death. Human activity recognition methods with wearable sensor signals as input have been proposed to improve the accuracy and automation of daily falling recognition. In order not to affect the normal life behavior of the elderly, to make full use of the functions provided by the smartphone, to reduce the inconvenience caused by wearing sensor devices, and to reduce the cost of monitoring systems, the accelerometer and gyroscope integrated inside the smartphone are employed to collect the behavioral data of the elderly in their daily lives, and the threshold analysis method is used to study the human falling behavior recognition. Based on this, a three-level threshold detection algorithm for human fall behavior recognition is proposed by introducing human movement energy expenditure as a new feature. The algorithm integrates the changes of human movement energy expenditure, combined acceleration, and body tilt angle in the process of falling, which alleviates the problem of misjudgment caused by using only the threshold information of acceleration or (and) angle change to discriminate falls and improves the recognition accuracy. The recognition accuracy of this algorithm is verified by experiments to reach 95.42%. The APP is also devised to realize the timely detection of fall behavior and send alarms automatically.

1. Introduction

Human behavior recognition has a rich and powerful role in many fields. It can be used for health detection, fall detection, and medical assistance, and it plays an important role in human health. In the field of medical assistance, a typical application is fall detection. Falling may cause serious injury to the elderly. Using smartphones or wearable smart devices can help detect falls in time and automatically send alarms. Therefore, the research and development of behavior recognition technology can not only help people’s daily life but also make people’s daily life more convenient and make the entire society more intellectual.

With the development of sensor devices, various sensors that can accurately collect human behavior activity data have been widely used on wearable devices such as bracelets, watches, and mobile phones. Compared with behavior recognition methods based on video images, behavior recognition based on sensors has the characteristics of low cost, flexibility, and good portability. Therefore, the research on human activity recognition based on wearable sensors has become a research hotspot in behavior recognition [1]. With the rapid development of sensors and electronic equipment, the improvement of performance, and the reduction of cost, these electronic components are more widely used in life [2]. In particular, the development of the smartphone and the good application prospects of human activity recognition in human health monitoring, entertainment, sports, and so on make sensor-based human activity recognition one of the research hotspots. Compared with the disadvantages of high cost and poor portability of deploying external devices to identify human activity status, wearable sensors and smartphones can easily collect various behavior data of the human body through integrated sensors to identify the human activity status [3].

The threshold analysis method is one of the main research methods in the field of human behavior recognition at home and abroad [4]. Data samples in domestic and foreign studies are mostly from accelerometers, gyroscopes, or both. At present, different machine learning algorithms or threshold analysis methods are used to study the falling behavior of the elderly at home and abroad. The machine learning algorithms used in the study include support vector machine (SVM) [5], decision tree, hidden Markov model (HMM) [6], k-nearest neighbor [7], extreme learning machine, neural network, and deep learning [8]. Some researchers determine the optimal threshold value by maximizing the possibility of fall to ensure a low misjudgment rate. The present research is very serious about the misjudgment of the fall process and similar motion state recognition. The recognition accuracy of human fall behavior is not high enough, and the recognition accuracy of different methods is not the same [9].

Human activities are complex and uncertain, so there will be a great misjudgment to judge the occurrence of human falls solely based on the relevant threshold information of acceleration or angle change. For example, the acceleration of a strenuous exercise, such as running or jumping, may exceed the threshold for falling behavior, but not for falling [10]. Due to different people and different sports behaviors, there will be a different energy expenditure. The analysis shows that the energy expenditure of human sports will have significant changes in the occurrence of falling behavior. Therefore, this paper proposes to take human movement energy expenditure a feature of fall behavior recognition.

From the view of the statistical analysis, the falling behavior is accompanied with great changes in body tilt angle, body movement energy expenditure, and acceleration in most cases and, then, other nonfalling behaviors, such as standing, walking, and sitting down in everyday life [11]. Therefore, this paper proposes a threshold analysis method to detect the occurrence of fall behavior from three aspects: human movement acceleration, human movement energy expenditure, and body tilt angle away from the vertical direction.

2. Falling Behavior Recognition Data Acquisition

With the improvement of people’s living standards, the smartphone has been widely used. People’s life is increasingly inseparable from smartphones, and they are basically inseparable from the phone. The research in this paper is to further make full use of the smartphone. To collect the data of users’ daily activities and behaviors for data analysis and verification of the algorithm proposed in this paper, a simple Android application (APP) is developed and deployed on the Huawei phone for data collection and real-time fall behavior detection. Figure 1 shows the user interface (UI) of the APP.

Details are in the caption following the image — **Figure 1 (a)**
Open in figure viewer PowerPoint

User interface for falling behavior recognition APP. (a) System login interface. (b) System function interface. (c) User feature interface. (d) Data acquisition interface. (e) Data management-interface. (f) Falling call interface.

Through the integrated acceleration sensor and gyroscope in the mobile phone, the data samples of kinematics-related information of the human body can be collected in real time. In the process of human movement, the acceleration and angle of different parts of the body are different [12]. To improve the accuracy of fall identification, the parts that are not easy to produce similar acceleration and angle with other daily activities should be chosen as the reference objects for information acquisition [13]. Because the wrist, upper arm, forearm, thigh, leg, and other parts in daily life are frequent and complex, have strong randomness, and are not conducive to the placement of the smartphone, they are not suitable as feature parts. However, the upper torso of the human body in normal daily life behaviors, such as walking and sitting, has a relatively stable movement change process. In contrast, in the process of falling, the speed change and angle change are more drastic, so it is more suitable for acceleration and other relevant information acquisition parts. In this paper, the upper pocket of the jacket is chosen as the placement position of the smartphone.

In view of [14–17] that use the threshold analysis method to identify human daily activities and behaviors, the number of experimental data collected is less than 10. For the safety of the elderly, it is not convenient to collect too many elderly people as volunteers for data collection. So, the data set required for this experiment came from the behavior and activity process of 5 old persons in daily life. The smartphone equipped with APP was placed in the chest pocket of each old person’s jacket, and the accelerometer and gyroscope data of 9 common behaviors and activities in daily life were collected. Since falling is not a frequent phenomenon in the normal life of the elderly, it is necessary to simulate the occurrence of falling on a mattress with a thickness of about 10 cm. Table 1 describes the details of the data collection.

Table 1. Basic data collection information.

Daily life behavior activities	Standing, walking, sitting, standing to sitting, sitting to standing, standing to squatting, squatting to standing up, going up the stairs, going down the stairs
Sensors	Triaxial accelerometer, gyroscope
Numbers of volunteers	5
Volunteers’ age	64∼78 years
Volunteers’ weight	57∼78 kg
Volunteers’ height	165∼176 cm
Sliding window size	0.5 s (25 sampling points)
Window overlap size	0.2 s (10 sampling points)

The frequency of data sampling directly affects the energy expenditure of the equipment, the number of data samples, and the complexity of future data processing. The higher the sampling frequency, the more detailed the data set, which can better represent different behaviors, but at the same time, it will bring much more burden to the equipment energy expenditure, calculation, and storage. On the contrary, when the sampling frequency is too low, the collected data sets cannot fully represent different behavioral characteristics. According to relevant work experience, the sampling frequency of each sensor is set to 50 Hz (i.e., 50 data points per second), which is enough to achieve excellent identification accuracy. The sampling period T = 20 ms.

Standing: 1 minute/time, 10 times.
Walking: 1 minute/time, 10 times, walking at normal speed in daily life.
Sitting: 1 minute/time, 10 times, sitting on a chair with armrests.
Standing to sitting: 10 times, at the normal rate of sitting in daily life.
Sitting to standing up: 10 times, at the rate of standing up normally in daily life.
Standing to squatting: 10 times, at the rate of normal squats in daily life.
Squatting to standing up: 10 times, at the speed of normal stand up in daily life.
Stepping up the stairs: 10 times per minute, walking up the stairs instead of taking the elevator.
Stepping down the stairs: 10 times per minute, walking down the stairs instead of taking the elevator.
Falling: 20 times, divided into forward fall, backward fall, right fall, and left fall, 5 times each. It simulates the fall of the human body under different circumstances, such as loss of control of lower limbs and dizziness, on a mattress with a thickness of about 10 cm.
Falling while walking: 10 times, falling forward while walking, that is, tripping.

The data obtained from data collection exists in the form of the data stream, which needs to be segmented. This paper uses the sliding window technique to split the raw data into data fragments that represent a single complete activity. The window size mainly refers to the duration of a single activity. Because a 0.5-second time window can well cover the whole process of a complete fall and impact, the sliding window size adopted in this paper for the segmentation of experimental data is 0.5 seconds (25 data sample points), with 40% overlap between the Windows. In the following paper, the unit time for calculating the energy expenditure of human movement is 0.5 seconds; that is, the energy expenditure value of human movement is calculated once within each time window [18].

3. Feature Extraction

Feature extraction is the key to behavior recognition research. The extracted features directly affect the subsequent calculation tasks and the accuracy of classification and recognition. The feature extraction methods of human behavior recognition research can be divided into three kinds: time-domain analysis, frequency-domain analysis, and time-frequency-domain analysis [19].

3.1. The Method of Feature Extraction

3.1.1. Time-Domain Analysis

The time-domain analysis method is used to extract features from sensor signals, that is, to directly extract features from acceleration and angle signals. This method is simple and has a small amount of calculation. The traditional time-domain characteristics include mean, standard deviation, signal power, correlation between axes, peak intensity, zero-crossing rate, and percentiles.

3.1.2. Frequency-Domain Analysis

After the sensor signal is converted from the time domain to the frequency domain, the corresponding features are extracted from the frequency domain. The commonly used time-domain to frequency-domain conversion methods include Fast Fourier Transform (FFT) and Discrete Cosine Transform (DCT). The frequency-domain features are FFT coefficient, DCT coefficient, Power Spectral Density (PSD), and frequency-domain entropy (FDE) [20].

3.1.3. Time-Frequency-Domain Analysis

Compared with the time domain analysis and frequency domain analysis, the traditional Fourier transform can only extract the frequency domain characteristics of the sensor signal but discard the time domain information of the signal. The emergence of wavelet transform can solve this problem well. Compared with the traditional Fourier transform, wavelet transform is the local transform of time and frequency, which solves many problems that cannot be solved by Fourier transform [21].

3.2. Extracted Features

Combined with the above feature extraction methods, the characteristics that distinguish human falls from other daily behaviors include combined acceleration, movement energy expenditure, and body tilt angle.

3.2.1. Combined Acceleration

Let the acceleration in the x-axis direction be a_x, the acceleration in the y-axis direction a_y, and the acceleration in the z-axis direction a_z. Then, the combined acceleration a is

(1)

The combined acceleration is an important parameter to distinguish the state of human movement. The smaller the acceleration a is, the gentler the human movement will be; the larger the acceleration a is, the more intense the human movement will be. When the body falls, there is a bump phase in contact with the ground, and the acceleration peaks, which is more pronounced than normal behavior [22]. Therefore, using a statistical method to determine the threshold value of the difference of the combined acceleration amplitude between the fall process and other daily life and behavior processes can provide a basis for the human fall identification method.

3.2.2. Movement Energy Expenditure

Different human movements will have different combined accelerations. According to the analysis of the data of combined accelerations collected by the smartphone and the calculation method of energy in physics, the equation of human movement energy expenditure is determined as follows [23]:

(2)

where E is the calorie of the human movement energy expenditure, u is the parameter (value u will be determined experimentally), mg is the body weight, a₁ and a₂ are the combined acceleration, and t₁ and t₂ represent time. E is used as a characteristic vector value of the algorithm in this paper. The threshold value selected by this characteristic quantity can better reflect the change of human movement and detect fall.

To calculate the u value in the calculation equation of human movement energy expenditure, based on the combined acceleration generated by volunteers when running at a speed of 10 km/h, the minimum and maximum values of the obtained combined acceleration data were, respectively, substituted into the calculation equation (2) of human movement energy expenditure. To improve the accuracy of calculation, the time unit of the experiment was set as 1 second; that is, the value of u was obtained by calculating the energy expenditure of the human body in motion within 1 second, which was verified in the movement energy expenditure detection at other speeds.

The detection of movement energy expenditure needs data comparison. “Double standard water detection method” is the gold standard, but its cost is high and the use is complicated, so it is not easy to implement it as a reference for the experiment. According to the existing human movement energy expenditure detection methods, in view of the advantages of convenient use, low cost, and high accuracy, the treadmill has become the standard of many energy expenditure detections at present. Life Fitness T5, the runner used in this experiment, can adjust the running speed range from 0.8 km/h to 19 km/h and the slope range from 0% to 15%, and the energy expenditure detection accuracy is above 95%.

When one volunteer was randomly selected to run on a treadmill at 10 km/h, the energy expenditure in 1 minute was 9500 cal, and the average energy expenditure in 1 second was 158 cal. So, the parameter u = 0.008 was obtained. By substituting this value into equation (2), the energy expenditure of the volunteer when running at 1 km/h and 5 km/h within 1 s is 15 cal and 46 cal, respectively, while the energy expenditure of the volunteer when running at 1 km/h and 5 km/h on the treadmill is 16 cal and 48 cal, respectively. Experimental results show that the measurement accuracy is above 94%.

3.2.3. Body Tilt Angle

The occurrence of human falling behavior will be accompanied by the change of the angle of the trunk. For example, in the case of a fall while walking or standing, the torso changes from a nearly vertical to a nearly horizontal orientation, with an inclination change of nearly 90 degrees. Since the human body rarely rotates when it falls, the angle of inclination (Yaw) around the x-axis θ_Y and the Pitch angle (Pitch) around the y-axis θ_p are selected as the angle features in this paper. The inclination threshold of the trunk deviating from the z-axis to distinguish the human fall process from other daily life behavior process is θ_T. The significance is as follows: in the process of fall, at least one of the angles of roll θ_Y and pitch θ_p after a fall is greater than θ_T. Before the fall and in the course of other daily life behaviors, the angle of roll θ_Y and pitch θ_p was less than θ_T.

4. The Support Vector Machine Method Is Adopted to Determine the Threshold

Threshold analysis is essentially a classification method based on linear discriminant function, which requires different categories of samples to be linearly separable. Support vector machine (SVM) is a kind of model classification technology with a good classification effect. In this paper, SVM is used to analyze human falling behavior and other activities of daily life, to obtain the threshold information of the combined acceleration value, human energy expenditure value, and angle value of the effective fall process.

4.1. Introduction to Support Vector Machine Methods

SVM uses linearly separable or nonlinearly separable training sample set; in the original feature space, the relative training sample set uses the optimization theory to generate the optimal linear discriminant function, which is implicitly mapped to the transformation feature space through the kernel function, so that the original pattern set is linearly separable in the transformation feature space.

Since the purpose of the research in this paper is to distinguish human falling behavior from other behaviors in daily life, it is a two-type problem. In the dichotomy problem, the problem can be described as follows: let the given n-dimensional training sample set {x₁, x₂, …x_N}. There are two sample sets, corresponding to two attribute values, which are, respectively, the attribute value y_i = 1 of the training sample {x_i} of the class P₁ and the attribute value y_i = −1 of the training sample {x_j} of the class P₂. Therefore, each training sample can be reexpressed as (x₁, y₁), (x₂, y₂), …, (x_N, y_N). Assuming that the training samples are linearly separable, there is a linear interface that can divide the training samples into two categories [24]. Obviously, there are many of these decision boundaries, many of the linear discriminant functions that have been selected.

The goal is to obtain a classification interface that can not only correctly separate the two types of sample elements but also maximize the distance between the classification interface and the nearest sample point. In other words, the selected linear classification interface should make the distance between the two planes parallel to the interface pass through the point closest to the classification interface of the two types of samples respectively to the maximum, so it should be the middle position of the two planes, as shown in Figure 2. This classification interface is also called the optimal interface, and its corresponding classifier is called the maximum allowance classifier. The distance between the two planes is called the classification interval, and the training samples on the two planes determine the optimal classification interface, which is called the support vector. Obviously, the generalization of the optimal classification interface is the best.

In the n-dimensional eigenspace Xⁿ of the original model, the general form of linear discriminant functions for the two kinds of problems can be expressed as [24]

(3)

The corresponding equation of classification interface H based on linear model-like decision surface (hyperplane) can be written as

(4)

where w is the normal vector of the decision-making surface, which determines the direction of the decision-making surface, and b determine the position of the decision plane. Thus, the distance between the sample point x and the decision surface can be written as

(5)

So, maximizing r is the same as minimizing ‖w‖. To obtain ‖w‖, the Lagrange function is made:

(6)

SVM can be used to classify human fall behavior and the kinematics process of other daily life behaviors and activities, and sample training of these two kinds of sample sets is needed to determine the threshold.

4.2. Determination of Thresholds

First of all, the data samples of falling behaviors and other movement behaviors in daily life collected by the smartphone worn by volunteers were preprocessed and grouped into two disjoint groups, which were, respectively, used for sample training and experimental verification of SVM:

P_T: training samples are used to obtain the optimal classification interface in the classification method of SVM. The training sample set contains 90 data samples of falling behavior and 30 data samples of other daily life behaviors, and a total of 360 data samples.
P_E: experimental samples are used to conduct experiments on the recognition effect of the threshold method to determine whether there is a fall. The experimental sample set contains data samples of 60 groups of fall behaviors (10 groups of forwarding falls, 10 groups of left falls, 10 groups of right falls, 10 groups of backward falls, and 20 groups of falls during walking) and 20 groups of other activities in daily life, a total of 240 data samples.
Then, the data samples in the training sample set P_T are divided into two disjoint sets according to whether it is fall or not, namely, the two kinds of sample sets in the SVM training process.
P_F: fall activities data set of human fall behaviors consist of 90 groups of fall behavior data samples, including 15 groups of forwarding falls, 15 groups of left falls, 15 groups of right falls, 15 groups of backward falls, and 30 groups of falls during walking.
P_O: data sample set of other daily life activities, consisting of 30 groups of each group and 270 groups of data sample sets.

The relation of sets P_T, P_E, P_F, and P_O is

(7)

whereϕ is the empty set.

Next, the training sample set is P_T trained by the SVM method to obtain the classification threshold. The threshold values of the combined acceleration, movement energy expenditure, and tilt angle of sensors used to distinguish human falling behavior from other daily life behaviors are denoted as a_T, E_T, and θ_T, respectively. The occurrence algorithm of human fall behavior detection based on the threshold method will be introduced in the next section. To increase the flexibility of the algorithm, the experiment in this paper sets, a_T, E_T, and θ_T, respectively, so that different classification method can be combined.

4.2.1. Determination of the Combined Acceleration Threshold a_T for Fall Recognition

Since the peak value of the combined acceleration in the process of falling is larger than that in other daily activities (except for violent activities such as running and jumping), the peak value of the combined acceleration in the process of human movement is taken as one of the characteristics to distinguish falling from other daily activities. Take the sum of the data sample sets P_F and P_O, calculate the combined acceleration of each sample, and get the combined acceleration peak of the moving process related to each sample, which are, respectively, denoted as sets P_a1 and P_a2.

The sum of the combined peak acceleration P_a1 and P_a2is extracted from the sum of two training sample sets P_F and P_O, respectively, so the sum of P_a1 and P_a2 is independent of each other. Assuming that they are linearly separable, the SVM method is adopted to determine the optimal classification interface.

Since the elements in sets P_a1 and P_a2 of the combined peak acceleration are all combined acceleration values and belong to one-dimensional data, the dimension n = 1 of the feature space Xⁿ. The optimal classification hyperplane is essentially a point:

(8)

According to the introduction in Section 4.1, the optimal classification hyperplane is x = 29.16 m/s².

So, the classification rule is as follows: if x ≤ b, then set x ∈ P_a1. If x > b, then set x ∈ P_a2. That is to say, the threshold value of the combined acceleration is a_T = 29.16 m/s². When the combined acceleration of human movement a > a_T occurs at a certain moment, the movement is judged to be a fall; otherwise, the movement is judged to be other movement behaviors in daily life.

In the above threshold calculation process, statistical analysis is carried out on the basis of the training sample set. In this sample set, P_a1∩P_a2 = ϕ, P_a1 and P_a2 are linearly separable for one-dimensional samples. But in real everyday life, P_a1 and P_a2 in some cases are nonlinearly separable. For example, running, jumping, and other more intense activities in the peak acceleration a_T may exceed the threshold, but there is no fall. One the contrary for the fall of leaning on the wall when dizziness occurs, the peak acceleration a_T may be less than the threshold value, but there is a falling behavior. Therefore, the threshold a_T value cannot be used as the only criterion to identify whether a fall has occurred. However, the combined acceleration threshold a_T is still important for the identification of falls because violent sports such as running and jumping rarely occur among the elderly, the impact force on the body is not large, and the impact of injury to the elderly is relatively small for the fall situation of slowly sliding on the wall.

4.2.2. Determination of Human Movement Energy Expenditure E_T for Fall Recognition

The body will act with a great shock in the process of falling; the movement energy expenditure has a close relationship with the force. Therefore, the movement energy expenditure in the process of falling is higher than other daily life actions, so take the movement energy expenditure as one of the characteristics to distinguish fall and other daily activities.

Since the research focus of this paper is to identify falls of the elderly living alone, the data samples collected are all from the daily indoor living activities of the elderly. To better distinguish violent sports, such as running, from falls and expand the application objects and scenarios for recognizing falls based on the smartphone, this part of the experimental sample set added 80 groups of running, fast upstairs, and fast downstairs sports from young volunteers. The training sample set increases fast running and going up the stairs and down the stairs in 50 groups; the experimental sample set increases running and going up the stairs and down the stairs quickly in 30 groups. The training sample set, experimental sample set, and other sports behavior data set in daily life after adding samples of running, going up the stairs, and going down the stairs quickly are denoted, respectively, as sets , , and .

Take the data sample sets P_F and , calculate the movement energy expenditure value E of each sample, and get the movement energy expenditure value related to each sample, denoted as sets P_E1 and P_E2, respectively.

Assuming that P_E1 and P_E2 are linearly separable, the SVM method is adopted to determine the optimal classification interface. Since the elements in the human movement energy expenditure sets P_E1 and P_E2 are all human movement energy expenditure values, which belong to one-dimensional data, the dimension n = 1 of the feature space Y_n. The optimal classification hyperplane is essentially a point:

(9)

According to the introduction in Section 4.1, the optimal classification hyperplane is y = 116 cal.

So, the classification rule is as follows: if y ≤ b, then set y ∈ P_E1 and if y > b, then set y ∈ P_E2. In other words, the threshold value of human movement energy expenditure is E_T = 116 cal. In general, during daily fast running, E will not exceed 116 cal. However, the impact force of falling is large, so the body’s combined acceleration will be large. As a result, the energy expenditure of the human body during falling will exceed 116 cal. When the energy expenditure value of human movement Ε > E_T at a certain moment, the movement is judged to be a fall; otherwise, the movement is judged to be other movement behaviors in daily life.

In the above threshold calculation process, statistical analysis is carried out on the basis of the training sample set . In this sample set, P_E1∩P_E2 = ϕ, and P_E1 and P_E2 are linearly separable for one-dimensional samples. But in real everyday life, P_E1 and P_E2 in some cases are nonlinearly separable. For example, when a person swings somewhere, the calculated energy expenditure of a person’s movement may exceed the threshold E_T, but no fall occurs. Therefore, the threshold E_T cannot be used as the only criterion to identify whether a fall has occurred. However, the threshold of human movement energy expenditure is still an important basis for fall recognition, and the threshold E_T can be used as a condition to identify whether a fall occurs. Combined with the judgment of the combined acceleration, the misjudgment can be further reduced, and the accuracy of fall recognition can be improved.

4.2.3. Determining the Tilt Angle θ_T of Fall Recognition

The direction of the body will change dramatically when the fall occurs. Generally, the body is in an upright position before the fall, and the final state of the fall is lying flat, lying on the back, prone, or side, and the body is in a horizontal or nearly horizontal state. Therefore, the tilt angle of the body away from the z-axis during human movement can be taken as one of the characteristics to distinguish falls from other daily behavior activities. Sample sets P_F of fall behavior data were taken to calculate the tilt angle θ of the human body from the z-axis after the fall in each sample. Here, the tilt angle included roll angle θ_Y and pitch angle θ_P, and they were recorded as the set P_θ1. Then, take the data sample set of other daily life behaviors and activities, calculate the inclination angle of the body deviating from the z-axis in each sample at each moment, and write them as set P_θ2.

Similar to processing the combined acceleration threshold, P_θ1 and P_θ2 of the inclination angle set are extracted from the two training sample sets P_F and P_O, respectively, so P_θ1 and P_θ2 are independent of each other. Assuming that P_θ1 and P_θ2 are linearly separable, the SVM method is adopted to determine the optimal classification interface.

Since the elements in sets P_θ1 and P_θ2 are all body tilt angle values and belong to one-dimensional data, the dimension n = 1 of the feature space Z_n. The optimal classification hyperplane is essentially a point:

(10)

According to the introduction in Section 4.1, the optimal classification hyperplane is z = 41.36°.

So, the classification rule is as follows: if Z ≤ b, then set Z ∈ P_θ1, and if Z > b, then set Z ∈ P_θ2. In other words, the threshold θ_T value of the body tilt angle away from the z-axis is θ_T = 41.36°. When at least one of the tilt angles of the body is away from the z-axis, the angle of roll θ_Y and pitch θ_P is greater than the angle of θ_T at a certain moment, the movement is considered a fall. Otherwise, the movement is considered as other movement behaviors in daily life.

In the above threshold calculation process, statistical analysis is carried out on the basis of the training sample set P_T. In this sample set P_θ1∩P_θ2 = ϕ, P_θ1 and P_θ2 are linearly separable for one-dimensional samples. But in real everyday life, P_θ1 and P_θ2 in some cases are nonlinearly separable. For example, when you lie down to sleep, bend over to get something, your body leans over the threshold θ_T, but you do not fall. Therefore, the value of the threshold θ_T cannot be used as the only criterion to identify whether a fall has occurred. However, combining with the judgment of the combination acceleration and movement energy expenditure can further reduce the misjudgment and improve the recognition rate of fall behavior.

5. The Three-Level Human Fall Recognition Algorithm Based on Threshold Analysis

Considering the complexity and randomness of the human movement process, there will be a lot of misjudgment when using a single acceleration or angle-related information to identify human fall behavior. Therefore, using the accelerometer and gyroscope sensor, respectively, for acceleration and angle information, acceleration in the process of human movement information is obtained by information fusion and angle information to assess the impact of movement energy expenditure and the change of the posture, set a threshold value, and complete the movement behavior of human body detection, to identify fall behavior. Falling behavior can be identified more accurately by combining the threshold of body combined acceleration a_T, the threshold of movement energy expenditure E_T, and the threshold θ_T of body tilt angle from the z-axis in the process of human movement to reduce misjudgment. This is an important method for human fall behavior recognition.

The whole process of human falls usually takes place within 2 to 3 seconds. Therefore, when combining the body’s combined acceleration information, the movement energy expenditure information, and the body’s tilt angle away from the z-axis information, we need to consider the correlation of the three pieces of information. When a general fall occurs, the body will be subjected to a large impact force, the combined acceleration will have a peak, and the energy expenditure per unit of time will have a large peak. However, in some cases, the body will be impacted by several forces (e.g., falling down the stairs, the body will hit multiple steps), the combined acceleration will have multiple peaks, and the energy expenditure of the movement will remain high. In this case, the fall can take more than 2 to 3 seconds.

The magnitude of the impact force suffered by the human body in the process of falling can be estimated and roughly expressed by the magnitude of the combined acceleration. Let the combined acceleration at time t be a(t). Combined with the threshold value a_T of the combined acceleration of the fall behavior obtained in Section 4, the basis of fall recognition can be obtained as follows:

(a)
When a(t) ≤ a_T, there will be no impact and no fall
(b)
When a(t) > a_T, there is an impact and there may be falling behavior

The movement energy expenditure can be estimated and roughly expressed by the magnitude and duration of acceleration. Let the energy expenditure in the t period be E(t). Combined with the threshold value E_T of human movement energy expenditure for fall behavior obtained in Section 4, the basis of fall recognition can be obtained as follows:

(a)
When E(t) ≤ E_T, the body dynamics change little, the energy expenditure is very low, and there will be no fall
(b)
When E(t) > E_T, the body dynamic changes greatly, the movement energy expenditure is high, and a fall may occur

When a fall occurs, the direction of the body will change. Generally, the body’s posture is upright before the impact. After the fall, the body’s posture becomes horizontal or nearly horizontal, and the tilt angle of the body away from the z-axis changes by nearly 90 degrees. Denote the inclination angle θ(t) of the body away from the z-axis at time t, and the inclination angle includes roll angle θ_Y and pitch angle θ_P. Combined with the threshold of the inclination angle θ_T of body deviation from the z-axis obtained in Section 2.4, the basis of fall recognition can be obtained as follows:

(a)
Before the fall: θ(t) ≤ θ_T; then, θ_Y ≤ θ(t) and θ_P ≤ θ(t)
(b)
After the fall: θ(t) > θ_T; then, θ_Y > θ(t) or θ_P > θ(t) or θ_Y, θ_P > θ(t)

Thus, a human fall recognition algorithm based on the combined acceleration of human movement, the energy expenditure of movement, and the tilt angle of the body away from the z-axis is obtained, which is denoted as Algorithm 1. Algorithm 1 will follow the following steps:

Algorithm 1: The threshold-based three-level human fall recognition algorithm.

Step 1: The system is initialized and ready to collect information related to human movement. According to the definition of coordinate direction introduced, the triaxial acceleration information of the upper trunk and the angular velocity information of the body around the x-axis and y-axis were collected, and the combined acceleration a(t) at time t, the movement energy expenditure E(t) at time t, and the tilt angle θ(t) of the body away from the z-axis were calculated. Define a data storage space and store the combined acceleration, human movement energy expenditure, and body tilt angle in the form of a sliding time window within 2 seconds before the moment. When the new data arrive, the sliding time window moves forward, the new data are stored, and the earliest data storage space is released.
Step 2: Test whether the first impact occurs at the current moment: if a(t) ≤ a_T, there is no impact state, and no fall will occur. Continue monitoring and return to Step 1. If a(t) > a_T, the first impact is confirmed, and a fall may occur, enter Step 3.
Step 3: Measure the energy expenditure of human movement calculated at the current moment: if E(t) ≤ E_T the dynamic change of body movement is not large, the energy expenditure of movement is low, and there will be no fall, but only fast walking, running, and so on. Continue monitoring and return to Step 1. If E(t) > E_T, the dynamic change of body movement is large, and the energy expenditure is high, a fall may occur. Record the time at this time as t₀, and enter Step 4.
Step 4: The movement information t₁ = t₀ − 2 of the human body in the first 2 seconds of the time t₀ is recorded, and the tilt angle of the body from the z-axis is denoted as below.
Step 5: Test whether the impact state is over and whether it is multiple impact falls. The detection method of whether there is still impact in this step is as follows: obtain the relevant motion information within 1 second after the current moment to detect whether there is impact, that is a(t) > a_T, and whether it is still valid. If it is true, it means that the fall is a multi-impact fall, and the impact is not over yet, so the detection of the impact state should be continued in Step 5.
Step 6: If a(t) > a_T, it is not true; it means that the impact has ended. The end time of the last impact is recorded as t₂. At this point, the relevant information of human motion is continued to be obtained to get the tilt angle of the body from the z-axis in seconds t₂ + 1, denoted as θ₂.
Step 7: To determine whether the change condition of body tilt angle meets the characteristics of falling behavior, if θ₁ ≤ θ_T and θ₂ > θ_T, it is judged that the fall behavior process has occurred. Otherwise, it is judged that the falling behavior did not occur, and real-time monitoring should be continued.

6. Experimental Results and Analysis

To verify the effectiveness and accuracy of the three-level threshold test method proposed in this paper, which is based on the combined acceleration, the movement energy expenditure, and the tilt angle of the body away from the z-axis, for the identification of human fall behavior, the information processing process of Algorithm 1 is tested. The sample allocation method used in Section 4.2 to determine the threshold is used to segment the data samples, and all the data sample sets are divided into two disjoint sets, P_T, P_E, and their relation is P_T∩P_E = ϕ, ϕ is the empty set.

P_T: training sample set, including 90 groups of falling behavior data samples and 30 groups of other behavior data samples in daily life, a total of 360 data samples.

P_E: experimental sample set, including 60 groups of fall behavior data samples (10 groups of forwarding falls, 10 groups of left fall, 10 groups of right fall, 10 groups of backward fall, 20 groups of fall in walking) and 20 groups of other behavior data samples in daily life, a total of 240 data samples.

Therefore, 240 data samples from the experimental sample set P_E are used to conduct a comparative experiment on Algorithm 1. In the algorithm, the threshold E_T of combined acceleration, the threshold of movement energy expenditure, and the threshold θ_T of body tilt angle from z-axis are, respectively, a_T = 29.16 m/s², E_T = 116 cal, θ_T = 41.36°

To compare and verify the recognition effect of Algorithm 1 on the fall behavior process and other behavior processes in daily life, the samples in the experimental sample set are divided into the following eleven disjoint sets according to the type of behavior and activity to which the samples belong:

P₁: standing sample set, including 20 groups of data samples of standing behavior.
P₂: walking sample set, including 20 groups of walking behavior process data samples.
P₃: the seated sample set, consisting of 20 data samples of chair sitting behavior.
P₄: the standing-sitting sample set, including 20 data samples.
P₅: the sitting-up sample set, including 20 data samples of the sitting-up behavior process.
P₆: standing and squatting sample set, including 20 groups of standing and squatting behavior data samples.
P₇: squatting and standing up sample set, including 20 groups of squatting and standing up behavioral process data samples.
P₈: the stair climbing sample set, including 20 groups of stair climbing behavior data samples.
P₉: stairway down sample set, including 20 groups of stairway down behavior process data samples.
P₁₀: fall sample set, including 40 groups of data samples of fall behavior process.
P₁₁: sample set of falls during walking, including data samples of 20 groups of forwarding falls during walking. Falling in walking is actually a coherent behavioral process integrating three behavioral processes of standing-walking-forward falling as one.

(11)

Algorithm 1 is based on the triaxial accelerometer and gyroscope fall recognition algorithm of the human body, with the experimental sample set on the experimental verification, namely, the body acceleration value and human movement energy expenditure value were calculated by three axial acceleration information, and the body angle deviating from the z-axis was calculated by the gyro information. According to the process of Algorithm 1, whether human falling behavior occurs is detected.

The process of falling in walking is actually a coherent movement of standing, walking, and forward falling, and its research and analysis are representative. Figures 3(a)–3(c) show the change curves of the combination acceleration, movement energy expenditure, pitch angle, and roll angle after information fusion in this process.

Figure 3(a) shows that, in the standing and prone stages, the combined acceleration value is close to G, and the body movement is relatively stable. However, in the walking stage, the resultant acceleration changes periodically, and its peak value is always less than the threshold value, failing to meet the conditions for the occurrence of falls. For the stage of standing, walking, and prone, the dynamic changes of the body were not significant, and the movement energy expenditure was lower than 35 cal. In Figure 3(c), the purple dotted line curve is the change curve of roll angle, and the red real curve is the change curve of pitch angle. As can be seen in the figure, the pitch angle changes a lot and the roll angle changes slightly when the forward fall occurs.

All the above experimental samples are tested by Algorithm 1 to get the recognition results of various movement behaviors and falls in the daily life of the elderly, as shown in Table 2.

Table 2. Experimental results of Algorithm 1.

Sample set	Identify the results
Sample set	Fall behavior	Other daily sports activities
Standing P₁	0	20
Walking P₂	0	20
Sitting P₃	0	20
Standing to sitting P₄	1	19
Sitting to standing P₅	2	18
Standing to squatting P₆	2	18
Squatting to standing P₇	1	19
Upstairs P₈	0	20
Downstairs P₉	1	19
Falling P₁₀	36	4
Fall while walking P₁₁	20	0
Test P_E	63	177

The research of human fall behavior recognition needs to have high enough recognition accuracy to identify fall behavior from people’s daily activities. Fall recognition is generally evaluated using three indicators based on the four possible situations in Table 3.

Table 3. Four possible cases of fall recognition.

Detect	Fall occur	Fall not occur
Fall detected	True positive (TP)	False positive (FP)
No fall detected	False negative (FN)	False negative (TN)

Among them, TP (true positives) is the sample number of falling behaviors recognized correctly. FN (false negatives) is the sample number of falling behaviors that have not been recognized. TN (true negatives) refers to the sample number of other daily life behaviors that can be correctly identifed without falls occurring. FP (false positives) is the sample number of other daily life behaviors misjudged as falling.

Sensitivity is the ability to detect falls. It is given by the ratio between the number of detected falls and the total falls that occurred:

(12)

Specificity is the ability to avoid false positives. Intuitively, it is the ability to detect a fall only if it occurs:

(13)

Sensitivity and specificity may contribute to a better knowledge of some of the limitations of fall recognition. Accuracy is a global indicator, and it is the ability to distinguish and detect fall (TP) and nonfalls (TN):

(14)

P represents the number of falls that occurred and N represents the number of falls that did not occur.

After calculation, the sensitivity = 93.33%, specificity = 96.11%, and accuracy = 95.42%.

In [24], the threshold analysis method was adopted to identify falling behavior based on acceleration threshold information. The experimental results were as follows: sensitivity = 88.75% and specificity = 96.25%, which were lower than the identification results in this paper. The advantages of the three-level threshold algorithm proposed in this paper for human falling behavior recognition are obvious.

In practical application, because the specificity of the algorithm is not very accurate, some misjudgments will occur. To reduce unnecessary medical expenses and the excessive concern of related personnel, the smartphone will beep an alarm when detecting the user’s fall. If it is a misjudgment, the user can manually cancel the alarm within 30 seconds. If the buzzer is not cancelled within 30 seconds [25], it means that the user may have lost the ability, and the APP will send an alarm automatically.

7. Conclusions

This research work made full use of accelerometer and gyroscope built in the smartphone, the body movement energy expenditure value is introduced as a new feature, and an algorithm is proposed through the combined acceleration, movement energy expenditure, and the relative vertical direction angle in the process of human movement to judge whether the fall occurs. The recognition accuracy of the algorithm is tested. The experiment verifies that the accuracy of the fall behavior recognition is 95.42%, higher than the recognition accuracy in the analysis research with a similar threshold value, which verifies the practical effectiveness of the fall behavior recognition based on the built-in sensor of the smartphone. The full use of the smartphone can realize the detection of falling behavior and automatically send an alarm timely.

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

This work was supported by the National High Technology Research and Development Program of China (863 Program) under Grant 2013AA01A215.

Open Research

Data Availability

All data used to support the findings of the study are included within the article.

References

1 Pyo S., Lee J., Kim W., Jo E., and Kim J., Multi-layered, hierarchical fabric-based tactile sensors with high sensitivity and linearity in ultrawide pressure range, Wiley Online Library. (2019) 5, no. 5.
Google Scholar
2 Dong X., Zhang L., Milholland B., Lee M., Maslov A. Y., Wang T., and Vijg J., Accurate identification of single-nucleotide variants in whole-genome-amplified single cells, Nature Methods. (2017) 14, no. 5, 491–493, https://doi.org/10.1038/nmeth.4227, 2-s2.0-85015632028.
10.1038/nmeth.4227
CAS PubMed Web of Science® Google Scholar
3 Wu D. C., Application of smart phone in elderly fall detection [J], Digital Communication World. (2019) 169, no. 1, 210–211.
Google Scholar
4 Liu J., Shahroudy A., Xu D., Kot A. C., and Wang G., Skeleton-based action recognition using spatio-temporal LSTM network with trust gates, IEEE Transactions on Pattern Analysis and Machine Intelligence. (2017) 40, no. 12, 3007–3021, https://doi.org/10.1109/tpami.2017.2771306, 2-s2.0-85033709940.
10.1109/TPAMI.2017.2771306
PubMed Web of Science® Google Scholar
5 Pan D., Liu H., Qu D., and Zhang Z., Human falling detection algorithm based on multisensor data fusion with SVM, Mobile Information Systems. (2020) 2020, 9, 8826088, https://doi.org/10.1155/2020/8826088.
10.1155/2020/8826088
Web of Science® Google Scholar
6 Lim D., Park C., Kim N. H., Kim S.-H., and Yu Y. S., Fall-detection algorithm using 3-axis acceleration: combination with simple threshold and hidden markov model, Journal of Applied Mathematics. (2014) 2014, 8, https://doi.org/10.1155/2014/896030, 2-s2.0-84937064781, 896030.
10.1155/2014/896030
Google Scholar
7 Mazurek P., Wagner J., and Morawski R. Z., Use of kinematic and mel-cepstrum-related features for fall detection based on data from infrared depth sensors, Biomedical Signal Processing and Control. (2018) 40, 102–110, https://doi.org/10.1016/j.bspc.2017.09.006, 2-s2.0-85029698638.
10.1016/j.bspc.2017.09.006
Web of Science® Google Scholar
8 Ma L., Cheng S., and Shi Y., Enhancing learning efficiency of brain storm optimization via orthogonal learning design, in IEEE Transactions on Systems, Man, and Cybernetics: Systems. (2021) 51, no. 11, 6723–6742.
10.1109/TSMC.2020.2963943
Web of Science® Google Scholar
9 Van Thanh P., Tran D.-T., Nguyen D.-C., Duc Anh N., Nhu Dinh D., El-Rabaie S., and Sandrasegaran K., Development of a real-time, simple and high-accuracy fall detection system for elderly using 3-DOF accelerometers, Arabian Journal for Science and Engineering. (2019) 44, no. 4, 3329–3342, https://doi.org/10.1007/s13369-018-3496-4, 2-s2.0-85059517024.
10.1007/s13369-018-3496-4
Web of Science® Google Scholar
10 Hakim A., Huq M. S., Shanta S., and Ibrahim B. S. K. K., Smartphone based data mining for fall detection: analysis and design, Procedia Computer Science. (2017) 105, 46–51, https://doi.org/10.1016/j.procs.2017.01.188, 2-s2.0-85016131722.
10.1016/j.procs.2017.01.188
Google Scholar
11 Chen L., Li R., Zhang H., Tian L., and Chen N., Intelligent fall detection method based on accelerometer data from a wrist-worn smart watch, Measurement. (2019) 140, 215–226, https://doi.org/10.1016/j.measurement.2019.03.079, 2-s2.0-85064197566.
10.1016/j.measurement.2019.03.079
Web of Science® Google Scholar
12 Kanarachos S., Mathew J., and Fitzpatrick M. E., Instantaneous vehicle fuel consumption estimation using smartphones and recurrent neural networks, Expert Systems with Applications. (2019) 120, 436–447, https://doi.org/10.1016/j.eswa.2018.12.006, 2-s2.0-85057747807.
10.1016/j.eswa.2018.12.006
Web of Science® Google Scholar
13 Xue X., Wu X., Jiang C., Mao G., and Zhu H., Integrating sensor ontologies with global and local alignment extractions, Wireless Communications & Mobile Computing. (2021) 2021, 10, https://doi.org/10.1155/2021/6625184, 6625184.
10.1155/2021/6625184
Web of Science® Google Scholar
14 Gjoreski H., Kozina S., Gams M., and Luštrek M., RAReFall--real-time activity recognition and fall detection system, Proceedings of the IEEE International Conference on Pervasive Computing & Communications Workshops, March 2014, Budapest, Hungary, IEEE, 145–147.
Google Scholar
15 Naranjo-Hernández D., Roa L. M., Reina-Tosina J., and Estudillo-Valderrama M. A., SoM: a smart sensor for human activity monitoring and assisted healthy ageing, IEEE Transactions on Biomedical Engineering. (2012) 59, no. 11, 3177–3184, https://doi.org/10.1109/tbme.2012.2206384, 2-s2.0-84868144308.
10.1109/TBME.2012.2206384
PubMed Web of Science® Google Scholar
16 Curone D., Bertolotti G. M., Cristiani A., Secco E. L., and Magenes G., A real-time and self-calibrating algorithm based on triaxial accelerometer signals for the detection of human posture and activity, IEEE Transactions on Information Technology in Biomedicine. (2010) 14, no. 4, 1098–1105, https://doi.org/10.1109/titb.2010.2050696, 2-s2.0-77954591953.
10.1109/TITB.2010.2050696
PubMed Google Scholar
17 Zhang S., Mccullagh P., Nugent C., and Zheng H., Activity monitoring using a smart phone’s accelerometer with hierarchical classification, Proceedings of the Sixth International Conference on Intelligent Environments, July 2010, Kuala Lumpur, Malaysia, IEEE.
Google Scholar
18 Md M. P., The promise of smartphone fall detection solutions for falls in older adults, Journal of the American Geriatrics Society. (2015) 63, 1969–1970.
10.1111/jgs.13640
PubMed Web of Science® Google Scholar
19 Shen X. W. and Mao W. T., Design of anti-fall system for the elderly based on extreme learning machine, Internet of Things Technology. (2015) 20, no. 5, 55–58.
Google Scholar
20 Hsu Y. W., Chen K. H., Yang J. J., and Jaw F.-S., Smartphone-based fall detection algorithm using feature extraction, Proceedings of the International congress on image and signal processing, biomedical engineering and informatics, October 2017, Datong, China, IEEE, 1535–1540.
Google Scholar
21 Ma L., Cheng S., and Shi Y., Enhancing learning efficiency of brain storm optimization via orthogonal learning design, IEEE Transactions on Systems, Man, and Cybernetics: Systems. (2020) 51, no. 11, 6723–6742, https://doi.org/10.1109/TSMC.2020.2963943.
10.1109/TSMC.2020.2963943
Web of Science® Google Scholar
22 Liu L., Zheng D., and Liu X., Design of fall detection system for elderly people based on MPU6050 sensor, Chinese Journal of Medical Instrumentation. (2015) 39, no. 5, 327–330.
Google Scholar
23 Zhu G. Z. and Wei C. H., Research on human motion energy consumption detection algorithm based on 3D acceleration sensor, Journal of Sensor Technology. (2011) 24, no. 8, 1217–1222.
Google Scholar
24 Tong L. N., Research on Human Fall Process Recognition Method Based on Mechanical Information Acquisition System, 2011, University of Science and Technology of China, Hefei, China.
Google Scholar
25 Weng P. C. and Zhang Y. H., Design of fall detection and positioning system based on MEMS, Electronic Technology and Software Engineering. (2017) 12, 85–86.
Google Scholar

Citing Literature

All articles

Human Falling Recognition Based on Movement Energy Expenditure Feature

Abstract

1. Introduction

2. Falling Behavior Recognition Data Acquisition