WO2021085231A1

WO2021085231A1 - Emotion estimation method and emotion estimation system

Info

Publication number: WO2021085231A1
Application number: PCT/JP2020/039356
Authority: WO
Inventors: 泰之浦岡; 耕一村田; 田窪　健二; 雅史古田; 弥佐藤
Original assignee: 株式会社島津製作所; 国立大学法人京都大学
Priority date: 2019-10-30
Filing date: 2020-10-20
Publication date: 2021-05-06
Also published as: JPWO2021085231A1; JP7311118B2

Abstract

This emotion estimation method comprises a first, second, and third step. The first step is a step for acquiring a plurality of myogenic potential signals (MS1, MS2) that respectively correspond to different types of muscles out of the muscles in the face of a subject. The second step is a step for calculating a plurality of emotion indices (E1, E2) on the basis of the plurality of the myogenic potential signals (MS1, MS2). The third step is a step for using information that indicates the relationship between the plurality of emotion indices (E1, E2) and human emotions to estimate the emotion of the subject from the plurality of emotion indices (E1, E2).

Description

Emotion estimation method and emotion estimation system

This disclosure relates to an emotion estimation method and an emotion estimation system.

In the field of psychology, various models for expressing human emotions have been proposed. For example, Non-Patent Document 1 discloses "Russell's ring model". Russell's ring model is a model that can express various emotions on a two-dimensional plane having a horizontal axis representing comfort-discomfort and a vertical axis representing arousal-sleepiness.

A technique has been proposed in which a myoelectric potential signal is acquired from the muscles of the subject's face and the emotion of the subject is estimated based on the myoelectric potential signal.

The present inventors have focused on the fact that human beings have more complicated emotions that cannot be classified only by pleasure / discomfort. For example, in sports or games, when points are lost or lost at the end of a fierce battle, feelings of "disappointing but fun" may occur. It is desirable that such complex emotions can be estimated from the myoelectric potential signal of the facial muscles of the subject.

This disclosure was made to solve such a problem, and the purpose of this disclosure is to provide a technique capable of estimating complex emotions.

The emotion estimation method according to the first aspect of the present disclosure includes the first to third steps. The first step is to acquire a plurality of myoelectric potential signals corresponding to different types of muscles among the facial muscles of the subject. The second step is to calculate a plurality of emotional indices based on a plurality of myoelectric potential signals. The third step is to estimate the emotion of the subject from the plurality of emotion indexes by using the information showing the relationship between the plurality of emotion indexes and the human emotion.

The emotion estimation system according to the second aspect of the present disclosure includes a plurality of myoelectric potential sensors and an arithmetic unit. The plurality of myoelectric potential sensors are arranged so as to correspond to different types of muscles among the facial muscles of the subject, and output the myoelectric potential signals of the corresponding muscles. The arithmetic unit is configured to estimate the emotion of the subject based on a plurality of myoelectric potential signals from a plurality of myoelectric potential sensors. The arithmetic unit calculates a plurality of emotion indexes based on a plurality of myoelectric potential signals, and uses information indicating the relationship between the plurality of emotion indexes and human emotions to obtain the emotions of the subject from the plurality of emotion indexes. presume.

According to this disclosure, complex emotions can be estimated.

It is a figure which shows schematic the whole structure of the emotion estimation system which concerns on embodiment of this disclosure. It is a figure for demonstrating the mounting part of a plurality of sensors in an embodiment. It is a conceptual diagram for demonstrating the emotion estimation method in the comparative example. It is a figure which shows an example of the myoelectric potential signal detected by each of the 1st myoelectric potential sensor and the 2nd myoelectric potential sensor. It is a conceptual diagram for demonstrating the problem of emotional value in a comparative example. It is a conceptual diagram for demonstrating the emotion estimation method in this embodiment. It is a figure for demonstrating the relationship between two emotion indexes. It is a conceptual diagram of a map for estimating the emotion of a subject from an emotion index. It is a flowchart which shows the emotion estimation method which concerns on embodiment. It is a figure which shows roughly the whole structure of the emotion estimation system which concerns on a modification. It is a figure for demonstrating the attachment part of the arousal degree sensor in the modification. It is a conceptual diagram (FIG. 1) of a plurality of maps in a modification. It is a conceptual diagram (FIG. 2) of a plurality of maps in a modification. It is a conceptual diagram (FIG. 3) of a plurality of maps in a modification. It is a flowchart which shows the emotion estimation method which concerns on a modification.

Hereinafter, the present embodiment will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals, and the description thereof will not be repeated.

[Embodiment]
<System configuration>
FIG. 1 is a diagram schematically showing an overall configuration of an emotion estimation system according to an embodiment of the present disclosure. With reference to FIG. 1, the emotion estimation system 100 acquires a myoelectric potential signal from the subject (subject) and estimates the emotion of the subject based on the muscle activity represented by the acquired myoelectric potential signal. The emotion estimation system 100 includes a wearable terminal 10 worn on the subject and a fixed terminal 90 installed in the environment around the subject. The wearable terminal 10 and the fixed terminal 90 are configured to enable bidirectional communication.

The wearable terminal 10 includes a plurality of myoelectric potential sensors 1, a signal processing circuit 2, a controller 3, a communication module 4, and a battery 5. The signal processing circuit 2, the controller 3, the communication module 4, and the battery 5 are housed inside a dedicated housing 6.

Each of the plurality of myoelectric potential sensors 1 is attached to the face of the subject and detects the myoelectric potential signal at the attachment site. The myoelectric potential signal means a weak electric signal generated when moving a muscle. In the present disclosure, the "face" of a subject is not limited to the face (front or side of the face), and the "face" may include the neck of the subject. For example, a myoelectric potential sensor may be attached to the throat of the subject to detect changes in the myoelectric potential accompanying the swallowing motion of the subject. The plurality of myoelectric potential sensors 1 are attached to different types of muscles. The type of muscle can be identified by site. When specifying the type of muscle, it is not always necessary to limit the composition or structure of the muscle, and different parts can be treated as different types of muscle. In the present embodiment, the plurality of myoelectric potential sensors 1 includes a first myoelectric potential sensor 11 and a second myoelectric potential sensor 12.

FIG. 2 is a diagram for explaining a mounting portion of the first myoelectric potential sensor 11 and the second myoelectric potential sensor 12 in the present embodiment. With reference to FIG. 2, the first myoelectric potential sensor 11 is attached to the eyebrows of the subject. More specifically, the first myoelectric potential sensor 11 includes a working electrode 111 and a reference electrode 112. The working electrode 111 and the reference electrode 112 are mounted directly above the corrugator supercilii. The mounting sites of the working electrode 111 and the reference electrode 112 may be slightly displaced from directly above the corrugator supercilii as long as they are in the vicinity of the corrugator supercilii. The first myoelectric potential sensor 11 detects the potential of the working electrode 111 with reference to the potential of the reference electrode 112 as the myoelectric potential of the corrugator supercilii muscle. The first myoelectric potential sensor 11 outputs a myoelectric potential signal indicating the activity of the corrugator supercilii muscle to the signal processing circuit 2.

The second myoelectric potential sensor 12 is attached to the subject's cheek. More specifically, the second myoelectric potential sensor 12 includes a working electrode 121 and a reference electrode 122. The working electrode 121 and the reference electrode 122 are mounted directly above the zygomaticus major muscle. However, the mounting sites of the working electrode 121 and the reference electrode 122 may be slightly displaced from directly above the zygomaticus major muscle as long as they are in the vicinity of the zygomaticus major muscle. The second myoelectric potential sensor 12 detects the potential of the working electrode 121 based on the potential of the reference electrode 122 as the myoelectric potential of the zygomaticus major muscle. The second myoelectric potential sensor 12 outputs a myoelectric potential signal indicating the activity of the zygomaticus major muscle to the signal processing circuit 2.

With reference to FIG. 1 again, the signal processing circuit 2 includes a filter, an amplifier, an A / D converter, and the like, although none of them are shown. The signal processing circuit 2 performs predetermined signal processing (noise removal, rectification, amplification, digitization, etc.) on each of the myoelectric potential signals acquired by the plurality of myoelectric potential sensors 1, and processes each signal after the processing to the controller 3 Output to. In the following, the myoelectric potential signal from the first myoelectric potential sensor 11 processed by the signal processing circuit 2 will be referred to as “myoelectric potential signal MS1”, and the myoelectric potential signal from the second myoelectric potential sensor 12 will be referred to as the signal processing circuit 2. The one processed by the above is described as "myoelectric potential signal MS2".

The controller 3 is an arithmetic unit including a processor 31 such as a CPU (Central Processing Unit), a memory 32 such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and an input / output port 33. The controller 3 executes arithmetic processing for estimating the emotion of the subject based on the myoelectric potential signals MS1 and MS2. This arithmetic processing will be described later. Further, the controller 3 is configured to be able to exchange information with the outside (fixed terminal 90, etc.) by controlling the communication module 4.

The communication module 4 is a communication device compliant with the short-range wireless communication standard. In response to the control by the controller 3, the communication module 4 transmits a signal indicating the calculation result of the controller 3 (a signal indicating the estimation result of the emotion of the target person) or the like to the fixed terminal 90.

Battery 5 is a secondary battery such as a lithium ion secondary battery. The battery 5 supplies an operating voltage to each device in the wearable terminal 10.

The fixed terminal 90 is, for example, a PC (Personal Computer) or a server. The fixed terminal 90 communicates with the wearable terminal 10 via a communication module (not shown), and receives a signal indicating the calculation result of the controller 3. The fixed terminal 90 includes an arithmetic unit 91 and a display device 92.

Like the controller 3, the arithmetic unit 91 includes a processor, a memory, and an input / output port (none of which is shown), and is configured to be capable of executing various arithmetic processes. The display device 92 is, for example, a liquid crystal display, and displays the calculation result of the controller 3 received from the wearable terminal 10.

Note that the hardware configuration of the emotion estimation system 100 shown in FIG. 1 is merely an example, and is not limited to this. For example, it is not essential that the signal processing circuit 2 and the controller 3 can be attached to the target person, and the signal processing circuit 2 and the controller 3 may be provided on the fixed terminal 90. On the other hand, it is not essential that the fixed terminal 90 is a stationary type, and a mobile terminal such as a smart phone may be adopted instead of the fixed terminal 90. Further, the wearable terminal 10 may be provided with a small monitor for displaying the estimation result of the emotion of the target person.

<Expression of emotions>
It is generally believed that unpleasant feelings, such as when unpleasant or worried, are manifested in corrugator supercilii activity. On the other hand, feelings of pleasure such as when happy or relieved are thought to appear in the activity of the zygomaticus major muscle. By monitoring the activity of these muscles, it is possible to estimate the emotions of the subject as in the comparative example described below.

FIG. 3 is a conceptual diagram for explaining the emotion estimation method in the comparative example. With reference to FIG. 3, in the comparative example, the “emotion value Z”, which is an index indicating comfort / discomfort, is calculated (see Non-Patent Document 2). The amount of activity of the corrugator supercilii muscle detected by the first myoelectric potential sensor 11 is represented by x, and the amount of activity of the zygomaticus major muscle detected by the second myoelectric potential sensor 12 is represented by y. Then, as shown in the following equation (1), each of the activity amounts x and y is multiplied by an appropriate coefficient, and the activity amounts after the multiplication are added together. The two coefficients (-0.25, 0.27) shown in the equation (1) are only examples.

The emotion value Z represents a pleasant feeling when it is positive, and an unpleasant feeling when it is negative. As can be understood from the equation (1), the emotion value Z is an index capable of expressing the pleasant / unpleasant feelings in a one-dimensional manner, and is therefore useful in treating the pleasant / unpleasant feelings in a unified manner. ..

On the other hand, human beings have more complicated emotions that cannot be classified only by comfort / discomfort. In the following, this emotion will be described by taking myoelectric potential signals MS1 and MS2 acquired from a subject during exercise (in this example, playing table tennis) as an example.

FIG. 4 is a diagram showing an example of a myoelectric potential signal detected by each of the first myoelectric potential sensor 11 and the second myoelectric potential sensor 12. The horizontal axis represents the elapsed time. The vertical axis represents the voltage of the myoelectric potential signal MS1 of the wrinkled eyebrows muscle after the signal processing by the signal processing circuit 2 and the voltage of the myoelectric potential signal MS2 of the zygomaticus major muscle after the signal processing by the signal processing circuit 2 in this order from the top. .. The greater the voltage fluctuation, the greater the muscle activity.

With reference to FIG. 4, first, the activity of the corrugator supercilii was detected in the time zone T1. This coincides with the timing when the target person made a mistake during play. No activity of the zygomaticus major muscle was detected during time zone T1.

In addition, activity of the zygomaticus major muscle was detected at time zone T2. This voltage change is due to the subject enjoying the conversation with the opponent. No corrugator supercilii activity was detected during time zone T2.

Furthermore, corrugator supercilii activity was detected at time zone T3. This is considered to be a signal change caused by the subject worried about why the play did not go well. Little activity of the zygomaticus major muscle was detected during time zone T3.

After that, in the time zone T4, the activity of the corrugator supercilii muscle and the activity of the zygomaticus major muscle were detected at the same time. When we interviewed the subjects about how they felt at this time, they answered, "I enjoyed playing and smiled, but I made a mistake during the play and felt frustrated." In this way, human beings may have complex emotions in which a plurality of emotions are mixed.

Also, as another example, when human beings are thinking or worried, they take the expression that the mouth is straight and sideways with wrinkles between the eyebrows, and the activity of the corrugator supercilii muscle is large. The activity of the zygomaticus muscle may be detected at the same time. Such emotions are also complex emotions that are different from mere pleasure / discomfort. However, as explained below, it is not possible to infer such complex emotions in the comparative example.

FIG. 5 is a conceptual diagram for explaining the problem of the emotion value Z in the comparative example. As described in the above equation (1) with reference to FIG. 5, the emotion value Z is an index that one-dimensionally expresses a pleasant / unpleasant feeling. Therefore, the negative term (-0.25x) representing the activity of the corrugator supercilii muscle and the positive term (0.27y) representing the activity of the zygomaticus major muscle can cancel each other out and become 0 (or a value close to 0). There is sex. Then, even though a plurality of emotions actually exist at the same time, it may be presumed that "I do not feel anything in particular" (when Z≈0). Alternatively, it can be presumed to be "slightly pleasant" (Z is positive but close to 0) or "slightly unpleasant" (Z is negative but close to 0). There is also sex.

In emotional value Z, pleasure and discomfort are arranged on both poles of a single axis. There is an implicit premise that this is the opposite of pleasure and discomfort, and that humans do not experience pleasure and discomfort at the same time. Therefore, in the comparative example, it is not possible to estimate a complex emotion in which a plurality of emotions are mixed.

FIG. 6 is a conceptual diagram for explaining the emotion estimation method in the present embodiment. With reference to FIG. 6, in the following, the amount of activity of the corrugator supercilii muscle detected by the first myoelectric potential sensor 11 is expressed as x1, and the amount of activity of the zygomaticus major muscle detected by the second myoelectric potential sensor 12 is expressed as x2. (X1 ≧ 0, x2 ≧ 0). In this embodiment, two "emotion indexes E1 and E2" are used instead of the emotion value Z. The emotion indexes E1 and E2 are _{expressed by the following determinant using four coefficients k 11} to k ₂₂ (see equation (2)).

Each of the coefficients k ₁₁ to k ₂₂ is a predetermined positive constant based on the results of prior psychological experiments. More specifically, various stimuli accompanied by emotional information are given to a certain number of subjects, and myoelectric potential signals MS1 and MS2 are acquired as responses to the stimuli. As another example, an emotion acquisition device other than the first myoelectric potential sensor 11 and the second myoelectric potential sensor 12 while giving various stimuli to a certain number of subjects and acquiring myoelectric potential signals MS1 and MS2 as responses to them. Emotional information may be acquired using (camera, heart rate sensor, sweat sensor, brain wave sensor, etc.). _{Then, the respective coefficients k 11} to k ₂₂ can be determined by obtaining the correspondence between the activity amount x1 of the corrugator supercilii muscle, the activity amount x2 of the zygomaticus major muscle, and emotions by using a method such as multivariate analysis.

If the formula (2) is written down, the following formula (3) is derived. From the formula (2) or the formula (3), it is understood that the emotion index E1 is calculated based on both the activity amount of the corrugator supercilii muscle x1 and the activity amount of the zygomaticus major muscle x2, and the emotion index E2 is also the corrugator supercilii muscle. It is understood that the calculation is based on both the muscle activity x1 and the zygomaticus major muscle activity x2.

In this way, in the calculation of the emotion indexes E1 and ES, the product of the muscle activity amount x corresponding to the myoelectric potential signal and the coefficient k is calculated for each myoelectric potential signal MS1 and MS2, and the sum of the products is calculated. It may be taken (see equation (3)). Alternatively, for each of the myoelectric potential signals MS1 and MS2, for each of the plurality of myoelectric potential signals, the product is calculated by multiplying the amount of muscle activity x corresponding to the myoelectric potential signal by a predetermined coefficient in a matrix. The sum of the products may be taken (see equation (2)).

FIG. 7 is a diagram for explaining the relationship between the emotion index E1 and the emotion index E2. With reference to FIG. 7, in the present embodiment, the emotion index E1 is an index showing the strength of positive emotions. Although the emotion index E1 mainly reflects the effect of the activity amount x2 of the zygomaticus major muscle, the activity amount x1 of the corrugator supercilii muscle may also have an effect. On the other hand, the emotion index E2 is an index showing the strength of negative emotions. Although the emotion index E2 mainly reflects the effect of the activity amount x1 of the corrugator supercilii muscle, the activity amount x2 of the zygomaticus major muscle may also have an effect.

The emotion index E1 and the emotion index E2 are not in a mutually canceling relationship, unlike the emotion value Z. Further, the emotion index E1 and the emotion index E2 are not in a trade-off relationship that when one becomes large, the other inevitably becomes small. In this sense, the emotion index E1 and the emotion index E2 are independent indexes.

In the present embodiment, first, the emotion indexes E1 and E2 are calculated from the myoelectric potential signals MS1 and MS2, and the emotions of the subject are estimated based on the emotion indexes E1 and E2. A map as described below is used for emotion estimation from the emotion indexes E1 and E2.

FIG. 8 is a conceptual diagram of a map for estimating the emotion of the subject from the emotion indexes E1 and E2. With reference to FIG. 8, in this map MP, the corresponding emotions of human beings are predetermined for each combination (E1, E2) of the emotion index E1 and the emotion index E2 based on the result of the preliminary experiment. When a complex emotion in which two emotions are mixed occurs in the subject, the combination of the two emotion indexes (E1 and E2) is located in the region Q. Therefore, in the controller 3, when the combination of emotion indexes (E1, E2) calculated from the myoelectric potential signals MS1 and MS2 is located in the region Q, a complicated emotion in which a plurality of emotions are mixed is generated in the subject. It can be estimated that there is. The map MP corresponds to an example of "information" according to the present disclosure, but a table may be used instead of the map.

<Processing flow>
FIG. 9 is a flowchart showing an emotion estimation method according to the present embodiment. In the flowcharts shown in FIG. 9 and FIG. 13 to be described later, the process executed by the wearable terminal 10 is shown on the left side, and the process executed by the fixed terminal 90 is shown on the right side. These processes are called from the main routine at a predetermined calculation cycle, and are repeatedly executed by the controller 3 of the wearable terminal 10 or the arithmetic unit 91 of the fixed terminal 90. Each step is realized by software processing by the controller 3 or the arithmetic unit 91, but may be realized by hardware (electric circuit) manufactured in the controller 3 or the arithmetic unit 91. In the figure, the step is described as "S".

In step 11, the controller 3 acquires the myoelectric potential signal MS1 indicating the activity of the corrugator supercilii muscle from the first myoelectric potential sensor 11. Further, in step 12, the controller 3 acquires the myoelectric potential signal MS2 indicating the activity of the zygomaticus major muscle from the second myoelectric potential sensor 12. It is preferable that these myoelectric potential signals MS1 and MS2 are acquired at the same time (or with a sufficiently short time difference).

In step 13, the controller 3 calculates two emotion indexes E1 and E2 from the two myoelectric potential signals MS1 and MS2 acquired in

steps

11 and 12 according to the above equation (2) or (3).

In step 14, the controller 3 estimates the user's emotions corresponding to the emotion indexes E1 and E2 calculated in step 13 by referring to the map MP shown in FIG.

The controller 3 controls the communication module 4 so as to transmit the data indicating the myoelectric potential signals MS1 and MS2 acquired in

steps

11 and 12 to the fixed terminal 90. Further, the controller 3 causes the fixed terminal 90 to transmit the data showing the calculation results (step 13) of the emotion indexes E1 and E2 and the data showing the estimation result (step 14) of the user's emotions. When the arithmetic unit 91 of the fixed terminal 90 receives various data from the wearable terminal 10, the arithmetic unit 91 controls the display device 92 so as to display the received data (step 19). By repeatedly executing the processes of steps 11 to 19, the emotions of the subject can be displayed on the display device 92 over time.

Note that FIG. 9 has described an example in which the controller 3 immediately calculates the emotion indexes E1 and E2 from the myoelectric potential signals MS1 and MS2. However, the controller 3 may execute batch processing instead of real-time processing. That is, the controller 3 stores the data of the myoelectric potential signals MS1 and MS2 in the memory 32 in time series, and later calculates the emotion indexes E1 and E2 (for example, at the timing when the operation for starting the emotion estimation is accepted). You may. Further, the controller 3 may transmit the data of the two myoelectric potential signals MS1 and MS2 to the fixed terminal 90, and calculate the emotion indexes E1 and E2 by the arithmetic unit 91 of the fixed terminal 90.

As described above, in the present embodiment, the emotion indexes E1 and E2 are adopted instead of the emotion value Z as the index for the emotion of the subject from the two myoelectric potential signals MS1 and MS2. The emotion indexes E1 and E2 in the present embodiment are calculated from the determinant equation (2) or the newly written equation (3), but in these equations, the amount of activity of each muscle is only one emotion. It is expressed that the amount of activity of each muscle can contribute to multiple emotions to varying degrees. Further, the emotion value Z is an index in which the contribution of the myoelectric potential signal MS1 and the contribution of the myoelectric potential signal MS2 can cancel each other, whereas the emotion indexes E1 and E2 are the contribution of the myoelectric potential signal MS1 and the myoelectric potential signal MS2. It is an index independent of each other in that the contributions of the above do not cancel each other out. Therefore, by adopting the emotion indexes E1 and E2, it becomes possible to estimate a complicated emotion in which a plurality of emotions are mixed.

In this embodiment, an example in which two myoelectric potential sensors are attached to the face of a subject has been described. However, the number of myoelectric potential sensors attached is not limited to two, and three or more myoelectric potential sensors may be attached to the face of the subject. For example, when calculating n emotion indexes using n (n is a natural number of 3 or more) myoelectric potential sensors, muscle activity x and emotions are calculated according to the following equation (4) using a square matrix. The relationship with the index E can be defined.

Also, the number of muscle activity x and the number of emotion index E may be different. In that case, the matrix with the coefficient k in Eq. (4) may be an non-square matrix.

[Modification example]
The emotions that occur in the subject can also be influenced by the subject's alertness. Alertness is an index that indicates the degree of physical or cognitive arousal caused by emotions, and takes a value between excitability (high arousal) and calmness (low arousal). In this modified example, a configuration for estimating the emotion of the subject in more detail by combining the emotion indexes E1 and E2 and the alertness will be described.

FIG. 10 is a diagram schematically showing the overall configuration of the emotion estimation system according to the modified example. With reference to FIG. 10, the emotion estimation system 200 further includes an arousal sensor 7 in addition to the plurality of myoelectric potential sensors 1 (in this example, the first myoelectric potential sensor 11 and the second myoelectric potential sensor 12). It is different from the emotion estimation system 100 (see FIG. 1) according to the embodiment. Since the other configurations of the emotion estimation system 200 are similar to the corresponding configurations of the emotion estimation system 100, the description will not be repeated.

FIG. 11 is a diagram for explaining a mounting portion of the alertness sensor 7 in the modified example. With reference to FIG. 11, the alertness sensor 7 is attached, for example, to the forehead of the subject. The alertness sensor 7 outputs a signal for monitoring skin electrical activity on the forehead to the signal processing circuit 2. Skin electrical activity can include various biological activities such as skin impedance (or skin conductance, which is the reciprocal of skin impedance), or potential activity of the skin. The information obtained by the electrical activity of the skin is an example of "biological information" according to the present disclosure.

In the following, an example of acquiring skin conductance among skin electrical activities will be described. The signal processed by the signal processing circuit 2 is referred to as "skin conductance signal RS". The controller 3 quantifies the arousal level A of the subject based on the skin conductance signal RS. More specifically, the skin conductance signal RS has a skin conductance level (SCL: Skin Conductance Level) that represents a relatively long-term level fluctuation and a skin conductance reaction (SCR: Skin) that represents a transient fluctuation on the order of several seconds. Conductance Response) is included. The controller 3 calculates the arousal level A of the subject based on the change in SCL. The attachment site of the arousal sensor 7 is not limited to the forehead, and may be, for example, the temple of the subject or the palm of the subject. In the present embodiment, a plurality of two-dimensional maps as described with reference to FIG. 8 are prepared for each arousal level A of the subject.

12A to 12C are conceptual diagrams of a plurality of maps in the modified example. With reference to FIGS. 12A to 12C, in the modified example, the arousal level A is divided into three categories of high arousal, medium arousal, and low arousal, and a total of three two-dimensional maps MP1 to MP3 are provided so as to correspond to each category. Created in advance. When the subject's arousal level A belongs to a high category, the map MP1 corresponding to the high arousal level is referred to. When the subject's arousal level A belongs to the medium category, the map MP2 corresponding to the medium arousal is referred to. When the subject's alertness A belongs to a low category, the map MP3 corresponding to the low alertness is referred to. It should be noted that the arousal level A is divided into three categories only as an example, and the number of categories may be two or four or more.

By preparing a plurality of maps according to the arousal level A in this way, if the arousal level A is different even if the combinations of emotion indexes (E1 and E2) are the same, the controller 3 causes the target person to express different emotions. It can be judged that it is experienced. As an example, it is possible to distinguish between the feeling of "disappointing but fun" and the feeling of the subject making and smiling.

FIG. 13 is a flowchart showing an emotion estimation method according to a modified example. With reference to FIG. 13, this flowchart further includes the process of steps 24 to 26, and includes the process of step 17 instead of the process of step 14 (see FIG. 9). Different from. The processes of steps 21 to 23 are the same as the processes of steps 11 to 13 in the embodiment, respectively.

In step 24, the controller 3 acquires the skin conductance signal RS from the alertness sensor 7. The acquired skin conductance signal RS is stored in the memory 32 in the controller 3 in time series.

In step 25, the controller 3 calculates the arousal level A of the subject by analyzing the skin conductance signal RS acquired in step 24 and the past skin conductance signal RS stored in the memory 32.

In step 26, the controller 3 selects a map according to the arousal level A calculated in step 25 from a plurality of maps MP1 to MP3 (see FIGS. 12A to 12C) prepared in advance.

In step 27, the controller 3 refers to the map selected in step 26 and estimates the emotion corresponding to the combination of emotion indexes (E1, E2) calculated in step 23.

Although it was explained that a plurality of two-dimensional maps are prepared in this modification, a three-dimensional map may be prepared instead. This three-dimensional map has an emotion index E1, an emotion index E2, and an arousal level A as the first to third axes. By referring to such a three-dimensional map, it is possible to estimate the emotion of the subject according to the combination of the emotion indexes E1 and E2 and the arousal degree A.

As described above, according to this modification, it is possible to estimate a complex emotion in which a plurality of emotions are mixed, as in the embodiment. Further, according to this modification, by further introducing the arousal degree A, even if the emotion indexes E1 and E2 are the same, if the arousal degree A is different, it is distinguished from different emotions, so that the emotions of the subject can be described in more detail. It becomes possible to estimate.

<Aspect>
It will be understood by those skilled in the art that the plurality of exemplary embodiments described above are specific examples of the following embodiments.

(Section 1)
The emotion estimation method according to the first aspect is
Steps to acquire multiple myoelectric potential signals corresponding to different types of muscles in the subject's face, and
A step of calculating a plurality of emotional indices based on the plurality of myoelectric potential signals, and
It may include a step of estimating the emotion of the subject from the plurality of emotion indexes by using the information showing the relationship between the plurality of emotion indexes and human emotions.

According to the emotion estimation method described in paragraph 1, complex emotions can be estimated by using information indicating the relationship between a plurality of emotion indexes and human emotions.

(Section 2)
In the emotion estimation method described in paragraph 1, the calculation step obtains the amount of muscle activity corresponding to each of the plurality of myoelectric potential signals, and calculates the plurality of emotion indexes based on the obtained amount of activity. May include steps.

According to the emotion estimation method described in paragraph 2, the emotion index can be calculated with high accuracy from the amount of muscle activity.

(Section 3)
In the emotion estimation method described in paragraph 2, the calculation step is
For each of the plurality of myoelectric potential signals, a step of calculating the product of the amount of muscle activity corresponding to the myoelectric potential signal and a predetermined coefficient, and
It may include the step of calculating the plurality of emotional indices by taking the sum of the products for all of the plurality of myoelectric potential signals.

(Section 4)
In the emotion estimation method described in paragraph 2, the calculation step is
A step of calculating the product by matrixally multiplying the amount of muscle activity corresponding to the myoelectric potential signal and a predetermined coefficient for each of the plurality of myoelectric potential signals.
It may include the step of calculating the plurality of emotional indices by taking the sum of the products for all of the plurality of myoelectric potential signals.

According to the emotion estimation method described in the third or fourth paragraph, the emotion index can be calculated with higher accuracy from the amount of muscle activity.

(Section 5)
In the emotion estimation method according to the first to fourth paragraphs, the estimation step is performed by referring to a predetermined map between the plurality of emotion indexes and the emotion of the subject. It may include the step of estimating the emotion of the subject from the emotion index.

According to the emotion estimation method described in Section 5, the emotions of the subject can be estimated with high accuracy from the plurality of emotion indexes by using a map in which the relationship between the emotions of the subject and the emotions of the subject is predetermined. ..

(Section 6)
In the emotion estimation method according to the first to fifth paragraphs, the step to be acquired is
The step of acquiring the myoelectric potential signal of the corrugator supercilii of the subject, and
It may include the step of acquiring the myoelectric potential signal of the zygomaticus major muscle of the subject.

According to the emotion estimation method described in item 6, a plurality of myoelectric potential signals can be easily acquired.
(Section 7)
In the emotion estimation method described in paragraph 6, the plurality of emotion indexes include a first emotion index that indexes positive emotions and a second emotion index that indexes negative emotions.
The calculation step may include a step of calculating the first emotion index and the second emotion index based on the activity amount of the corrugator supercilii muscle of the subject and the activity amount of the zygomaticus major muscle of the subject. ..

According to the emotion estimation method described in paragraph 7, the first emotion index and the second emotion index can be calculated with high accuracy.

(Section 8)
The emotion estimation method according to the first to seventh paragraphs further includes a step of calculating the arousal level of the subject based on the biological information of the subject.
In the estimation step, the emotion of the subject is estimated from the plurality of emotion indexes and the alertness by referring to the correspondence between the plurality of emotion indexes, the alertness, and the emotion of the subject. May include steps.

According to the emotion estimation method described in paragraph 8, it can be determined that the subject is experiencing different emotions when the arousal level is different even if the combination of emotion indexes is the same.

(Claim 9)
The emotion estimation method according to the first to eighth paragraphs may further include a step of displaying the emotion of the subject estimated by the estimation step on a display device over time.

According to the emotion estimation method described in Section 9, changes in the emotions of the subject over time can be easily observed.
(Section 10)
The emotion estimation system according to the first aspect is
Multiple myoelectric potential sensors that are arranged to correspond to different types of muscles in the subject's face and output myoelectric potential signals of the corresponding muscles,
It is provided with an arithmetic unit configured to estimate the emotion of the subject based on a plurality of myoelectric potential signals from the plurality of myoelectric potential sensors.
The arithmetic unit
A plurality of emotion indexes are calculated based on the plurality of myoelectric potential signals, and a plurality of emotion indexes are calculated.
The emotion of the subject may be estimated from the plurality of emotion indexes by using the information indicating the relationship between the plurality of emotion indexes and human emotions.

According to the emotion estimation system described in paragraph 10, complicated emotions can be estimated in the same manner as the emotion estimation method described in paragraph 1.

The embodiments disclosed this time should be considered to be exemplary in all respects and not restrictive. The scope of the present disclosure is indicated by the scope of claims rather than the description of the embodiment described above, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Multiple myoelectric potential sensors, 11 1st myoelectric potential sensor, 12 2nd myoelectric potential sensor, 111, 121 working electrode, 112, 122 reference electrode, 2 signal processing circuit, 3 controller, 31 CPU, 32 memory, 33 input / output Port, 4 communication module, 5 battery, 6 housing, 7 arousal sensor, 10 wearable terminal, 90 fixed terminal, 91 arithmetic unit, 92 display device, 100,200 emotion estimation system.

Claims

Steps to acquire multiple myoelectric potential signals corresponding to different types of muscles in the subject's face, and
A step of calculating a plurality of emotional indices based on the plurality of myoelectric potential signals, and
An emotion estimation method including a step of estimating the emotion of the subject from the plurality of emotion indexes using information indicating a relationship between the plurality of emotion indexes and human emotions.
The emotion according to claim 1, wherein the calculation step includes a step of obtaining a muscle activity amount corresponding to each of the plurality of myoelectric potential signals and calculating the plurality of emotion indexes based on the obtained activity amount. Estimating method.
The calculation step is
For each of the plurality of myoelectric potential signals, a step of calculating the product of the amount of muscle activity corresponding to the myoelectric potential signal and a predetermined coefficient, and
The emotion estimation method according to claim 2, further comprising a step of calculating the plurality of emotion indexes by taking the sum of the products for all of the plurality of myoelectric potential signals.
The calculation step is
A step of calculating the product by matrixally multiplying the amount of muscle activity corresponding to the myoelectric potential signal and a predetermined coefficient for each of the plurality of myoelectric potential signals.
The emotion estimation method according to claim 2, further comprising a step of calculating the plurality of emotion indexes by taking the sum of the products for all of the plurality of myoelectric potential signals.
The estimation step includes a step of estimating the emotion of the subject from the plurality of emotion indexes by referring to a predetermined map between the plurality of emotion indexes and the emotion of the subject. The emotion estimation method according to claim 1.
The step to acquire is
The step of acquiring the myoelectric potential signal of the corrugator supercilii of the subject, and
The emotion estimation method according to claim 1, further comprising a step of acquiring a myoelectric potential signal of the zygomaticus major muscle of the subject.
The plurality of emotion indexes include a first emotion index that indicates a positive emotion and a second emotion index that indicates a negative emotion.
The calculation step includes a step of calculating the first emotion index and the second emotion index based on the activity amount of the corrugator supercilii muscle of the subject and the activity amount of the zygomaticus major muscle of the subject. The emotion estimation method according to claim 6.
Further including a step of calculating the arousal level of the subject based on the biological information of the subject.
In the estimation step, the emotion of the subject is estimated from the plurality of emotion indexes and the alertness by referring to the correspondence between the plurality of emotion indexes, the alertness, and the emotion of the subject. The emotion estimation method according to claim 1, which comprises a step.
The emotion estimation method according to claim 1, further comprising a step of displaying the emotion of the subject estimated by the estimation step on a display device over time.
Multiple myoelectric potential sensors that are arranged to correspond to different types of muscles in the subject's face and output myoelectric potential signals of the corresponding muscles,
It is provided with an arithmetic unit configured to estimate the emotion of the subject based on a plurality of myoelectric potential signals from the plurality of myoelectric potential sensors.
The arithmetic unit
A plurality of emotion indexes are calculated based on the plurality of myoelectric potential signals, and a plurality of emotion indexes are calculated.
An emotion estimation system that estimates the emotion of the subject from the plurality of emotion indexes using information indicating the relationship between the plurality of emotion indexes and human emotions.