JPH0358518B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention relates to a speech synthesis device, and an object of the present invention is to improve the quality of a synthesized speech signal. In general, the quality of speech signals (words, phrases, speech) synthesized by combining and editing phoneme fragments, that is, words, syllables, or even shorter speech segments, is determined by the processing of the connections between phoneme fragments, which are the constituent units of speech. It can be said that it depends on the situation. For example, a sudden change in the waveform that occurs at the connection, that is, a discontinuity in the waveform, causes harmonic noise, lowers the S/N ratio of the synthesized sound, and reduces the clarity. It is also known that fluctuations in the pitch frequency, which is the fundamental frequency of vocal fold vibration, degrade the naturalness of synthesized speech. Human hearing is extremely sensitive to changes in pitch frequency (the detection limit is
(said to be 0.1%). If the pitch frequencies of the combined phoneme segments are discontinuous, the synthesized speech will be difficult to hear and unnatural. The present invention makes it possible to obtain high-quality synthesized speech by recognizing phoneme waveform patterns and combining phoneme pieces in a natural manner. As the phoneme segment waveform, there are methods to use one cut out from natural speech, for example, for each pitch interval, or to use a phoneme segment waveform synthesized by another speech synthesizer, but the present invention uses a comparison method. for a short period of time,
Specifically, the purpose of this study is to clarify a method for combining phoneme segments of several milliseconds in length without discontinuities in the waveform at the connection point and without fluctuations in the pitch frequency. In other words, the waveforms of such short-duration phonemes should be similar at least at the joints of adjacent phonemes,
Therefore, by slightly modifying the time axis of each phoneme, the connecting parts can be smoothly connected. The present invention grasps the degree of waveform similarity in the form of a signal level for the connecting portions of phoneme segments to be combined, and makes appropriate temporal corrections to the time axes of the phoneme segments based on this. The detailed content of the present invention will be explained below using an audio time axis conversion device as a specific example. FIG. 1 is a block diagram illustrating a conventional time axis expansion device. In the figure, terminal 1 is an audio input terminal, 2 is an output terminal, and 3 and 4 are N-bit analog shift registers such as BBD.
5 is a low pass filter (LPF). 6,7,
8 and 9 are analog switches, and input terminal 1
From analog shift register 3 or 4, LPF
The switch controls the audio signal that reaches the output terminal 2 via the terminal 5. In addition, these analog switches are connected to the write clock circuit 1 of the analog shift registers 3 and 4.
Opening/closing is controlled as shown in the figure by the Q and output of a frequency dividing circuit 11 that divides 0 by 2 mN (m will be described later). The analog shift registers 3 and 4 are connected to OR gates 14 and 15 by the clock circuit 10 and the Q of the frequency divider circuit 11, and the output AND gates 12 and 13.
Q of the read clock circuit 16 and the frequency divider circuit 11 are alternately controlled by the 7-input clock through
Similarly, OR is performed by output AND gates 17 and 18.
It is alternately read clock controlled via gates 14 and 15. That is, for example, an audio signal in which the time axis applied to the input terminal is compressed by m times (m>1) (such a compressed signal can be obtained by, for example, increasing the playback speed of a tape recorder to m times the recording speed). When the Q output of the frequency divider circuit 11 is 1, the signal is sent to the analog shift register 4 via the analog switch 8.
written to. The number of bits of the shift register is N
Therefore, when the input audio signal completes sequential input as mN sampling strings, the rear end N of the mN sampling strings are stored in the shift register, and the Q output of the frequency dividing circuit 11 is inverted. becomes 0, and the switch 8 is closed. At the same time, the Q output of the frequency dividing circuit becomes 1, the switch 6 is opened, and data is written into the analog shift register 3 in the same way. As is clear from the structure shown, the analog shift register 4 is then clocked by a readout clock circuit 16 and read out via a switch 9 which is also controlled by the output. During the write period to the analog shift register 3, another analog shift register 4 performs reading in this way, and then when the Q and output of the frequency divider circuit 11 is inverted, the analog shift register 4 writes again, and 3
performs the read. Here write clock circuit 1
0 clock frequency as f ₁ , read clock circuit 1
If the clock frequency of 6 is f ₂ , then if each clock frequency is determined so that f ₁ /f ₂ = m (1), the time axis will be expanded by m times, and the compressed signal input to audio input terminal 1 will be expanded by m times. The audio appears at the output terminal 2 with the time axis restored. The readout clock frequency _f2 is naturally determined to satisfy Nyquist's sampling theorem for the required output audio frequency band. In the conventional device as described above, the connection timing of the phoneme pieces that are alternately output from the analog shift registers 3 and 4 is determined by the output of the frequency dividing circuit 11 that divides the write clock 10 by 2 mN every mN/f ₁ second. Since it is determined automatically, discontinuous waveform changes and pitch frequency fluctuations occur at the connection portions of phoneme pieces, as shown in FIG. As mentioned above, such discontinuities in waveform and pitch at the junctions of phoneme segments significantly degrade sound quality and clarity. Next, in order to improve the drawbacks of the conventional device, the applicant first filed Japanese Patent Application No. 56-94802 (June 1983).
We proposed a technology as described in (filed on the 18th). First of all,
The technical content of this prior application will be explained with reference to the block diagram of FIG. In the figure, 101 is an audio signal input terminal, 102 is an audio signal output terminal, and 103 is an audio signal input terminal.
is an analog-digital conversion circuit (hereinafter referred to as A/D) that converts an audio signal into digital data. 104 is a random access memory (hereinafter referred to as RAM) having a storage element of 2A bytes, and when the control input terminal (LT ₃ ) is at logic level "0", it is applied to the data input terminals I ₁ to Id (lower I ₁ ). The digital value is input to address input terminals A ₁ to Aa (lower
A ₁ ) is stored at the address given by A 1 ). When the control input terminal LT ₃ is at logic level "1", the contents of the address given by the address input terminals A ₁ -Aa are outputted to the data output terminals 0 ₁ -0d.
106 and 108 are clock generation circuits. The output fR of the clock generation circuit 106 is output from the OR gate 120.
The clock signal is supplied to the clock input terminal T of the read counter 107 via the read counter 107, and the output of the address control circuit 107 consisting of the read counter is incremented. The address control circuit 107 is an A-bit counter, and its initial value is set by the output of the arithmetic control circuit 105. Here, we will discuss how to set this initial value. First, the arithmetic control circuit 105 is a read counter 10.
The output of the read counter 107 is cleared by applying a pulse to the clear input terminal GL of No. 7. Next, the initial value of the read counter 107 is set by applying the number of pulses to be initialized from the SC (Set Counter) terminal of the arithmetic control circuit 105 to the input of the OR gate 120. Incidentally, the cycle for setting this initial value is the interval at which the output fR of the clock generation circuit 106 is counted for a predetermined value, and therefore, the output value of the read counter 107 at this time is the value initialized in the previous cycle. This is the value obtained by adding a predetermined number to
Read counter 107 via OR gate 120
It is sufficient to supply the clock to the clock input end row T of the clock. In this case, there is no need to clear the read counter.
Incidentally, the clock generation circuit 106 increments the read counter 107 by the arithmetic control circuit 105 described above.
This must be done when the output fR of is at logic level "0". If the above setting is to be performed even when the logic level of fR is "1", an AND gate 121 is placed at the input terminal from fR of the OR gate 120 as shown in FIG. This fR is supplied, the output terminal of the arithmetic control circuit 105 is connected to the other input terminal, and the output of the AND gate 121 is connected to the input terminal of the OR gate 120. If one of the inputs is prohibited, the initial value of the read counter 107 can be set regardless of whether the logic level of fR is "0" or "1". Further, the initial value setting of the read counter 107 by the arithmetic control circuit 105 is similarly performed by using the output fH of the clock generation circuit 123 as shown in FIG. In this case, fH is a clock with a sufficiently high frequency compared to fR, and this
One input terminal of the AND gate 122 and an input terminal of the arithmetic control circuit 105 are connected. When the arithmetic control circuit 105 performs initial value setting with the read counter 107, it applies a logic level "0" to the input of the AND gate 121, a logic level "1" to the input of the AND gate 122, and outputs the clock circuit 123. After counting a predetermined number, the read counter can be initialized by returning the input of AND gate 121 to logic level "1" and the logic level of AND gate 122 to "0". Furthermore, it is clear that the same effect can be achieved even if the read counter is constructed from a preset counter and the initial value is directly preset. After the initial value is set in this manner, the read counter divides fR. Note that the lower bit of the outputs Y ₁ to Ya of the read counter is Y ₁ . Now, the clock generation circuit 108 is the RAM 104.
Provides write clock timing. The output fw of the clock generating circuit 108 is supplied to the clock input terminal T of the A-bit frequency dividing circuit 109, and the outputs W ₁ to Wa (lower W ₁ ) of the frequency dividing circuit 109 are incremented sequentially. 110 is a switching circuit, and a control input
When LT ₁ is at logic level “1”, the frequency divider circuit 109
The outputs W ₁ to Wa of
output to address inputs A ₁ to Aa. 114,1
16 is an inverter, 115 is an AND gate,
117 is a NAND gate. R ₁ , R ₂ and R ₃ are resistors, and C ₁ , C ₂ and C ₃ are capacitors. R ₁ and C ₁ , R ₂ and C ₂ , and R ₃ and C ₃ each constitute an integrating circuit. Letting these time constants be τ ₁ , τ ₂ , and τ _{3 ,} respectively, they are all sufficiently smaller than the period of the write clock fw, so that τ ₁ >τ ₃ >τ ₂ holds. That is, as shown in Figure 6,
The output of the AND gate 115 (b in the figure) becomes logic level "1" at the rising edge of fw (a) in the figure, and falls when the capacitor _C1 is charged with the time constant _τ1 . The output of the NAND gate 117 (c in the same figure) is fw
It falls later than the rise of (a) in the same figure,
It rises before the output of the AND gate 115 falls. Reference numeral 111 denotes a latch circuit, which transmits the input to the output when the logic level of the control input terminal _LT2 is "0", and latches and outputs information at the time of rising when it is "1". 112 is digital
It is an analog conversion circuit (hereinafter referred to as D/A),
Convert digital values to analog values. A low-pass filter 113 removes sampling noise from the D/A converted audio signal. 130 is
It is a NAND gate, and the output of the AND gate 115 and the output of the arithmetic control circuit 105 are connected as inputs, and the output is connected as the LT2 input of the latch circuit 111. The arithmetic control circuit 105 keeps the logic level “0” while setting the initial value of the read counter 107.
Output to NAND gate 130. In a transient state where the initial value of the read counter is thereby set, the latch circuit 111 is configured not to transmit the input to the output. With this configuration, the audio signal applied to the input terminal is converted into a digital value by the A/D 103, and is sent to the RAM 104 at the cycle of the write clock fw.
is memorized. That is, when the output of the AND gate 115 is "1", the address input A ₁ of the RAM 104
~Aa is given the output of the frequency dividing circuit 109, the control input terminal _LT3 becomes "0", and the output of the A/D 103 is stored. Since the frequency dividing circuit 109 advances with a period of fw, the addresses of the RAM 104 where the audio signal is sampled and stored are continuous.
However, the address of ^2A is 0. write clock
The audio signal sampled according to fw and stored in the RAM 104 as a digital value is read out according to the readout clock fR, and the D/A conversion 1
12, and the audio signal is reproduced as an analog signal. This write clock FW and read clock
The ratio of fR is the ratio of time axis conversion. The read counter is incremented with the period of the read clock fR, and therefore the address from which the contents of the RAM 104 are read is incremented with the period of fR. The reason why the latch circuit 111 is provided is to prevent the contents of an erroneous address from being read out when writing to the RAM 104. That is, reading from the RAM 104 is performed at all times except when writing. In the present invention, instead of the configuration described above, the write clock fw is constructed by (a) connecting the output of the clock generation circuit as the clock for starting A/D conversion of the A/D 103; Therefore, the A/D 103 is based on the clock.
Conversion is performed, and each time this conversion ends, a conversion end signal (END OF CONVERSION, abbreviated as EOC) is sent.
occurs. (b) Use this EOC in place of the write clock fw in Figure 3. It is clear that even with this configuration, it functions in the same way as described above. In addition, according to this configuration, since the writing to the RAM is performed after the A/D conversion of the audio signal is completed, the data in the middle of A/D conversion is not written to the RAM, and the timing of the control clock is can be easily considered. Now, as explained in the conventional example shown in FIG. 1, the prior art shown in FIG. . The arithmetic control circuit 105 may be an arithmetic processing unit CPU (computer) programmed with a ROM. FIG. 7 shows the operation of the arithmetic control circuit 105. Each processing period is a period in which N read clocks are counted. Hereinafter, the direction of the time axis t will be described in units of the write clock FW. Of the N phoneme segment sample strings read out in [processing cycle 2], the last M sample strings are stored in [processing cycle 1] in accordance with the write clock fw. Take in (M+r) sample strings from the beginning of [processing cycle 2], and regarding this and the above M sample strings,
Calculate points (K) with high correlation. The calculation of this (K) will be described later. Since the correlation of the above-mentioned M sample strings is high after (K) samples have passed from the beginning of [processing cycle 2], at the beginning of [processing cycle 3], (K+r) from the beginning of [processing cycle 2] The output of the read counter 107 is initialized to the value of the output of the frequency divider circuit 109 at the time when the frequency exceeds the value. As a result, the sample string of the audio waveform read out at the connection point between [processing cycle 2] and [processing cycle 3] can be continuous. From the beginning of [processing cycle 2] (K+N)
The M sample strings from the time when the write clocks FW are counted are read out in [processing cycle 3] and are the trailing end M sample strings, which are used to calculate the connection points between the surroundings of the next process. , remember this. Thereafter, by performing this operation every processing cycle, the waveforms will be smoothly connected. Now, calculation of the value K of a connection point with a high degree of correlation will be described below. Figures 8a and b are M samples of the rear end of the preceding phoneme written in [processing cycle 1] and (M+r) samples of the front end of the succeeding phoneme at the tip of [processing cycle 2] in Figure 7, respectively. A sample is shown below. The sequence of samples at the rear end of this preceding phoneme is (Xp) (p=1, 2,...M), and the sequence of samples at the front end of the following phoneme is (Yp) (p=1, 2,...M+r)
shall be. This (Xp) and (Yp) are A/D103
It is obtained by sampling the output of with write clock fw. To calculate the similarity of this phoneme,
It is better to calculate the squared error (ek ² ) between (Xp) and (Yp). The squared error (ek ² ) is: ek ² = 1/M _M 〓 ^P=1 (Xp-X/σX-Yp+k-Y/σY) ² ...(2) However, = 1/M _M 〓 ^P=1 Xp, =1/M _M 〓 ^P=1 Yp,

【formula】

[Formula] It is expressed as K=0, 1, 2,..., r-1. This is the sampling waveform (Xp)
It represents the degree of similarity when (Yp) is shifted by K points and superimposed. However, the arithmetic processing based on equation (2) actually involves a huge number of calculation steps, and requires a high-performance computer to perform calculations in a short period of time (at least several tens of milliseconds). originally
Equation (2) examines the correlation between two waveforms with different amplitudes and levels, and therefore the standard deviation (σx),
Normalize the waveform with (σy) and further average level ()
The error is calculated by calculating the sum of squares of the difference between () and (). By the way, in the case of the speech synthesis apparatus shown in FIG. 3, the phoneme pieces handled have waveforms that are close in time, and therefore it can be considered that they are originally similar in amplitude and level. In this case, the difference between the two waveforms may be calculated by ek ² =1/M _M 〓 ^P=1 (Xp−Yp+k) ² (3) instead of equation (2). Moreover, in this case, it is sufficient to know the timing at which the similarity between the two waveforms is maximum, and therefore, equation (3) can be further replaced with the following equation (4). e _k = _M 〓 ^P=1 |Xp−Yp+k| (4) Here, only the most significant digit of the A/D converter may be used for (Xp) and (Yp+k). Alternatively, the polarity near the AC crossover point of the input signal may be used. In this case, (Xp) and (Yp+k) are both [1] or

It is [0]. That is, this is the integral of the absolute value of the difference between the corresponding sampling values, and the connection timing is determined by knowing k at which this is the minimum. In order to minimize the calculation processing time in the apparatus shown in FIG. 3, gk= _M 〓 ^P=1 (XpYp+k)...(5) may be calculated instead of equation (4). In equation (5), (Xp) and (Yp+k) are the most significant digit data of the A/D converter, and [1] or

It is [0]. The symbol is the symbol for exclusive OR, and therefore, (XpYp
+k) is the exclusive OR of (Xp) and (Yp+k),
That is, both (Xp) and (Yp+k) are [1], or

When [0]

[0] is given, and at other times [1] is given. Therefore, the similarity between the binary signal sampling data (Xp) at the rear end of the preceding phoneme and the binary signal sampling data (Yp) at the tip of the following phoneme is given by (gk), and this (gk) The connection timing is determined by knowing the value (K) that minimizes . That is, the arithmetic control circuit 105 calculates (gk) for k=0, 1, .
That is, as shown in FIG. 8, the least error is achieved when the M sample strings at the rear end of the preceding phoneme are superimposed from a portion shifted by k from the beginning of the high-speed phoneme. The waveform connection error in the first speech synthesizer described above is the sampling interval (the sampling period is τ) of the sample sequences (Xp) and (Yp). Therefore, in order to reduce this connection error, the sampling interval of the sample string can be shortened, but (M-τ) requires approximately one period of the pitch frequency of the input analog signal, and r・τ requires a time longer than one period of the pitch frequency of the input analog signal, and to put it simply, the sampling interval of the sample strings (Xp) and (Yp) is increased by that amount. The amount of data increases, and the time required for arithmetic processing increases. In other words, if arithmetic processing can be performed at high speed, waveform connection errors will be reduced accordingly, and high quality reproduced sound can be obtained. Therefore, in the present invention, a band filter is provided to limit the pass band of the input analog signal, and the input analog signal whose band is limited by the band filter is converted into a binary signal, and the binary signal is sampled to form a sample sequence. Configure to obtain (Xp) and (Xp). That is, in the present invention, the signal path (d ₁ ) (d _o ) connecting the A/D conversion circuit 103 and the arithmetic control circuit 105 in FIG. 3 is deleted, and the signal input terminal 1 is used instead.
01 and the input terminal 201 of FIG. 9 are connected, and the output terminal 202 of FIG. 9 is connected to the arithmetic control circuit 10 of FIG. 3. In FIG. 9, an input analog signal is applied to a terminal 201, and 203 is a bandpass filter having a characteristic of passing the fundamental frequency component of the input analog signal. 204 is an amplifier circuit, 205
is a hysteresis circuit. With the configuration shown in Fig. 9, a binary signal responsive to the fundamental frequency of the input analog signal is obtained, and this is sampled to create the sample sequence (Xp) of the rear end of the preceding phoneme and the front end of the succeeding phoneme. , (p=1,
2,...M) and (Yp), (p=1, 2,...M+τ)
get. The reason why the hysteresis circuit 205 is provided in FIG. 9 is to prevent the binary signal from responding to minute displacements of the band-limited input analog signal.
Regarding the sample sequences (Xp) and (Yp) obtained in this way, calculations are performed based on the above equation (4), but (Xp) and (Yp) are input analog signals that are close in time. It is obtained by converting the signal into a binary signal. Let the polarity reversal point of the sample sequence of (Xp) be lMth, and let the samples of (Yp) at the same polarity reversal point be l ₁ , l ₂ ,...th,
(XlM) and (Yl ₁ ), (XlM) and (Yl ₂ ), etc. Together with the polarity reversal points of (Xp) and (Yp), gli = _M 〓 ^P=1 (XpYli−lM+p−1) …( 6) Calculate equation (6), lM≦li≦τ+lM …(7) Find (li) that minimizes (gli) in equation (6) among (li) in the range of equation (7). Similarly to the above, the binary signal sampling data (Xp) at the rear end of the preceding phoneme and the 2-value signal sampling data (Xp) at the front end of the following phoneme are
The similarity of the sampling data (Yp) of the value signal is given by (gli), and minimize this (gli)
By knowing li, the connection timing is determined.
In other words, the smallest error is obtained when the M sample strings at the rear end of the preceding phoneme are superimposed from a portion shifted (li-lM) from the beginning of the front end of the succeeding phoneme. The calculation based on equation (5) had to be performed τ times, whereas the calculation based on equation (6) had to be performed the number of times the polarity of the band-limited input analog signal was reversed within the range that satisfied equation (7). The number of calculations can be reduced. In other words, the sampling interval between sample sequences (Xp) and (Yp) can be shortened,
Therefore, since the connection error can be reduced, the synthesized sound quality can be improved. As explained above, the arithmetic control circuit 105 receives the audio signal applied to the input terminal 101 from the A/D 103.
The digital values converted by the above are sampled by the write clock fw, which is the output of the clock generating circuit 108, to obtain the sample strings (Xp) and (Yp). The timing to take in this sample string (Xp) and (Yp) is determined by the frequency divider circuit 1.
It is indicated by the value of the output (W ₁ to Wa) of 09.
Further, the arithmetic control circuit 105 is connected to the clock generation circuit 10.
When N clocks are counted, the read counter 107 sets an initial value and enters the next processing cycle. The values to initialize this read counter are (Xp) and (Yp)
The value indicated by the frequency divider circuit when (Yp) is taken in is added to (k) obtained by the calculation. In the above description, the address for reading out the memory contents of the RAM 104 is given by the address control circuit 107 consisting of a read counter, and the waveform connection timing is given by initializing the read counter 107 to a predetermined value. However, it goes without saying that the same operation can be achieved by connecting an addition (or subtraction) circuit to the output of the read counter 107 and adding (or subtracting) a predetermined value to this addition (or arithmetic) circuit. As described above, the present invention provides a time base conversion circuit that can obtain smooth connection points by the operation of the arithmetic control circuit 105, and therefore eliminates discontinuities in connection waveforms and fluctuations in pitch frequency that occur in conventional devices. You can get a synthesized sound that is not possible.

[Brief explanation of drawings]

Fig. 1 is a block diagram of a conventional speech synthesis device, Fig. 2 is a drawing showing the characteristics of the conventional device, Fig. 3 is a block diagram showing the configuration of the speech synthesis device of the prior application of the present invention, and Fig. 4 5 and 5 are circuit diagrams showing configuration examples of main parts when initializing the read counter 107 shown in FIG. 3, and FIG. 6 explains the outputs of the gates 115 and 117 of the same device shown in FIG. 3. Drawing showing the time chart for
8 is a diagram showing a time chart for explaining the operation of the arithmetic control circuit 105 of the same device shown in FIG.
The figure is a waveform diagram of the sample strings (Xp) and (Yp) of M leading phoneme pieces and (M+r) following phoneme pieces, and FIG. 9 is a block circuit diagram of the main part of the speech synthesis apparatus of the present invention. 101,201...Signal input terminal, 102,2
02...Signal output terminal, 103...Analog-digital conversion circuit, 104...Random access memory, 105...Arithmetic control circuit, 106...Clock circuit that generates a read clock, 107...
...Address control circuit (read counter), 108
...Clock circuit that generates a write clock, 1
10...Switching circuit, 111...Latch circuit,
112...Digital...Analog conversion circuit, 11
3...Low pass filter, 203...Band filter, 204...Amplification circuit, 205...Hysteresis circuit.

Claims (1)

[Claims] 1. A speech synthesis device that performs editing and synthesis using phoneme segments extracted from an analog speech waveform, which includes: (a) analog-to-digital conversion means for converting an analog input signal into a digital signal; (b) ) digital storage means for storing the output of the conversion means in accordance with a first clock; (c) address control means for controlling an address from which the stored contents of the digital storage means are read; and (d) the basics of the analog input signal. a bandpass filter that passes frequency components and blocks harmonic components; (e) binary signal conversion means that converts an analog input signal band-limited by the bandpass filter into a binary signal; (f) a binary signal The vicinity of the rear end of the preceding phoneme and the vicinity of the front end of the succeeding phoneme of the binary signal converted from the analog input signal by the signal conversion means are converted into the first
Sampling is performed in response to a clock, and the similarity calculation is performed while making the sample strings correspond relatively to each other, and the correspondence relationship between the two sample strings at the point in time when the similarity is highest is calculated. (g) digital-to-analog conversion means for converting the digital signal read from the digital storage means into an analog signal and reproducing an analog audio signal; A speech synthesis device, characterized in that the address control means is stepped by a second clock and instructs an address from which to read out the stored contents of the digital storage means. 2. The binary signal conversion means includes a hysteresis circuit, and the hysteresis circuit converts the signal into a binary signal that does not respond to minute fluctuations in an analog input signal whose frequency band is limited by the bandpass filter. The speech synthesis device according to item 1.