Speech Coders for Wireless Communication

Digital representation of the speech waveform

x(t)

Sampler

x(n) = x(nt)

Quantizer

x(n)

Continuous-time Continuous-amp.

Discrete-time Continuous-amp.

Discrete-time Discrete-amp.

Courtesy: Communication Networks Research (CNR) Lab. EECS, KAIST

Three acoustic signals

Frequency range Telephone speech 300 3,400Hz* 50 7,000Hz 10 20,000Hz Sampling rate 8kHz 16kHz 48kHz PCM bits per sample 8 14 16 PCM bit rate 64kb/s 224kb/s 768kb/s

Wideband speech
Wideband audio

* Bandwidth in Europe : 200 3200Hz in the United States and Japan

Frequency response of Telephone transmission channel

Courtesy: Communication Networks Research (CNR) Lab. EECS, KAIST

Telephone 0 Hz

Speech 4kHz

Music (CD Quality)

7kHz 20kHz

Talk

A/D

Encoder Compress (record/store) Storage

Decoder Decompress (play)

D/A

Listen

Courtesy: Communication Networks Research (CNR) Lab. EECS, KAIST

Source coding techniques

Waveform coding

The reconstructed signal matches as close to the original signal as possible. Robust for wide range of speaker and noisy environment.
A parametric coding based on the quasistationary model of speech production.

Vocoder

Hybrid coding

The combined form of waveform coding and vocoder.

Courtesy: Communication Networks Research (CNR) Lab. EECS, KAIST

Hybrid coders

Multi-Pulse Excitation

Efficient at medium bit rates. A sequence of nonuniformly spaced pulses as an excitation signal Amplitudes and positions are excitation parameters Efficient at medium bit rates. A sequence of uniformly spaced pulses as an excitation signal The position of first pulse within a vector and amplitudes are excitation parameters Efficient at low bit rates (below 8 kbps) A code book of excitation sequences Two key issues; the design and search of a codebook

Regular-Pulse Excitation (RPE)

Code-Excited Linear Prediction (CELP)

Courtesy: Communication Networks Research (CNR) Lab. EECS, KAIST

a)

g1 n2 n1 g2 0 5 g1 g2

g3 n4 n3 g4 10 15

gk

nk

20 g6

b) Examples of excitations
a) multipulse b) regular-pulse c) Code-excited Linear Prediction

g4

K g3 0 5 Codebook 10 g5 15 20

c)

Codevector # 1 Codevector # 2 Codevector # 3

2M = N (M = Transmission Bit)

Codevector # N
Communication Networks Research (CNR) Lab. EECS, KAIST 7

Speech Compression Standards

64 kbps -law/A-law PCM(CCTT G.711) 64 kbps 7kHz Subband/ADPCM(CCITT G.722) 32 kbps ADPCM(CCITT G.721) 16 kbps Low Delay CELP(CCITT G.728) 13.2 kbps RPE-LTP(GSM 06.10) 13 kbps ACELP(GSM 06.60) 13 kbps QCELP(US CDMA Cellular) 8 kbps QCELP(US CDMA Cellular) 8 kbps VSELP(US TDMA Cellular) 8 kbps CS-ACELP(ITU G.729) 6.7 kbps VSELP(Japan Digital Cellular) 6.4 kbps IMBE(Immarsat Voice Coding Standard) 5.3 & 6.4 kbps True Speech Coder(ITU G.723) 4.8 kbps CELP(Fed. Standard 1016-STU-3) 2.4 kbps LPC(Fed. Standard 1015 LPC-10E)
8

Communication Networks Research (CNR) Lab. EECS, KAIST

Performance of speech codec

Speech Quality (SNR/SEGSNR, MOS, etc) Bit Rate (bits per second) Complexity (MIPS) Coding Delay (msec)

Communication Networks Research (CNR) Lab. EECS, KAIST

Requirements of speech codec for digital cellular

More channel capacity Noise immunity Encryption Reasonable complexity and encoding delay

Communication Networks Research (CNR) Lab. EECS, KAIST

10

Vocoders

Anatomy of Speech Organs :

The source of most speech occurs in the larynx. It contains two folds of tissue called the vocal folds or vocal cords which can open and shut like a pair of fans. The gap between the vocal cords is called the glottis and as air is forced through the glottis the vocal cords will start to vibrate and modulate the air flow. This process is known as phonation. The frequency of vibration determines the pitch of the voice for a male is typically in the range 50-200Hz for a female the range can be up to 500Hz.

Amplitude

Opening phase Closing phase

50 Time (ms)
Closure

Period = 12.5ms Fundamental frequency = 1/.0125 = 80Hz

Glottal Pulse
Rosenberg JASM 49, 1971

Intensity

Spectrum of glottal pulse

Frequency (Hz)
Harmonics of spectrum spaced at 80 Hz, corresponding to pitch period of 12.5ms.

Intensity

Spectrum of glottal pulse filtered by the vocal tract

Frequency (Hz)
Harmonics of spectrum spaced at 80 Hz, corresponding to pitch period of 12.5ms.

/ee/

/ar/

/uu/

Properties of Speech in Brief

Vowels
oo in blue o in spot ee in key e in again

Quasi-periodic Relatively high signal power Consonants

s in spot k in key

Non-periodic (random) Relatively low signal power

Wrong

/r/ /o/ /ng/

Moving

/m/ /uu/ /v/ /i/ /ng/

Southampton

/s/ /ou/ /th/ /aa/ /m/ /p/ /t/ /a/ /n/

Digital speech model

A basic digital model for speech production

periodic signal gen.

x
random signal gen. Gain

linear time variant filter

Vocoder

Send three kinds of information to the receiver:

(1) voiced or unvoiced signal, (2) if it is voiced, the period of the excitation signal, (3) the parameters of the prediction filter

Vocoder

voice classification pitch recognition determine filter coeff.

Encoder/Decoder

digital filter excitation signal gen

LPC Introduction

This speech coders are called Vocoders (voice coder). Basic Idea
Encode Parameters
Transmit Parameters

Estimate parameters

Decode Parameters

Synthetise Speech

They usually provide more bandwidth compression than is possible with waveform coding (24009600bps).

Generalities

LP Model Parameter Estimation Typical Memory requirements

LP Model
Pitch Period Impulse Generator
Voice/Unvoice Switch

Voice

All-pole filter
Gain

Speech Signal

White Noise Unvoice Generator

Glottal filter Vocal tract filter Lip Radiation filter

Parameter Estimation

Therefore, for each frame:

estimate LP coefficients (ais) estimate Gain estimate type of excitation (voice or unvoice). Estimate pitch.

V/UV Estimation

Several Methods Energy of Signal Zero Crossing Rate Autocorrelation Coefficient

Speech Measurements (1)

Zero Crossing Rate Log Energy Es

1 Es 10 log( N

S
n 1

( n))

Normalized Autocorrelation Coefficient

C1

s (n) s (n 1)
n 1

( s ( n))( s 2 ( n))
2 n 1 n 0

N 1

Comparison between actual data and V/U/S determination results.

V S U

Pitch Detection

Voiced sounds

Produced by forcing air through glottis Vocal cords oscillate and modulate air flow into quasi periodic pulses Pulses excite resonances in reminder of vocal tract Different sounds produced as muscles work to change shape of vocal tract Resonant frequencies or formant frequencies Fundamental frequency or pitch rate of pulses

Pitch Detection

400 200
amplitude

Short sections of

0 -200 -400

Voiced speech

100

200

300 400 sample number

500

600

700

400 200
amplitude

Unvoiced speech

0 -200 -400

100

200

300 400 sample number

500

600

700

Time-domain pitch estimation

Well studied area Variations of fundamental frequency are evident Time-domain speech processing should be capable of detecting pitch frequency

400 300 200 100

amplitude

-100 -200 -300 -400

100

200

300 400 sample number

500

600

700

Pitch Period Estimation Using the Autocorrelation Function

Periodic signals have periodic auto-correlation function

Rn (k )

N 1 k m 0

' ' [ x ( n m ) w ( m )][ x ( n m k ) w (k m)]

Basic problems in choosing window length: Speech changes over time (N low) but at least 2 periods of the waveform Approaches: Choose window to catch longest period Adaptive N Use modified short-time auto-correlation function

Pitch Period Estimation Using the Autocorrelation Function (Contd)

Auto-correlation representation - retains too much of the information in the speech signal => auto-correlation function has many peaks

14

12

10

100

200

300

400

500

600

700

Spectrum flatteners techniques

Remove the effects of the vocal tract transfer function Center clipping - nonlinear transformation, clipping value depends on maximum amplitude => Strong peak at the pitch frequency

400 300 200 100

amplitude

0.16 0.14 0.12 0.1

0.7

0.6

0.5

0.4
0 -100 -200 -300 -400

0.08

0.3
0.06 0.04 0.02
0 100 200 300 400 sample number 500 600 700

0.2

0.1

100

200

300

400

500

600

700

100

200

300

400

500

600

700

Fundamental Frequency

F0 estimation: (Hess) determining the main period in quasi-periodic waveform

usually using autocorrelation function and the average magnitude difference function (AMDF)
AMDFt (m) 1 Np

| s (n) s (n m) |,
t t n ,m

0 n m L 1

where L is the frame length Npis number of point pairs (peak in ACF and valley in AMDF indicates F0)

Typical Memory Requirements

Pitch coefficient (6 bits). Gain (5 bits) Model parameters:

LP coefficients (8-10 bits)

Small changes in the LPC results in large changes in the pole positions. If |rk| near 1, then large distortion. Represent a non-linear transformation of the Reflection Coefficients to expand the scale near to |rk| near 1.

Reflection coefficients (6 bits)

Log-Area Ratio:

The main difference of the LP vocoders is the calculation of the source of excitation.

LPC-10
Speech Signal ADC (8kHz) Sample Speech Window (180 samples) AMDF and Zero Crossing LP Analysis (Covariance Method)

Non-linear warping

E n Voice/Unvoice c Switch (1 bit) o LAR coefficients d (4 bits and 5 bits) e Reflection r Coefficients
Pitch Frequency (7 bit)
(4 bits)

Channel D e c o d e r

Pitch Period (7 bits)

Gain (5 bits) Impulse Generator White Noise Generator

Voice/Unvoice Switch(1 bit)

10 Reflection Coefficients. (5 bits for one and 4 bits for the others).

Synthesized Speech Signal

1/A(z)

RELP

Simple vocoder offers poor sound quality and is usually unsatisfactory.

An improvement is to use the prediction error rather than the periodical pulse (for voiced signal) or the random noise (for unvoiced signal) to excite the digital filter to reproduce the speech. The prediction error is also called the residual.
This scheme is called Residual Excited Linear Prediction (RELP) coding.

RELP

quantization

encoder

digital filter determine filter coeff.

RELP

RELP follows essentially the same idea as DPCM. However, in RELP the speech signal is divided into blocks (20ms/block).

The optimum linear predictor is designed for each block. For each block, the filter coefficients and the prediction error should be sent to the receiver.

In DPCM, the predictor can be fixed or adaptive. Only the prediction error is sent to the receiver.

Modeling of the prediction error

In each block of speech signal (a frame), the prediction error may also be correlated. To decorrelate the prediction error, each frame is further divided into 4 sub-frames (5ms). The prediction error u(n) is then modelled as
u( n) hu( n M ) e( n)

where M (40<=M<=120) is called the lag, and h is called the gain. They are determined by using the LMSE principle.

Long-term prediction

The decorrelation of the prediction error is called long-term prediction.

s(n) A(z) digital filter determine filter coeff. u(n)

U(z)
long-term prediction

e(n) encoder

RPE-LTP

The RPE-LTP has been adopted as the speech coding method in the GSM 06.10 standard
long-term prediction digital filter determine filter coeff.
Regular pulse selection and coding

RPE-LTP

Speech is sampled at 8 kHz, quantised to 8 bits/sample The speech signal is pre-processed to remove any DC component and to pre-emphasis the high-frequencies component, partly compensating for their low energy. The signal is then dived into frames (20ms, 160 samples). An eighth-order optimum linear predictor is designed using the Shur algorithm. The reflection coefficients (related to the filter coefficients) are nonlinearly mapped to another set of values called log-area ratio(LAR).

RPE-LTP

The 8 LAR parameters are quantized using 6,6,5,5,4,4,3,3 bits.

So a total of 36 bits for the LAR (or for the filter coefficients).

The frame is filtered using this filter and produces u(n). u(n) is then divided into 4 sub-frames (5ms each, 40 samples). Long-term prediction is performed for each sub-frame. The lag M is quantized to 7 bits and the gain h is represented by 2 bits. Long-term prediction produces e(n).

RPE-LTP

e(n) is down-sampled by a factor 3. For each sub-frame, there are 4 down-sample patterns. Need 2 bits to specify the pattern used. The down-sampled e(n) has 13 samples. The maximum of them is quantized to 6 bits, others are normalised then represented by 3 bits.

So in each sub-frame, e(n) is represented by 6+13*3=6+39 bits. A frame has 4 sub-frames, 4*(6+39)=180 bits

The above method is called regular-pulse-excitation (RPE)

13 kbps RPE-LTP coder - Encoder Short term LPC analysis

Reflection coefficients (36 bit / 20 ms)

RPE parameters (13 pulses / 5 ms)

Input signal

Preprocessing

Short term Analysis filter

+ -

RPE grid Selection and coding

Synthesis filter 1/A(z/5)

RPE grid decoding

LTP parameters LTP analysis

- Decoder Reflection coefficients

RPE parameters

RPE grid decoding

+
Synthesis filter 1/A(z/5)

Short term synthesis

Postprocessing

Output signal

LTP parameters

RPE-LTP

Summary

8 LAR coefficients For each sub-frame pattern code

36 (bits)

lag gain regular pulse total 4 sub-frames Total one frame bit rate

2 7 2 6+39 56 4*56=224 224+36=260 260 bits/20 ms=13kbs