1

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
1 INTRODUCTION
HD-Voice Implementation

Operators worldwide are rolling out high definition voice (HD-Voice) services, with 32 networks and
52 devices released at the time of writing. Along with high-speed data, video and applications,
HD-Voice is a major new enhancement to the subscriber experience, and a competitive differentiator
for service providers. In the U.K., ‘3’ and Orange offer HD-Voice today.

This service is based on an evolution from narrowband telephony (300Hz-3400Hz) to wideband

speech transmission (50Hz-7000Hz). The codecs for wideband are ready and approved by the
standardization bodies (G.722.2 AMR-WB based on AMR, EVRC-WB based on EVRC-B as well as
ITU-T G.729.1 which is the wideband extension of ITU-T G.729), the handsets are powerful enough
and the core networks are being upgraded to support new transmission loads.

HD-Voice Quality Measurement

Since 2000, PESQ (Perceptual Evaluation of Speech Quality, ITU-T P.862) has been the
standardized benchmarking algorithm of voice quality in lab and live-network test systems. POLQA
(Perceptual Objective Listening Quality Analysis, ITU-T P.863) is the long awaited technology update
to PESQ, covering the latest speech coding and network transport technology, with higher accuracy
2
for 3G, 4G/LTE and VoIP networks as well as for networks delivering HD-quality voice services.
These algorithms provide end-to-end testing of telecommunications networks using actual speech
samples, comparison of the reference signal (transmitting side) to the degraded signal (listening
side), and generation of mean opinion scores (MOS) to model subjective listening patterns. This
technique has been widely adopted due to its capability to automate collection of large sample sets
simulating real-world subscriber experience. POLQA improves on PESQ in several ways, including
improved perceptual accuracy and support for a broader range of codecs, two operational modes
to distinctly address narrowband and super-wideband communication, and a three-fold increase in
the validation set, confirming the higher accuracy of POLQA for voice quality measurement in both
traditional and HD-Voice services.

According to POLQA, the difference between a traditional mobile call (narrowband EFR/AMR
12.2kbps) and an HD-Voice mobile call (AMR-WB) is likely to be about 0.6 MOS. Metrico undertook
a study of the two U.K. operators’ traditional and HD-Voice offerings, to consider POLQA’s prediction.

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
2 TEST METHODOLOGY

2.1 TEST EQUIPMENT

Nomad

Using Metrico’s Nomad solution equipped with OPTICOM’s next-generation wideband POLQA
algorithm, Metrico engineers performed mobile-to-mobile test calls to assess the relative difference in
speech quality offered by HD-Voice services. The evaluation was performed twice, once on each of
the two networks supporting AMR-WB codecs used to deliver the service. The tests were repeated
in sequence with the devices switched between AMR-WB mode active and then switched to standard
narrowband mode representing the traditional mobile user audio experience.

Devices

In both cases, the devices used for testing were the popular Samsung Galaxy S II Android
smartphones. For all tests the same two Galaxy devices were used conducting a mobile-to-mobile
call on the same mobile operator’s network. SIM’s were rotated between the two devices to test the
two networks sequentially.
3
The devices were switched between AMR-WB (HD-Voice) and AMR-NB (standard narrowband
speech) support via the engineering mode screens on the handset.

The volume levels on both devices were set to approximately ¾ of maximum level. Additional
amplification/attenuation is provided within Nomad’s hardware interface and adjusted via the
software GUI by Metrico’s engineers to set the ‘Attenuation’ value returned by the POLQA algorithm
to as close to 0 as possible. The waveform is visible and audible in the software allowing the
engineers to monitor audio levels during the tests.

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
2.2 TEST PROCEDURE
Measurement activities were conducted by Metrico Wireless, and POLQA scores were validated by
OPTICOM.

Test Locations

All tests were performed in static locations in the Reading area where WB-AMR/HD-Voice service
was verified as being supported by the networks via engineering mode screens on the handsets.

Reference Speech Sample

All tests were performed using a Super-Wideband, English language, natural voice, reference
sentence pair featuring two speakers, one male, one female of 10 seconds duration.

Call Pattern

Nomad sequences a half-duplex conversation between the two mobile devices, each sample lasting
approximately 10 seconds in each direction, thus yielding 3 samples per speech path per minute.
A statistically meaningful sample size (approximately 90 per speech path) was captured on each
network in both the wideband and narrowband device setting, this represents approximately 40
minutes of call per test.

4
Typically the calls are restarted every 10 minutes. The results for each speech path
(mobile 1   mobile 2, mobile 2   1) are then aggregated for each test.

Key Performance Indicators

In Super-Wideband (SWB) mode the POLQA algorithm returns a MOS value ranging from 1-4.75 for
each sample. Nomad provides mean, standard deviation and maximum values for the sample set
along with a distribution of scores as a standard output.

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
3 RESULTS
The results for the separate tests performed on both networks are presented below. The two results for
each network represent the devices configured with and without AMR-WB (HD-Voice) enabled.

3.1 NETWORK 1

Mobile-to-Mobile Calls (All Samples)

Network 1 AMR-WB AMR-NB
HD-Voice Standard

Average MOS (SWB) 3.79 2.93

Standard Deviation MOS (SWB) 0.34 0.22

Maximum Score MOS (SWB) 4.03 3.14

Count 256 248

% MOS (SWB) greater than or equal to 3.0 98.4% 42.3%

% MOS (SWB) less than 2.7 1.6% 6.9% 5

Network 1 MOS Distribution Both Speech Paths

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
3.2 NETWORK 2

Mobile-to-Mobile Calls (All Samples)

Network 2 AMR-WB AMR-NB
HD-Voice Standard
Average MOS (SWB) 3.72 2.73

Standard Deviation MOS (SWB) 0.29 0.21

Maximum Score MOS (SWB) 4.03 3.12

Count 243 240

% MOS (SWB) greater than or equal to 3.0 97.5% 12.9%

% MOS (SWB) less than 2.7 0.8% 53.8%

Network 2 MOS Distribution Both Speech Paths

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
Within Network 2 there was the observed dip and broader range of scores in narrowband speech quality.
Although call data was not captured for confirmation it is believed that this is due to a selection of half-rate
and EFR calls.

Regarding overall quality on both networks, a significant improvement in average MOS (SWB) score,
percentage of scores greater than 3.0, and maximum recorded value was evident on both networks when
using WB-AMR/HD-Voice. Typically the difference between the averages was well in excess of 0.75 of a
MOS point, a substantial change given the majority of subscribers would perceive a difference in speech
quality where relative MOS values exceed 0.3.

This case study demonstrates a clear increase in the predicted perception by end users of the quality of
new HD-Voice services over that of traditional narrowband speech. Measurements showed the increase to
be in excess of 0.6 MOS (SWB), and even more so where half-rate codecs are in use.

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
4 ABOUT THE TECHNOLOGY
Nomad is a uniquely scalable, centralized platform utilized by leading carriers and OEMs to evaluate and
improve voice and call performance in 2G, 3G and 4G devices, networks and services. Nomad leverages
patented voice testing technology, which enables you to test thousands of calls automatically and instantly
generate performance statistics. Nomad enables large-scale performance testing for new device launches,
post-launch troubleshooting and competitive benchmarking. As 3G and 4G networks increasingly drive
converged voice and data traffic, Nomad enables service providers to assure call and voice quality in
a multi-radio access bearer or multi-service scenario. Nomad is considered the most comprehensive,
accurate and easy to use mobile call and voice user experience measurement platform commercially
available.

4.1 NOMAD ARCHITECTURE

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
4.2 POLQA
For some ten years now ITU-T P.862 PESQ has been available from OPTICOM, and has served the
industry well in that time. The evolution of networks has however introduced new challenges for perceptual
measurement of voice, not recognized at the time of PESQ standardization.

ITU-T P.863 POLQA (which stands for “Perceptual Objective Listening Quality Analysis”) is the next-
generation voice quality testing technology for fixed, mobile and IP-based networks and will be used with
HD-Voice, 3G and 4G/LTE.

POLQA’s radically revised psycho-acoustic and cognitive model allows, for the first time in the history of the
ITU-T P.86X series, a true quality prediction for:

• EVRC type codecs

• Noise reduction and voice quality enhancement
• Time-warping, UCC and VoIP
• Non-optimal presentation levels
• Filtering and spectral shaping
• Recordings made at an ear simulator

POLQA fits well with new transmission technologies in service now or to be launched in the near future, and
provides stable and accurate results along with an improvement in performance for existing technologies.

More info on POLQA can be found on http://www.polqa.info/

9

4.3 AT A GLANCE: PESQ VS POLQA

PESQ POLQA
Codecs AMR AMR EVRC-WB
GSM EFR AMR-WB iLBC
GSM FR GSM FR AMB+
GSM HR GSM HR AAC
GSM EFR Skype/SILK
EVRC G.711
EVRC-B G.729
Reference Speech Material 8 kHz 8 kHz
(sampling frequency) 16 kHz 48 kHz
Applications POTS POTS Voice Enhancement Devices
VoIP VoIP Skype calls
2G 2G Benchmarking CDMA and GSM
3G 3G Acoustical Measurements
HD-Voice

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
4.4 NARROWBAND AND SUPER-WIDEBAND SCORES

In traditional telephony scenarios, the expectation is set at a perfect narrowband (300-3400Hz) voice
signal. A signal that is close or identical to such a signal is scored subjectively by human listeners with a
high quality value (usually a MOS-LQ of around 4.5 on a five-point scale). Additional degradations will
decrease the quality value towards 1.0. A typical value for a perfect mobile connection using EFR (or AMR
-FR 12.2) is still around 4.0. Interruptions, unconcealed transmission errors and noises easily push the
quality into lower regions.

Looking to the foreseeable future, within a super-wideband (50-14000 Hz) scenario the expectation of
excellent quality is a perfect super-wideband speech signal. Since the same five-point scale is used, such a
perfect super-wideband signal is also subjectively scored close to excellent in the range of 4.75. Consider
a time in the future when super-wideband call quality is the norm and the expectation of good quality is
set. The experience of taking a call with narrowband characteristics would be disliked and scored low as a
result. For today therefore, two scales are required, one to cater for traditional (legacy) narrowband
measurements, and one for current and future measurements, covering narrowband, wideband and
super-wideband and allowing objective comparisons. Using the Super-Wideband scale, voice quality
measured of a wideband call will be scored with marginally lower scores than a true super-wideband signal
due to the bandwidth limitation. Narrowband signals will be scored even lower. Hence, scores given on the
two different scales must not be mixed or directly compared.

10

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
4.5 POLQA NARROWBAND AND SUPER-WIDEBAND PERCEPTUAL MEASUREMENT

For the reasons explained above, the new ITU-T P.863 algorithm covers two operational modes,
narrowband telephony (superseding PESQ) and super-wideband communication.

The following table shows typical values to be expected from POLQA. These were also confirmed by
subjective auditory experiments. (Note: These are averages over a large number of different samples.
Individual measurement points may vary significantly).

MOS-LQO super- MOS-LQO narrowband

wideband 50-14000 Hz 300-3400 Hz
Transparent transmission 4.75 -
50-14000 Hz or wider
Transparent transmission 4.3 -
50-7000 Hz (‘old’ wideband)
AMR-WB 12.65 kbps (50-7000 Hz) 3.8 -
Transparent transmission 3.0 3.6
300-3400 Hz (‘POTS’)
G.711 (A-Law standard PCM) 3.5 4.3
EFR/AMR-FR 12.2 kbps 3.2 4.2
EVRC 9.5 kbps 3.0 3.9
11
EVRC-B 9.5 kbps 3.0 3.9

From this table it is seen that the difference between a traditional mobile call (narrowband EFR/AMR
12.2kbps) and an HD-Voice mobile call (AMR-WB) is likely to be about 0.6 MOS.

4.6 WIDEBAND MEASUREMENTS

While PESQ ITU-T P.862.2 was designed for measurement of wideband voice quality, the original validation
process was not as extensive as it has been for POLQA ITU-T P.863. Consequently POLQA is more
closely aligned to wideband subjective results when used with a Super-Wideband reference.

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com
For More Information:

Metrico Wireless, Inc. OPTICOM GmbH

7340 Executive Way, Suite A Nägelsbachstrasse 38
Frederick, MD 21704 D - 91052 Erlangen
USA Germany
800.714.6421 | 410.489.9133 +49 (0) 91 31 - 5 30 20 - 0
info@metricowireless.com info@opticom.de

Metrico Wireless Limited

Davidson House
Forbury Square
Reading RG1 3EU
United Kingdom
+44 (0) 1189 001617
metricoemeasales@metricowireless.com

Metrico Wireless Taiwan, Ltd.

Neihu Road, Section 1, Lane 360, No. 8, 8th Floor
Taipei, Taiwan 114
12
886-2-2659-9119
asia@metricowireless.com

Metrico, the stylized Metrico logo, and Nomad are trademarks of Metrico Wireless, Inc.

POLQA ®, PESQ ® are registered trademarks of OPTICOM GmbH, Erlangen, Germany. Used by
permission under license.
www.polqa.info 050312

Metrico Wireless, Inc | 7340 Executive Way, Suite A | Frederick, Maryland 21704 | www.metricowireless.com