ATSC README .

DOC
OVERVIEW
This document is intended to serve as a guide to the ATSC models and associated files.
All the files from this ZIP, and the downloaded data files (see below), must reside in one
directory for everything to work properly. MATLAB must be pointed to this directory.

The objective of this endeavor was to acquire the RF from several ATSC (North
American Television Standard) signals and create a set of models in Simulink that can
decode the signals into MPEG-2 transport streams. These streams can be played with
applications like Windows Media Player to recreate video and audio output. In other
words, create the entire processing chain from RF input to MPEG-2 output in Simulink.
ATSC documents can be found at:
http://www.atsc.org/cms/
http://www.atsc.org/cms/standards/a53/a_53-Part-1-6-2007.pdf

Although many of the models can be run without captured real-world data, the last
stages of this set of examples requires this data. The full set of data can be found at:
http://www.mathworks.com/supportfiles/spc/ATSC/ATSC_Data.zip
This file is about 2GB in size, so be prepared!
Place these files in the same folder as the models.

The files are as follows:

WCVB_Captured_IQ_int8.dat:
The raw captured baseband IQ data.

WCVB_Resampled_int8.dat:
A re-sampled version (4 samples per 8-VSB symbol) of the above file.

WCVB_symbols_int8.dat:
The above file processed though the demodulation and synchronization Simulink
model the output being a file of 3 bit 8-VSB symbols.
wcvb_1.ts:
The end result of processing the above file to produce a high definition MPEG
2 video transport stream.

Aside from MATHWORKS tools, you will need an application that can play MPEG-2
transport streams in order to view the recovered video and audio. Windows Media
Player or this: http://www.videolan.org/vlc/ will do the job. You should associate one
of these applications to the *.ts MPEG-2 transport stream files in the unzipped files.
NOTE: To run the models, type atsc at the MATLAB prompt to open
the example selector.

The final models marked with *** will generate output files and due to the large
amount of data being processed, can take an hour or more to complete. To avoid
overwriting the reference file set, these outputs have an _X appended to the file name.

Also note that there are links to the ATSC standard and the Zenith tutorial. You will
likely want to download the pdfs for reference. Here is the location of the Zenith tutorial
as of September 2013:
http://81-208-25-53.ip.fastwebnet.it/Television/DVB/8vsb-zenith.pdf
Processing Flow Overview
This section will provide an overview of the steps that are required to generate the
MPEG-2 output file for viewing.

1) Signal Capture
By a stroke of good fortune, MathWorks had a Tektronix RSA6106A on extended
loan:

This is a high performance spectrum analyzer with a deep memory for capturing signals.
The ATSC television signals live in the VHF / UHF frequency range which is well within
the frequency range of this fine instrument. The primary function of the instrument for
this application was to down convert the VHF/UHF RF to a baseband (IQ format) signal
that can be stored to its deep local memory. Aside from this, the instrument provides
important real-time spectrum information to aid in setting up and assessing the quality of
the signal before the capture

The ATSC signal occupies a 6 MHZ bandwidth which is identical to the NTSC format
that it replaced. To insure a complete capture, the 6106A was set for a 12.5 MSa/s
(complex) capture rate. The next lower selection of 5 MSa/s would not be sufficient. At
12.5 MSPS, the 6106a is able to capture 20 seconds of ATSC signal gap free into its local
RAM. The RAM content is then saved to hard disk. The resulting IQ data file is in a
Tektronix IQ format (TIQ) and is 2GB in size, with the data in a 32 bit format.

For this file set, a weather forecast broadcast from station WCVB was captured and saved
to a file. The original 32 bit TIQ file format was converted to an 8 bit format to minimize
the pain of downloading this via FTP. Consequently the file size was reduced by a factor
of 4 and ultimately by 5 after zipping.
2) Resampling
As a general rule, signals that are captured are virtually never sampled at a rate that is
needed. In this case, the IQ data was sampled at 12.5 MSa/s, but the rate that is
needed for the synchronization and symbol stream recovery is 4 times VSB symbol
rate (4*10.76223776223776 Msym/s) which turns out to be 43.0489510 Msym/s.

A Simulink model was created to re-sample the captured 476 MB 8bit IQ file by a
factor of 3.4439 (43.04895/12.5) and it creates a 1.75GB IQ file in a 8 bit IQ binary
format. The input data is an 8 bit file format and needed a custom file read block
which was implemented using (embedded) MATLAB. The re-sampled IQ output file
is also an 8bit format and it too relies on EML. The output file grows in size due to
the up-sampling operation.

3) IQ to Symbol Stream
The re-sampled IQ baseband data needs significant processing to transform it into the
required stream of 3 bit symbols. This is where the art comes into play. Carrier
recovery, timing recovery, frame synchronization, and equalization are required to
recover the 3 bit symbol stream from the 8 VSB modulation format in the captured
IQ data. The 1.75 GB IQ data is processed down to a mere 216 MB symbol stream
file. Each 3 bit symbol occupies a byte to simplify the reading and writing of these
files.

4) Symbol Stream to MPEG-2 Transport Stream

The 216 MB 3 bit symbol stream is now fed to the final decoding model which
implements four operations:
Trellis Decoder
De-Interleaver
Reed-Solomon Decoder
De-Randomizer.
All per the ATSC specification. Processing this 3 bit symbol stream input file
generates an MPEG-2 transport stream that is circa 50 MB long for the 20 seconds of
captured data.

In general, MPEG-2 transport streams (file_name.ts) can contain MULTIPLE

independent video streams. For instance, our local PBS station (KQED) emits 3
virtual channels simultaneously (channels 9.1 9.2 9.3). Typically one stream is
high definition (1920x1080) and others will be standard definition (704x480).
The broadcasters love this since one transmitter can now emit multiple streams of
content in parallel. So the 20 second captures can actually have 60 (or even more if
there are 4 streams) seconds of content.

The broadcast station for this example was emitting only one (very nice) 1920x1080i
stream.
Decoded Video File
The final output file wcvb_1.ts contains 20 seconds of a local weather broadcast:
System Level Models
1) ATSC Standard
The ATSC standard has a set of related documents that can be downloaded from here:
http://www.atsc.org/cms/
http://www.atsc.org/cms/standards/a53/a_53-Part-2-2007.pdf

The important one for this work is Part-2 :

In particular, page 9 shows the top level processing that goes from an MPEG-2
transport stream to 8 VSB symbols which are transmitted:
2) Step 1,2,3,4
This sequence of models shows the elaboration of the symbol stream processing
portion of the decoder.
The most difficult item in the
entire project, by far, was the
implementation of the Trellis
Interleaver and Trelis De-
Interleaver. A separate test
harness was built and the gory
details are covered in pages 15-
19 of the ATSC Part 2
document. It took ten different
attempts before this section of
the processing was successfully
implemented. The point here is
you can construct the pair to be
self consistent but they can
deviate from the standard, and
therefore simply will not work
at all for real world signals.

The main hurdle in debugging

this is there is no
straightforward way to tell if
the output data from the
deinterleaver is correct. There
are no test vectors that have known input and output. Real world data must be used
as the input, and the data out of the interleaver must then be processed by the
remainder of the model. This means that the remainder of the model must be 100%
correct to determine if the deinterleaver matches the standard, and one cant be
certain if the remainder of the model is correct! In short, there is no foolproof way to
know if ANYTHING is correct till after the Reed Soloman decoder. This was a very
painful conundrum since stage by stage testing was impossible. It either works, or it
does not, and you are flying blind. Thankfully rapid iterations are made possible by
using Simulink.
3) Simulink System Level Model
The final overall TX and RX system level model is as follows:

The receiver (lower half of the model) incorporates enabled subsystems with Valid
signals that establish the time at which each subsystem starts processing data. The key
here is that the processing of downstream subsystems must be inhibited until there
is valid data available to operate on and this is accomplished with the Valid signals
that propagate between the blocks.

Again, there is no way to know with any certainty that this model truly implements
the ATSC standard. Defects in the modeling of the transmitter can be hidden, and
often are, by complementing defects in the receiver. If known good input / output
test vectors were available for each subsystem, this would help considerably. These
test vectors may well be available to the anointed ones, but I could not locate them.
Synchronization Models
The synchronization models are used to process the captured IQ data and create a 3bit
symbol stream to drive the receiver symbol stream to MPEG-2 transport stream.
Synchronization is required at several levels including carrier, segment, frame, and
equalization.
1) Zenith Tutorial
The Zenith tutorial on the ATSC transmission system was a quite helpful for this
synchronization work.
Unfortunately both of the sources for this important paper have vanished::
http://www.zenith.com/sub_hdtv/hdtv_papers_atsc_system.html
http://choudeng.com/ChrisUniData/uni%20work%200506/Uni2005/Enel332/VSBTutorial.pdf

This document fills in numerous gaps that are missing from the standard, and
provides good clues as to how one constructs a receiver. The document was written in
1997. At this point in time, pure digital signal processing was not quite up to the task,
at least for commercial products. Consequently many of the techniques are explained
in terms of analog and analog mixed signal processing.
For instance, Figure 10 on page 9 is a pure analog carrier lock loop:

Whereas figure 12 is a mixed analog and digital approach using a VCXO to drive
the main sampling ADC:

The approach taken in this Simulink example will be purely discrete time and draw
heavily on the synchronization models and techniques that have been developed over
the past 9 years.
2) Synchronization Model (Synthesized Data)
This model is rather complex. It uses an 8 VSB transmitter model (small leftmost
block in red circle) as the data source for the receiver model:

As can be seen, there are numerous feedback paths in this model. It makes heavy use
of enabled subsystems so that it could (with enough work!) be implemented on an
FPGA or ASIC. This model makes a compelling example of the power of using
Simulink for modeling receiver synchronization. Creating this in MATLAB or any
other text based tool, would be far more difficult.

The transmitter model includes symbol timing rate error, carrier error, and a variety
of impairments that can be switched in and out for testing:

The receiver employs numerous measurement visualization scopes to aid in

debugging and performance quantification. Unlike the previous receiver processing
chain that had the recovered 3 bit symbol stream, this portion of the design can be
debugged and tuned using the information in these test points. For maximum
processing speed, most of the scopes can be disabled and the Simulink accelerator
is a very welcome tool as well.

The single most important display is that of the Equalized Constellation:

This constellation represents the successful

recovery of the 3bit (8 state) symbol stream.
3) IQ re-sampler
The previous synchronization model is built to accept an IQ data stream that is
sampled at 4X the 8VSB symbol rate. Assuming perfect clocks, the exact 8VSB
symbol rate, per the ATSC standard (page 11 of Part 2), is (4.5/286)*684*1e6 =
10.762237762237763 million symbols per second. The desired sampling rate is
therefore 4X this or 43.048951048951054 MSa/s. The Tek 6106a sampling rate was
12.5000 MSa/s so this must be reconciled. A root raised cosine filter that is built to
the ATSC specification (6.9.3 pp 41) is used to up-sample by a factor of 4 and is then
followed by a down-sample of 0.860979020979021. As a side note, using File I/O
between the various models worked out well since the data sizes are so large and
there is no need to build one gargantuan model.

4) Synchronization Model (Captured Data)

This model is simply a copy of 2) above with the transmitter block being replaced
with a block that reads the data generated in 3) above. Testing RX models such as
this with data from the Real World always turns up discoveries. In particular, some
stations have periodic phase modulation at a rate of 360 Hz, which is likely due to a
switching power supply. Other stations have a considerable carrier frequency offset
(10s of kHz) from the standard channel frequency and this is actually intentionally set
to reduce interference to adjacent NTSC stations when both standards were coexisting
for a period of time. Finally, the component that seems to make the largest difference
in terms of receiver performance is the Equalizer. The propagation path from the
transmitter site to the receiver can be tortuous and fast changing multi-path is the rule.
Consequently it was found that using both training and decision feedback
provided the best performance. There could still be much more energy invested in this
area. It is the subject of both papers and patents. Try Googleing atsc equalizer.
 Here is a plot of the input constellation. No trace of the 8VSB can be seen:

The proof of the pudding is the recovered constellation:

The 8 levels are now clearly visible. The resulting 3 bit symbol stream is written to
another file that will serve as the input to the final processing that needs to be done in
the receiver model. The Frame Sync segment has been removed since it is not
needed by the down stream processing and has served its purpose.

ATSC Receiver Decoding

This is last model that is required to create the MPEG-2 transport stream.
1) ATSC Decode of Real World Data.
This model is simply the Receiver portion of the System Level Model outfitted
with a block that can read the 3 bit symbol stream that has now been recovered from
the captured data.
 After the final processing element (Reed Soloman Decoder and DE Randomize) a file
is written out that is (ostensibly!) the recovered MPEG-2 transport stream. The
quality of this recovery is measured by the Error Scope and the Total of Corrected
Bytes readout. A perfect signal would have zero corrected bytes. The RS decoder can
actually correct up to ten byte errors per 187 byte packet. Zenith tutorial page 6:
The Reed-Solomon decoder can correct 10 byte errors or less in a data packet, regardless of how
many bit errors occurred in each byte error, providing good performance in both white noise and burst
noise environments.

Dick Benson

September 2011
December 2011
September 2013