PTP IEEE 1588 Testing at Itaipu Binacional Dam - First preliminary results

towards an integrated time synchronization solution.

Maurcio Menon (Itaipu Binacional), Elena Villalba (Centro de Investigacion FPUNE), Igor Habib
(Itaipu Binacional)
menon@itaipu.gov.br, selva.villalba@fpune.edu.py
Brasil, Paraguay

1.

INTRODUCTION

This paper aims to publish and discuss the first preliminary results of Precision Time Protocol
(PTP) version 2 testing for Itaipu Binacional in a laboratory set up with a set of multi-vendor
equipments.
Itaipu Binacional (IB) is a hydroelectric dam (power plant) on the Parana River located on the
border between Paraguay and Brazil, with installed capacity of 14.000 MW. Its the largest
operating hydroelectric facility of the world in terms of annual generation capacity, exceeding 90
TWh.
As the main supplier of electricity for Brazil and Paraguay, any change in the power plant is a very
complex job. For example, IB has a large substation in the Paraguayan side and a transmission
line with few kilometers in the Brazilian side, making it a special case of a large environment
containing power generation and transmission (substation and transmission line) parts of the
electrical system. This context brings the need of an integrated time synchronization solution, one
that could use the same type of network, shared and compliant with IEC 61850 standards, such as
PTP over Ethernet networks.
PTP (Precision Time Protocol) is a robust time synchronization solution that can be used for
several purposes in telecommunications, power systems, instrumentation and automation fields,
among others. In the power systems field, some applications need special performance
requirements, sometimes including time synchronization. Some Intelligent Electronic Devices
(IEDs), Phasor Measurement Units (PMU), Digital Fault Registers (DFR) and Merging Units (MU)
are examples of current power systems automation technologies that require timing information
with high-level accuracy.
Considering the case of PMUs and the application of IEEE 1588 for synchrophasors measurement
and diffusion, the IEEE C37.118-2005 standard defines some time error limits as PMU
requirements. It states that, since a time error of 1 s corresponds to a phase error of 0.022 for a
60 Hz system and 0.018 for a 50 Hz system, a phase error of 0.01 radian or 0.57 will by itself
cause 1% TVE (Total Vector Error), which corresponds to a maximum time error of 26 s for a 60
Hz system, and 31 s for a 50 Hz system.
PTP attends these requirements since it was designed to produce nanosecond-level accuracy.
Besides that, it works well in Ethernet networks, de facto standard compliant with IEC 61850
standards, the current state of the art for substation automation.
There is a PTP profile which aims to define PTP IEEE 1588 version 2 utilization for IRIG-B
replacement and use with IEC 61850 and IEEE C37.118 applications. It is published as IEEE
C37.238 standard, entitled "Use of IEEE Std. 1588 Precision Time Protocol in Power System
Applications".

Pgina

This paper shows preliminary results of Precision Time Protocol (PTP) testing at Itaipu Binacional
(IB) dam. It presents results of PTP performance under low and heavy traffic (FTP bulk file transfer)
situations, showing that PTP does not suffer any significant performance penalties, even in adverse
situations. This study represents the first practical experiment using PTP for IB, paving the way for
newer and more complex scenarios towards a technological modernization considering time
synchronization for power systems automation equipments in compliance with IEC 61850.

Pgina

2.

ITAIPU BINACIONAL

Itaipu Binacional (IB) is a hydroelectric dam (power plant) on the Parana River located on the
border between Paraguay and Brazil, with installed capacity of 14.000 MW. Its the largest
operating hydroelectric facility of the world in terms of annual generation capacity, exceeding 90
TWh. The peak of IBs participation in the Brazilian market was reached in 1997, when 26% of the
countrys electricity demand was supplied by the plant. The construction began in 1974, with the
arrival of the first engineers. In 2007, the last two units were added to the plant, and the total
number of 20 generating units was reached.
IBs main goal is to keep the reliability of the power plant and provide quality in the power
generation. A fault that cuts its power generation off from the Brazilian integrated system could
leave several states without electricity and cause a national blackout. The only way to avoid it is to
invest in maintenance and new technologies.
As the main supplier of electricity for Brazil and Paraguay, any change in the power plant is a very
complex job. For example, IB has a large substation in the Paraguayan side and a transmission
line with few kilometers in the Brazilian side, making it a special case of a large environment
containing power generation and transmission (substation and transmission line) parts of the
electrical system. This context brings the need of an integrated time synchronization solution, one
that could use the same type of network, shared and compliant with IEC 61850 standards, such as
PTP over Ethernet networks.
Aiming to implement a plant upgrade, IB is under a process called technological modernization,
which consists of choosing solutions based on IEC 61850 standards. In order to succeed, the
engineering department of IB formed a special group of professionals from several different sectors
to research IEC 61850 compliant technologies and other ones that could work together with them.
PTP tests at IB started in mid 2010 using a lab within its limits. The study was the beginning of a
process of PTP analysis, specially related to power system automation [4]-[6].
Time synchronization is very important in substation automation. It applies to Ethernet
transmission, to protocols such as GOOSE and MMS, in the acquisition of real-time data from the
IEDs, RTUs, and MUs and also in real-time process control of equipments such as protection
relays and performance analysis.
There are generally two types of time synchronization: direct time and through a LAN.
Replacement of IRIG-B cabling with Ethernet cabling, is a benefit that ensures a milliseconds
precision for event sequence time in substations. Another benefit is a microseconds precision for
critical applications (IEC 61850-9-2), or to the synchrophasors implementation. Using time
synchronization also gives the benefit of a complete restructuring of the substation with Ethernet
networks, facilitating the transition from legacy solutions to the version II of the IEC 61850. IEEE
1588 also known as PTP Protocol is able to offer a possible synchronization accuracy to attend the
requirements in the standard time syncrophasor C37.238

2.1

TESTING COMPONENTS AND SCENARIO

The scenario used for testing consisted of 5 main components, described as follows:

A.

PTPv2 Time Server

The PTPv2 time server used for testing was the Meinberg LANTIME M600/GPX/PTPv2 Server,
equipped with coaxial cable and Global Positioning System (GPS) antenna.

Pgina

The M600 acts as the IEEE 1588-2008 grandmaster clock, providing accurate source of
synchronization for PTP clients (slaves like the Meinberg PTP270PEX PC board see item D as
follows) and additionally introducing the absolute time (current time of day) to the PTP network,
since it has GPS integration.
The configuration of the system can be done by using front panel buttons and the display, a
standard web browser to access the HTML interface or, alternatively, a text based and menu driven
setup utility that can be started from the shell prompt after logging into the unit via Telnet or SSH.

B.

PTP Switches

The PTP compliant switch used for testing was the Hirschman (Belden) MICE-20 MS20 industrial
switch with the MM23 media module containing 4 Fast-Ethernet (10/100 Mbps) RJ-45 ports.
The switch is IEEE 1588 PTPv2 compliant, implementing both the transparent and boundary clocks
functionalities.
As a transparent clock, it can be configured in de P2P (peer-to-peer) or E2E (end-to-end) modes.

C.

PTP Client

The PTP client used for testing was an industrial PC with the following specification for its main
components:

4U SDC SDC-600 chassis;

HPE-8S0-R passive backplane;

4 PCI slots;

2 PCI Express x1 slot;

PICOe-945GSE CPU card;

Intel Atom N270 1,6GHz processor;

2,5GB of RAM memory.

D.

PTP Board

The PTP board used for testing was the Meinberg PTP270PEX PC board, attached to a PCI
Express x1 slot in the industrial PC backplane.
It plays the ordinary clock role on the PTP network, getting synchronized to the grandmaster clock.
The chosen oscillator was the OCXO MQ option with the following specifications [7]:
-10

short term stability: 2x10

accuracy of PPS (pulse per sec): < 100 nsec;

(1 sec interval);

phase noise: -75dBc/Hz (1Hz), -110dBc/Hz (10Hz), -130dBc/Hz (100Hz), -140dBc/Hz

(1kHz);
-9

accuracy free run, one day: 1.5x10 15mHz;

accuracy free run, one year: 1x10 1Hz;

-7

Pgina

12

accuracy GPS-synchronous, averaged 24h: 5x10- ;

accuracy of time free run, one day: 65 s;

accuracy of time free run, one year: 1.6 s;

temperature dependant drift free run: 5x10 (-20...70C).

E.

Measurement Unit

-8

The measurement unit used was the Tektronix MSO4104 Mixed signal Oscilloscope with the
following features:

logical analyzer functionality;

4 analog channels;

16 digital channels;

1 GHz, 5.0 GS/s and 10 M sample record length on all channels;

USB port and network interface;

support for the TekVISA protocol, an implementation from Tektronix to the well known
Virtual Instrument Software Architecture (VISA).

The testing scenario is illustrated in Fig. 1, showing the connections among the described
components.

Time Server
Meinberg Lantime M600

Tektronix MSO4104
Oscilloscope

Hirschmann PTPv2 Switch

CH2
CH1

Ethernet

PC with Matlab

PTP v2 Client with Meinberg PTP270PEX board

Figure 1.

3.

Ethernet Network

Testing Scenario

METHODOLOGY

The test was based on measuring the time offset between client and server PPS outputs with the
oscilloscope. It was able to send data collected through the network for a computer database for
later processing, using TekVISA. The computer had MATLAB installed with the Instrument
Control Toolbox available and the Tektronix MSO4104 oscilloscope driver installed. The program
itself was able to read the values of the measurement in the oscilloscope, one by one, and store it
in an internal MATLAB database for later use. The trigger adjustment was set to 1.4V, value

Pgina

indicated by the equipment supplier to TTL signals. The graphical and statistical analyses were
made with MATLAB.
Two tests were made. The first one without network traffic followed by the second one containing a
bulk file transfer (FTP), in order to simulate heavy traffic conditions. Only tests with end-to-end
(E2E) mechanism were made.
The offset value was considered with its absolute value for calculation and evaluation in this paper.
However, in the graphics, positive and negative values were used for better understanding of
protocol behavior and compensation.
A simple tool was used to see the Fast Ethernet bandwidth utilization in the tests, called Bandwidth
Monitor. The FTP server and client showed values coherent with the tool. The only goal was to
check visually the utilization.
The time server was rebooted in the beginning of each test to certify that no previous condition or
state could interfere in the results. The capture started shortly after the client changed its status to
SLAVE.

4.

OBTAINED RESULTS

A.

Test without traffic load

Without heavy traffic occupying the bandwidth available in the network, the performance of PTP
measured by the time offset between client and server stayed in the nanoseconds to tens of
nanoseconds range. Fig. 2 shows a sample value of -21,12 ns for the offset, captured during the
tests. The upper signal corresponds to the PPS output of the client (ordinary clock) and the signal
below it corresponds to the PPS output of the time server (grandmaster clock).

Figure 2.

Oscilloscope capture of an offset sample. (test without traffic)

The oscilloscope data for the whole set of offset sampling is plotted in Fig. 3.

Pgina

Figure 3.

Whole set capture of offset sampling. (test without traffic)

Fig. 4 shows a detailed offset sampling for the time interval between 600 and 660 seconds.
The following figure contains a detail between entre 600 e 660 seconds.

Figure 4.

Offset samples between 600 and 660 secs. (test without traffic)

The statistical analysis of the entire interval presents:

mean value: 12,16 ns;

minimum value: -99,63 ns;

maximum value: 23,27 ns.

standard deviation: 18,83 ns;

first standard deviation interval: contains 93,17% of the samples.

The time interval after the stabilization period, arbitrarily set at 180 s, until the end of the testing
period (20 minutes), has a total of 1020 samples. For this interval without the transient behavior,
the statistical analysis presents:

mean value: 7,170 ns;

standard deviation: 4,912 ns;

first standard deviation interval ( ): contains 697 samples (68,33%);

Pgina

second standard deviation interval ( 2): contains 975 samples (95,59%);

third standard deviation interval ( 3): contains 1008 samples (98,82%).

Fig. 5 below presents the histogram with all data obtained in the test without heavy load (e.g.,
considering also the transient behavior). The range between -10 ns and 10 ns has 69,75% off the
samples.

Figure 5.

B.

Complete histogram of the offset samples. (test without traffic)

Test with load (FTP)

A second testing session was made to check the interference of heavy load in the PTP
performance. Between the reboot of the time server and the client status change to SLAVE, a bulk
FTP transfer of a 4GB sized file was initiated.
The transfer was made in both directions, each direction at a time, to check any asymmetry. The
samples were collected just after the client status change (SLAVE).
A Fast Ethernet link was used for the tests. Fig. 6 shows the typical bandwidth utilization during the
FTP transfer, evidencing that a real heavy traffic condition was created.

Figure 6.

Bandwith utilization. (test with FTP traffic)

Pgina

Fig. 7 contains the whole set of offset sampling. The major part of the values is situated between
25 ns and -25ns.
The statistical analysis of the entire interval presents:

mean value: 20,38 ns;

minimum value: -198,22 ns;

maximum value: 17,23 ns.

standard deviation: 33,99 ns;

first standard deviation interval: contains 90,25% of the samples.

Its noticeable the adjustment in the beginning of the synchronization.

Figure 7.

Whole set capture of offset sampling. (test with FTP traffic)

Fig. 8 shows a detailed offset sampling for the time interval between 600 and 660 seconds.

Figure 8.

Offset samples between 600 and 660 secs. (test with FTP traffic)

Considering only samples between 180 and 1200 seconds (e.g., excluding transient behavior), the
statistical analysis presents:

mean value: 11,74 ns;

standard deviation 19,52 ns;

Pgina

first standard deviation interval ( ): contains 1009 samples (98,92%);

second standard deviation interval ( 2): idem;

third standard deviation interval ( 3): idem.

Fig. 9 below presents the histogram with all data obtained in the test with FTP traffic load. The
range between -25 ns and 25 ns has 88,41% off the samples.

5.

ANALYSIS OF RESULTS

Using a bulk transfer was the manner utilized to test the PTP performance, considering that FTP
has an aggressive bandwidth utilization behavior (see Fig. 6) and could be a real case of backup
and file transfer in a power system environment. Performance data in the ideal condition, without
traffic load, was also collected in order to permit comparison between both scenarios. It was
expected that the traffic load had little to none influence in the performance.
The initial transient was expected. It presented duration from 2 to 3 minutes in both testing
sessions. There is no significant difference in the transient period of time.

Figure 9.

Complete histogram of the offset samples. (test with FTP traffic)

In the detailed interval included, between 600 and 660 seconds, its noticeable the compensation
done by the protocol. After some samples, when its possible to perceive the internal oscillator job,
the protocol starts its task and recovers the offset again, cyclically.

Pgina

10

Figure 10.

Offset samples: without (blue) and with traffic (red).

It is important to notice that PTP quickly converges to tens of nanoseconds, even with heavy traffic
load. In the beginning of the operation there were some samples with hundreds of nanoseconds,
indeed better than other synchronization protocols such as Network Time Protocol (NTP).
A PTP daemon contains fast controlling loops that follow changes in the time server. If there is a
fast change, the control loop quickly follows it. So, an oscilloscope was necessary to correctly
check the offset, using the PPS interface.
The paper shows that there is no significant difference in the performance considering both testing
situations studied, with and without heavy traffic, after the initial transient (Fig. 10).

6.

CONCLUSION

The PTP is a robust and high performance protocol that can be turned into a useful solution for
critical systems. In these tests, tens of nanoseconds were obtained for the offset value, with the
worst values being situated in the hundreds of nanoseconds range. As expected, the FTP bulk
transfer (traffic load) didnt cause any performance penalties.

ACKNOWLEDGMENT
The authors of this paper would like to thank Meinberg staff for all the technical support provided
during these experiments.

REFERENCES
[1]
IEEE 1588 Standard for a Precision Clock Synchronization Protocol for Networked
Measurement and Control Systems, IEEE Std 1588, 2002.
[2]

IEEE Standard for Synchrophasors for Power Systems, IEEE Std C37.118, 2005.

[3]
IEEE 1588 Standard for a Precision Clock Synchronization Protocol for Networked
Measurement and Control Systems, IEEE Std 1588, 2008.

Pgina

11

[4]
M. Menon, Anlise de desempenho do PTP (Precision Time Protocol), Specialization
dissertation, Unioeste, Foz do Iguau, PR, 2010.
[5]
M. Menon and I. Habib, Funcionamento e utilizao do PTP (Precision Time Protocol) na
IEC 61850 presented at the IX SESEP (Seminario del Sector Elctrico Paraguayo), Asuncin,
2010.
[6]
M. Menon and I. Habib, Anlise de desempenho do PTP(Precision Time Protocol)
presented at the XIV ERIAC (Encontro Regional Ibero-Americano do CIGR), Ciudad Del Este,
AP, 2011.
[7]
I. Habib et al., Plataforma aberta para aquisio e processamento de dados aplicada a
sistemas eltricos de potncia presented at the XIV ERIAC (Encontro Regional Ibero-Americano
do CIGR), Ciudad Del Este, AP, 2011.

Pgina

12