Industrial Communication Whitepaper

Corporate IT vs.
Industrial Ethernet
Contents
Introduction ....................................................................................................................................... 3
Network Architecture......................................................................................................................... 4
Performance ..................................................................................................................................... 5
Environmental Factors ...................................................................................................................... 8
Configuration and Maintenance ........................................................................................................ 9
Conclusion ........................................................................................................................................ 9

2
Introduction
As Ethernet grows increasingly popular as an Industrial application protocol, many choices exist
relating to the nature of infrastructure equipment for a particular application. In migrating to
Industrial Ethernet, how much can be lifted from the Business IT world rather than integrating a
purpose-designed Industrial Ethernet regime? The Ethernet protocol has traditionally been within
the realm of the IT department. Potential conflict can arise when engineering staff begin placing
what are essentially field level devices – until recently unrelated to mainstream IT – onto the
corporate Ethernet based network. Where different application priorities must coexist on a
common network, how should the resulting compromise be decided?

Network designs used by IT for business solutions generally follow a well established methodology
and the equipment has been optimized for the environmental and application requirements of
business. While IT personnel understand Ethernet and the corporate network very well, they may
not be aware of the differences and requirements of that same network standard running in an
industrial environment.

For example, Industrial Ethernet has several performance and redundancy requirements which set
an unusually high benchmark for reliability and recovery in the event of a fault. In a business
setting, if network congestion causes a file server or email server to hang for a few seconds, this
does not typically present a problem to end-users. With hundreds of clients sharing the same
resources, business network congestion often goes unnoticed. Should an Industrial Ethernet
application experience a similar delay, the result could manifest itself as extended production
downtime.

With network characteristics so greatly dependent on the structural components and the way in
which they are set up, let’s examine some key differences between traditional business Ethernet
infrastructure and its industrial counterpart.

3
Network Architecture
Businesses share many similarities regarding network architectural requirements. Typical offices,
cubicles, and meeting rooms drive port density requirements to the edge of the network. Starting
from a floor plan of the building, overlapping circles representing a 90m horizontal cabling radius
are located to ensure complete coverage for present and future requirements. Within each of
those circles there are closets or rooms identified to locate Ethernet switch stacks ensuring that no
Ethernet cable will exceed the 100m length limit. Stackable switches, located within that radius,
have the advantage of accommodating additional ports as the business grows or changes. The
closets they are located in house standard 19’ rack frames sharing the climate control of the
building. These are termed Intermediate Distribution Frames (IDFs).

The IDFs located throughout the building or campus are then connected back to the Main
Distribution Frame or MDF, typically a raised floor computer room which houses the network core.
This core comprises critical resources of the business including servers, routers, and the network
interface to the Internet or Intranet. The media used to connect the IDF to the MDF is typically high
bandwidth Gigabit Ethernet multi-mode fiber that accommodates an inter-connection and does not
exceed a 2km distance limitation. Taken overall, this approach represents a virtual business
infrastructure standard.

For industrial applications, port density varies widely. Some applications require just a few
Ethernet ports in each location and those applications may be several hundred meters apart.
Unlike offices, industrial applications have very different node density requirements. Placing a
‘corporate standard‘ 24-port Ethernet switch in a location that only requires four ports is expensive
and wasteful. Add to that a possible need for a fiber optic interface that can span 3km or more to
the next switch and you end up with a rather different infrastructure requirement – not typically met
with a business class component.

4
Performance
An even more obvious difference between Business and Industrial Ethernet relates to performance
criteria. The business performance requirements are driven by whether the application is client-
server, such as connectivity to a database or a Voice-over-IP (VoIP) phone connecting to VoIP
PBX, versus a core network resource such as an e-mail server. Client-server application classes
are more critical in terms of connectivity and this drives the benchmark requirement for
performance as well as redundancy. Such client server applications tend to be more sensitive to
packet loss or delays, but can still maintain connectivity for several seconds before terminating a
session. Applications such as e-mail replication from servers to the desktop client can tolerate
more substantial delays without impact. Because file server resources for VoIP PBX, database,
e-mail, and enterprise are typically located at the network core, redundancy and performance
requirements tend to be focused on this same area rather than at the edge of the network where
the users actually are.

There is also a matter of network redundancy. For Business applications, should a key
infrastructure device such as a switch or router experience device or media failure, recovery within
seconds is considered acceptable. Redundancy methods employed include the Spanning Tree
Protocol (STP), or the evolved derivative, Rapid STP (RSTP). The timers used in these failover
protocols do not resolve values below a second: healing the network by finding an alternative
route typically takes from between a second to a minute. While this accommodates resources
such as e-mail, web, Internet, Intranet, and file services, there is the minor inconvenience of
perhaps dropping a phone call or having to log back into a database when the session was
terminated. In any event, the productivity lost is measured in seconds or minutes and only on an
individual user basis. Should there be device or media failure, the impact to the business is
localized depending on the activities at the time and limited to individual users or resources.

Compare this to the industrial application with essentially different traffic patterns and sensitivity to
recovery time in the event of a switch or media failure. While business applications steer traffic
from the edge of the network to core resources, Industrial Ethernet applications tend to be peer-to-
peer. Data must be exchanged between peer devices at the network edge so that programmed
logic can respond to upstream or downstream changes, as well as share those changes with
SCADA, HMI, I/O devices and other PLCs. The peer-to-peer nature of Industrial Ethernet drives
communications and redundancy requirements rather than the edge-to-core requirements used by
business/IT.

These Industrial Ethernet essential requirements may be defined as:

 Should a media failure occur, the recovery time does not affect the production application.
 The packet rate of communications must be supported by the infrastructure.

Regarding the first requirement, media recovery taking longer than a second is unacceptable in
most Industrial applications. Any interruption longer than this would usually stop production
leading to downtime. Redundant Ethernet ring topologies provide sub-second recovery time and
substantially address the problem. Other topologies such as dual homing in conjunction with
Rapid Spanning Tree may also provide an acceptable solution.

Media or device failure also takes on another dimension in an industrial application.

While business systems typically operate during business hours, many industrial processes are
continuous. Media failure out of hours may require expensive correction.

In terms of the second essential requirement, Ethernet switches and devices now support full
duplex communications. Any device or switch can transmit and receive simultaneously and the
bandwidth available is 100Mbps. Fast Ethernet is more than sufficient. In fact, the bandwidth
exceeds the capability of the devices to completely consume it. Additional traffic management
5
services such as Quality of Service reinforce efficient and reliable message delivery. Broadcast
Rate Limiting is also employed to prevent excessive broadcasts from congesting device queues
that would delay automation message processing. While business Ethernet switches of course
offer such features, the industrial counterparts have evolved by making these attributes prominent
and more easily managed using a web browser so that engineering or maintenance can use them.

Performance differences between business and Industrial applications reveal that packet rate,
packet size, and bandwidth use differ significantly and are summarized in Table 1.

Table 1: Performance Differences Between Business and Industrial Applications

Graphing test application packet captures for both industrial and business edge devices illustrates
the difference. In Figures 1 and 2, the business application trend is in black and the automation
application trend is in red.

Business applications have packet burst rate requirements that define the upper limit bandwidth
requirement of the infrastructure. The available 100Mbps bandwidth from the network edge to its
core can be easily saturated when multiple users send large packet bursts. The data volume
represents the larger packet sizes used by business, for example, when loading web pages or
network printing.

6
Industrial applications on the other hand, have a higher, steady state packet rate with lower overall
data volume due to the much smaller packet size as shown in Figures 3 and 4. Industrial Ethernet
packet sizes are typically less than 500 bytes. These smaller packets interleave more effectively
than larger packets and therefore reduce the likelihood of congestion.

Since most Industrial Ethernet applications use cyclic, steady state communications, packet bursts
are rare. Coupling the packet sizes and packet rates together reveals the average bandwidth
occupancy over a five minute test capture period. As Fig. 5 illustrates, the overall utilization is
under 2 Mbps and this is typical for many Industrial Ethernet applications. Though some voices
may express concern about the high volume of communications in Industrial Ethernet, the reality is
that 100Mbps full duplex Ethernet is quite sufficient for most applications. For business
applications, applying increased bandwidth generally yields better performance. For Industrial
Ethernet applications, additional bandwidth has little effect. Most automation edge devices do not
have the processing speed and memory of business desktop PCs and servers. The key factors for
industrial applications are efficiency and reliability.

7
Environmental Factors
Business infrastructure Ethernet switches are typically installed in temperature, air quality, and
humidity controlled areas with filtered power, standard surge suppressed outlets, and UPS backup
the event of a power failure. There is less consideration for power redundancy in terms of dual
power supplies or separate feeds for edge switches. Core switches are more likely to be equipped
with redundant power supplies and this is reflected in the premium price of the core business
switch.

Business Ethernet switches are equipped with fans to cool the switch mother board. Should
moisture or dust accumulate on the system board, the board could overheat and fail. Since the
stackable edge switches are designed to accommodate typically 24—48 ports should an edge
switch fail due to environmental factors, the impact to the business is localized, affecting only those
users attached. This hardware philosophy extends to many business edge switch designs. Cost
considerations dictate that redundant power is not even an option due to the relatively localized
user impact of component failure.

Since the switch stacks at the edge of the IDF employ standard plug grounding, and office
installation cable trays physically avoid EMI/RFI- producing sources such as HVAC and fluorescent
lighting ballasts, there is less consideration given to the use of shielded horizontal cabling. This
assists cost control when deploying or locating IDF closets.

Slightly below the line but important none the less, corporate IT expects to replace edge switch
equipment as Ethernet bandwidth increased on three to five year intervals to accommodate new
technology and services. This has subtly affected the corporate attitude to MTBF in favor of
providing higher bandwidth to the desktop edge through hardware upgrade on a regular basis.
The historic example of Moore’s Law drives business Ethernet switch OEM product life cycle and
development.

Industrial Ethernet systems are expected to keep working for substantially longer in much harsher
environments. Fans that can draw in dust to the detriment of the enclosed electronics can threaten
premature failure – never convenient and often expensive. For this reason, fanless convection
cooling is widely preferred by Industrial Ethernet switch OEMs (and sensible users). Industrial
systems should require minimal upkeep. Environmental factors such as temperature, dust, debris,
or humidity have to be properly designed for. System architects must be prepared to see
production yields cost savings in the light of a cost increment on the switch itself. Industrial switch
OEMs have responded by testing and certifying their product offers to survive these environments
with long MTBF cycles.

Electro-magnetic and radio frequency-emitting devices are common in industrial environments.

Using shielded media and positive grounding not only protects the network infrastructure itself, it
also reduces the risk that data packets will be corrupted during transit. The integrity of each packet
is critically important to prevent delay as well as relying on higher layer protocols such as TCP to
manage retransmission and error recovery mechanisms.

8
Configuration and Maintenance
Business has highly evolved Help Desk and Network Operations support systems for configuring,
managing, and maintaining the network. Problem reports are typically dispatched through a call
center, which prioritizes each request and assigns it to the appropriate group or individual. Often
these people share their time with projects related to deployment or roll-outs, or learning a new
application while endeavoring to support it. When rolling up industrial applications and their
problems into this type of organization, support request may get prioritized wrongly. If a VP cannot
get their email for example, that may have a much higher priority than a person on the production
line reporting that a packaging machine is down and is holding up the entire line. Help Desk
personnel may respond to requests more towards the political hierarchy than the bottom line
business impact.

Industrial network operators usually require immediate assistance in the event of a problem. Using
general Help Desk personnel is a risk, particularly if they are unfamiliar with Automation
applications.

Responsibility for keeping the industrial network in service normally falls to the maintenance
department. Maintenance usually has a fraction of the Ethernet knowledge held by the Help Desk.
Industrial Ethernet OEMs have responded by developing simple tools, methods, and interfaces to
allow the maintenance person to resolve the problem. Special setup and diagnostic features with
web interfaces simplify the configuration options, or allow discovery and primary configuration of a
switch by simply powering it up and plugging into the network…with little or no reliance on the IT
Help Desk.

Fast replacement in the event of a device failure is essential for industrial applications and has led
to simple methods for replacing a device. Should a switch or I/O device fail, maintenance can
replace the device and the configuration – often with just a screwdriver – by matching settings on
rotary switches.

Signal interfaces can also help in alerting network problems. Ethernet switch housekeeping
signals can be wired directly to PLC I/O modules and monitored for errors in logic. Should an error
occur, this can be reported to an operator via the HMI console. All of this recommends Industrial
Ethernet switches for use in industrial applications.

Conclusion
Industrial Ethernet continues to evolve and mesh with the traditional business Ethernet
environment. Engineering system architects would do well to partner with IT to find
complementary technical solutions and cross training resources to support industrial application
according to their need.

Business IT should also recognize the importance of Ethernet infrastructure devices suited to the
task and that for Industrial Ethernet, the corporate standard for Ethernet is not necessarily best
applied in an Industrial environment.

9
> Make the most of your energy

Schneider Electric Industries SAS

35, rue Joseph Monier Due to possible changes in standards and equipment, the features described in this document in the form
F-92500 Rueil-Malmaison of text and images are subject to confirmation by Schneider Electric.
FRANCE Whitepaper by Michael Roche
Tél. : +33 (0) 1 41 29 70 00 Design: Schneider Electric

www.schneider-electric.com 10