Performance Evaluation of Real-Time Operating Systems

RTEMS, RTLinux, eCos

Real-Time Systems
Systems that interact predictably with events in the outside world Examples:
Flight control systems Collision avoidance systems Satellite guidance systems Patient Monitoring System Control Systems for synchrotrons

Real-Time Operating Systems

Free the applications programmer from writing code for task scheduling and dispatching. Manage time-sharing of a processor for a number of different tasks and interrupt sources, while adhering to strict time constraints. Widely used in all kinds of applications both on PCs and in embedded systems. Generally operate unattended -- consequences of failure can be catastrophic.

What is available?
Many RTOSs have been developed:
Commercial Open source

RTOS vendors and developers publish performance metrics to showcase their products. However, their evaluations are:

not performed on a common platform not comprehensive

Evaluation of an RTOS
How would you choose an appropriate RTOS for your application?

Need for an impartial evaluation. Only evaluated some performance characteristics Only looked at a few commercial operating systems Many are now out of date

Several such projects have been conducted.

Real-Time Consult introduced a more comprehensive project.

Developed a thorough test methodology Evaluated several commercial RTOSs Have not evaluated open source RTOSs

Research Objective
Performance evaluation, on a common platform, of three open source real-time operating systems:
RTEMS RTLinux eCos

Real-Time Operating Systems

Overview
RTOS

Task Scheduling

Task Synchronization

Task Dispatching

Task Scheduling
Time Slicing
Last

Arrival Order
Task 3 Task 2 Task 2 Task 3

First Task 1

Task 1

Time

Task Scheduling
Preemptive Priority
High
Task 3

Priority
Task 2 Task 2

Low Task 1

Task 1

Time

Task Dispatching
Task-control blocks is the most popular method of identifying and managing tasks. Task-Control Block (TCB) contains:

A Typical Task Control Block

Program Counter Task Status Task ID Number Register Contents Pointer to next TCB

a context (e.g. program counter and register contents) an identification string a status, such as ready, executing or blocked a priority (if applicable)

:
Priority Other Context

Task Synchronization
Events Messages Semaphores Mutexes

Desired Performance Characteristics

Tasks should run with:
minimal event latency (the time between the triggering of a task and its start) minimal jitter (the variation in running times of a task that is supposed to run at a fixed period).

Performance Evaluation
Throughput speed at which the system executes instructions Responsiveness how fast it starts to handle in interrupt request Determinism how it reacts under load

how long does it take to finish what it is doing and start handling the interrupt request

Performance Metrics
Low Level Tests
Context switching Interrupt latency Exclusion objects

Semaphores Mutexes

Synchronization events

Context Switch Time

Switch to Task B Save context of Task A Load context of Task B
PC

Task A

PC

Register 0

Register 0

Start Task B

: :
Other Context

: :
Other Context

Context Switch Time

Time

Other Considerations
Not an isolated event Affected by number of tasks pending Depends on priorities of pending tasks Amount of context to be saved

Interrupt Latency
Interrupt and Interrupt Dispatch Latencies
Interrupt Occurs First Instruction in Interrupt Service Routine (ISR) is executed ISR ends Task resumes

Task

ISR

Task
Time

Interrupt Latency

Interrupt Dispatch Latency

Multiple Interrupt Latencies

Interrupt-to-Task Run
Interrupt Interrupt Dispatch Time Interrupt Service Routine Other Interrupt Pre-emption Disabled Scheduling Context Switch Return from System Call

Task Run

Interrupt Task Response Time

Real response to an interrupt often occurs in a task synchronized by, but outside of, the actual ISR. Task response also depends on whether the kernel is preemptable. If not, then kernel call must be completed before ISR.

Performance Metrics
High Level Tests Network throughput Stress tests

Real-Time Consult Project

Evaluation and comparison of RTOS performance Commercial systems only, to date Evaluations available for:

Intime 1.20 RTX 4.2 Hyperkernel 4.3 VxWorks/x86 5.3.1 pSOSystem/x86 2.2.6 QNX 6.1 CE 3.0

Real-Time Consult Project

Test Method: timing is measured using an external PCI bus analyzer. Two types of tests:
Performance Tests Stress Tests

Qualitative Evaluation (API richness, etc.)

Real-Time Consult Project

Performance Tests (context switch & latencies):

Thread switch latency

time required to preempt current thread preempt current thread and start interrupt handler

Interrupt latency

Interrupt dispatch latency

switch from interrupt context to context of interrupted thread or next thread in queue

Thread creation and deletion

Task Switch Latency Test

Task start TSK0

Each Task
Create tasks TSK1, , TSKN Task start TSKN

End of Test? No Write trace on PCI Bus TRC_TSK0_TS0 Yield Processor

Yes

Delete tasks TSK1, , TSKN

Task end TSK0

Write trace on PCI Bus TRC_TSKN_TSN

Yield Processor

Real-Time Consult Project

Performance Tests (objects, file, network):

Synchronization objects

time to create and delete a synchronization object time to create and delete an exclusion object creating and deleting files, reading from files and writing to files in synchronous mode

Exclusion objects

File system operations

Network stack
performance of TCP/IP stack bandwidth for various packet sizes, and CPU usage

Real-Time Consult Project

Stress Tests:
Two simultaneous interrupts Interrupt nesting maximum sustainable interrupt frequency maximum number of objects memory leaks

Focus of my Research
Evaluation of three Open Source RealTime Operating Systems:

RTEMS
Real Time Executive for Multiprocessor Systems

RTLinux
Real Time Linux

eCos
Embedded Configurable Operating System

RTEMS
Developed by the U.S. military as alternative to using commercial RTOS Small, easy to port High level of user configurability Kernel is preemptible
Application Dependent Software
Standard Application Components Device Drivers

RTEMS

Target Hardware

RTLinux
Abstraction layer between the hardware and the standard Linux kernel
Appears as actual hardware to standard Linux kernel Lowest priority is assigned to standard Linux kernel, which then runs as a independent task.

RTLinux executive is nonpreemptible Leaves Linux kernel essentially untouched so it doesnt hinder future Linux development

ECOS
Targeted at high-volume applications:

consumer electronics, telecommunications, automotive other deeply embedded applications. lets developer configure a system that best matches the needs of the application. Typical configuration options:

Configurable:

type of scheduler number of priority levels.

current release of the system has over 200 options

Portable Fully preemptable

Typical Application
Control system for the Canadian Light Source (CLS) uses an open source real-time operating system and control software:
RTOS RTEMS Control Software EPICS

(Experimental Physics and Industrial Control System)