Real-Time Simulation with Virtual Reality Visualization

Singgih S. Wibowo1 and Hari Muhammad2

1
Research Assistant
Department of Aeronautics and Astronautics
Insitut Teknologi Bandung
e-mail: singgih.wibowo@ae.itb.ac.id
2
Associate Professor
Department of Aeronautics and Astronautics
Insitut Teknologi Bandung
e-mail: harmad@ae.itb.ac.id

Abstract

Real-time simulation of any dynamic system is made possible today as there are many high speed computers with
high accuracies are available. Computer based simulation, also called digital simulation or numerical simulation, is
widely used in many applications for safety and economical reason. Digital simulation is also used for scientific
reason, i.e. solving complex mathematical model, which usually do not have analytical solution yet, by numerical
methods. Other advantage of using computer simulation is its capabilities to visualize the numerical results in a
graphical representation. The higher level of graphical visualization is called virtual reality. Virtual Reality (VR) has
advantage for visualizing the real motion of dynamic system. This paper discusses the real time simulation of a
dynamic system using virtual reality visualization. Some examples of the simulation that will be visualized in the VR
environment are given in this paper.

Keywords – real-time simulation, virtual reality, mathematical model

Nomenclature known as Virtual Reality Modeling Language (VRML),

and also known as Virtual Reality Modeling Markup
h
CVR Transformation matrix from local Language. The VRML is an ISO standard language
horizon axes to VR axes that is open source, text-based, and uses a Web-
H Altitude oriented format. One can use VRML to define a virtual
Vb Translational velocity in body axes world that can be displayed in a Web browser and
X b , Yb , Z b Body coordinate system connect to a Simulink model. The Virtual Reality
Toolbox uses many of the advanced features defined
X o , Yo , Z o VR coordinate system in the current VRML97 specification. The term VRML
X h , Yh , Z h Local-horizon coordinate system used here always refers to VRML as defined in the
α ,β Angle of attack, sideslip angle VRML97 standard ISO/IEC 14772-1:1997. This format
includes description of three-dimensional scenes,
ωb Angular velocity in body axes
sound, internal actions, and Web anchors [3].
ϕ , θ ,ψ Euler angles
VRML Coordinate System uses the right-handed
Cartesian coordinate system. It can be describe as
1 Introduction follows: held thumb, index finger, and middle finger of
the right hand so that they form three right angles, then
Virtual reality (VR) toolbox is a tool for visualizing the the index symbolizes the x-axis, the middle finger the
dynamics motion of a body in Three-Dimensional (3D) y-axis (pointing up), and the thumb the z-axis, see
space with adequate fidelity. In this paper, the toolbox Figure 1.
will be elaborated and used for visualize the WiSE craft
motion and robot motion.
Before elaborating the VR tools into detail, it is better to
understanding the basic concept of the VR environment.
The most important of the VR is its language which
 2 Principle and Procedure of using Virtual
Reality

The most important principle of VR visualization is that

the translational and rotational motion can be
visualized by inputting recent values of position and
attitude angles. These position and attitude must be
respected to VR axes system. Therefore it needs to
transform the body axes or local horizon axes system
which is used in the simulation into VR axes system,
using the following relation

Figure 1 Right-handed Cartesian coordinate system ⎧Xo ⎫ ⎧Xh ⎫

⎪ ⎪ h ⎪ ⎪
⎨ Yo ⎬ = CVR ⎨ Yh ⎬ (1)
⎪ ⎪ ⎪ ⎪
⎩ Zo ⎭ ⎩ Zh ⎭

h
where CVR is the transformation matrix from local
horizon axes system to VR axes system, having the
following value

⎡1 0 0 ⎤
h
CVR = ⎢⎢0 0 −1⎥⎥ (2)
⎢⎣0 1 0 ⎥⎦

Figure 2 The VRML (VR) coordinate axes definition Figure 3 shows the Simulink library browser of Virtual
Reality toolbox. It can be seen in this figure that the
toolbox contains six blocks: Joystick input, Magellan
The VRML coordinate system is differing from the space mouse, VR placeholder, VR signal expander,
MATLAB and Aerospace Blockset coordinate systems. VR sink and VR Source.
VRML uses the world coordinate system in which the y-
axis points upward and the z-axis places objects nearer
or farther from the front of the screen, see Figure 2. It is
important to realize this fact in situations involving the
interaction of these different coordinate systems.
Rotation angles -- In VRML, rotation angles are defined
using the right-handed rule. It can be explained as
follows: hold the right hand while the thumb points in the
direction of the axis toward its positive end. The four
remaining fingers point in a counterclockwise direction.
This counterclockwise direction is the positive rotation
angle of an object moving around that axis.
Child objects -- In the hierarchical structure of a VRML
file, the position and orientation of child objects are
specified relative to the parent object. The parent object
has its local coordinate space defined by its own
position and orientation. Moving the parent object also
moves the child objects relative to the parent object.
Measurement units -- All lengths and distances are
measured in meters, and all angles are measured in
radians.

Figure 3 Simulink library browser: VR Toolbox

Real-Time Simulation with Virtual Reality Visualization 2/7

The blocks which will be used in the simulation are the
VR Sink and Joystick Input. VR Sink is block having a
function of writing data from the Simulink model to the
virtual world, or in other word connecting Simulink
model to virtual world. The data is specified in block
parameter dialog box, see Figure 4. Joystick Input is a
block which processing input from an asynchronous
joystick device. This block provides a convenient and
real-time interaction between a Simulink model and the
virtual world.
Figure 4 VR parameter window
Virtual world can be built using V-Realm builder which is
available in MATLAB or other CAD software such as
AutoCAD and CATIA. The virtual world should be
defined in VR Sink parameter as source file. For
example, the source file is wise_CAT_C which describe
the virtual world for WiSE craft motion, see again Figure
4. This virtual world that has been defined in VR sink
will show VR windows as depicted in Figure 5.
Figure 5 Example of the VR window, viewed from right-
behind
3 Simulation Examples
In this section, two examples of simulation with VR
visualization will be presented. The first simulation is
WiSE craft motion, taken from [1] and [2], and the
second simulation is robot-arm motion.
Real-Time Simulation with Virtual Reality Visualization
3.1 Example 1: WiSE craft Motion
In this work, simulation of WiSE craft motion is carried
out using parameter shown in Table 1. Other
parameters such as aerodynamic and hydrodynamic
which also used are available in [1] and [2].
Table 1 Parameter used in WiSE craft simulation [2]
Parameter Value Unit
m 40 Kg
I yy 7.7440 Kg.m2
V0 0 m/s
α 0 ,θ0 5 deg
H0 0 m
Power Setting 100 %
δe -4 deg
3.1.1 Simulink Model of WiSE Motion
The complete Simulink model of WiSE craft motion is
presented in Figure 6. It consists of ten subsystems.
The subsystems are (1) 6-DOF Equations of Motion,
(2) Forces & Moments, (3) Atmosphere & Gravity, (4)
Control Surface, (5) Data Bus Selector, (6) VR
transform, (7) VR Sink, (8) Avionic Signal Conditioning,
(9) Avionic instruments and (10) Power Setting.
Subsystem (6) and (8) are very important for
visualizing purpose, and therefore will be elaborated in
the next section.
Figure 6 The Simulink model of WiSE craft Motion
Combined with VR and Avionics Instruments [1]
3.1.2 VR Transform and Avionic Signal
Conditioning
VR Transform subsystem transforms local horizon
axes system to the VR axes system, the so-called
VRML coordinate system. The block contains the
algorithm for transforming local horizon axes to VR
axes and calculation of viewpoint angle, see Figure 7.
The input for this block is motion bus, and the outputs
3/7
are position of the vehicle in VR axes systems, roll
angle, pitch angle, yaw angle and observer viewpoint
angles: Angle A (for first observer), Angle B (second
observer) and Angle C (third observer).

LH to VR matrix

[3x3] Matrix
1
Multiply
<signal2> X Y Z [m] (0 1 0)
1 200 5
Motion Bus atan2 Angle A [rad]
<signal3> Xo [m]
25
(1 0 0) Zo [m]
2
roll angle [rad] (0 1 0)

600 6
atan2 Angle B [rad]
Xo [m]1
(0 0 1) 25
3 Zo [m]1
pitch angle [rad]
(0 1 0)

[0 -1 0] 1100 7
4 atan2 Angle C [rad]
Xo [m]2
yaw angle [rad]
25
Zo [m]2 Figure 9 Hardware platform for simulation with
Joystick Device
Figure 7 The VR Transform subsystem
3.2.1 Simulink Model of Robot-arm Motion
Avionic signal conditioning subsystem provides data Robot-arm Simulink model as presented in Figure 6
that is needed by avionic instruments. The inputs for consists of four subsystems. The subsystems are (1)
Joystick Input, (2) Equation of Motion, (3) VR
this block are translational velocity Vb , angular velocity
Transformer, (4) VR Sink. The virtual world for robot-
ωb , Euler angles ( ϕ , θ , ψ ), altitude H and vertical arm is shown in Figure 11.
velocity Vz . The outputs of the subsystem are airspeed
in knots, roll and flight path angles, sideslip angle and HArm_VPos (m)
?
Arm_Pos (m) Arm_Trans (m)

HArm_Rots (rad)
?

?
3
Hig_Arm_Trans.translation

Hig_Arm_Rots.rotation
HArm_Rot (rad) Hig_Arm_Rot (rad) 4

yaw rate, heading, altitude in meter, and vertical velocity Axes Axes_Input (norm)

Joint_Rot (rad)
?

?
Joint_Rot (rad)
Joint_Rots (rad)

LArm_Rots (rad)
?

?
4
Joints_Rots.rotation

Low_Arm_Rots.rotation

(rate of climb) in feet per minute. Figure 8 shows the LArm_Rot (rad)
?
Low_Arm_Rot (rad)
LFinger_Rots (rad)
?

?
4

4
Finger_Rots_L.rotation

Buttons
joyinput Buttons_Input (logic) Finger_Rot (rad) Finger_Rot (rad) RFinger_Rots (rad) Finger_Rots_R.rotation
content of avionic signal conditioning subsystem. VArm_Scale
?
VArm_Scale
VArm_Scale (norm)
3
4
Ver_Arm.scale

Equation of Motion VR Transformer VR Sink

Point of View
m/s to kts

1.943844
Airspeed_kts Joystick Input

α
asin
gamma
Mux 57.3
Roll_Fltpath_deg Figure 10 The Simulink model of Robot-arm Motion
rad to deg1
1 V
b β Combined with Joystick Input
u,v,w Mux -K-
V Sideslip_Y awrate_deg
2
rad to deg
5 -1 p,q,r 1
Vz Avionics
57.3 Bus
Heading
rad to deg2
3
phi, theta, psi
4 1000
Altitude
H
m to mm

1.9685
Vertical_Rate_f pm

m/s to 100ft/min

Figure 8 The avionic signal conditioning subsystem

3.2 Example 2: Robot-arm Motion

This example will show the advantage of VR for
visualizing robot-arm motion. The motion will be Figure 11 Robot-arm virtual world
commanded using external joystick to give illustration of An important subsystem in robot-arm simulation is
real-time simulation. The hardware platform of this Joystick input or subsystem (1). This subsystem
simulation process is shown in Figure 9. connects joystick device to Simulink. Joystick input will
be elaborated in the following section.

3.2.2 Joystick Input Subsystem

As has been mentioned before, the joystick input
subsystem connects the joystick device to Simulink.
This connection is real-time which mean that the signal
is inputting directly into simulation without delay. This

Real-Time Simulation with Virtual Reality Visualization 4/7

subsystem is depicted in Figure 12. It can be seen that 20
Incidence and Attitude

this subsystem produces three outputs: Axes, Buttons Alpha

Theta
and Point of View. 15

10

α,θ (deg)
0
Axes
-5

-10

-15
joyinput Buttons
-20
0 1 2 3 4 5 6 7 8 9 10 11
Time (s)

Point of View Figure 14 Incidence and attitude angle history [2]

Joystick Input Velocity

30

Figure 12 Joystick Input Subsystem

25

The joystick input subsystem will identify the joystick

device automatically, i.e. it will produce axes output, 20

buttons output and point of view output automatically

V (m/s)

depends on the device. 15

10

4 Simulation Results and Discussions

5

The following figures show the simulation results of

Example 1 and Example 2. Time history of some 0
0 2 4 6 8 10 12
parameters is shown in Figure 13 trough Figure 15, and Time (s)

the corresponding VR visualization is presented in

Figure 16, Figure 17 and Figure 18. Figure 15 Velocity history [2]

Motion Path
6
5
4
H (m)

3
2
1
0
-1
0 25 50 75 100 125 150 175 200 225
X (m)
Horizontal Path History
250
200
X (m)

150
100
50
0
0 1 2 3 4 5 6 7 8 9 10 11 12
Time (s)
Vertical Path History
6
5
4
H (m)

3
2
1
0
-1
0 1 2 3 4 5 6 7 8 9 10 11 12
Time (s)
Figure 16 VR visualization of WiSE motion, viewed
Figure 13 Motion path history [2] from right observer

Real-Time Simulation with Virtual Reality Visualization 5/7

 0

Larm Rot (rad)

-10

-20
0 2 4 6 8 10 12 14 16 18 20
time (s)
1

Finger Rot (rad)

0.5

0
0 2 4 6 8 10 12 14 16 18 20
time (s)

Varm Scale (normalize)

2

0
0 2 4 6 8 10 12 14 16 18 20
time (s)

Figure 17 VR visualization of WiSE motion, viewed Figure 20 Time history of Low-arm and Finger rotation
from pilot observer

Figure 18 VR visualization of WiSE motion, viewed

from right-front observer
Figure 21 Robot-arm during simulation process,
By comparing Figure 13 trough Figure 18, the action one
visualization method using VR is more advantage then
x-y graphic since it inform the real situation of the
motion. The following figures show the simulation
results of robot-arm motion. Figure 19 and Figure 20
show the time-history of motion while Figure 21 and
Figure 22 show the corresponding VR visualization.

0.5
Harm Vpos (m)

-0.5
0 2 4 6 8 10 12 14 16 18 20
time (s)
4
Harm Rot (rad)

0
0 2 4 6 8 10
time (s)
12 14 16 18 20
Figure 22 Robot-arm during simulation process,
2 action two
Joint Rot (rad)

-2
From above examples, it can be seen that visualization
-4
using VR give better understanding of motion. One
0 2 4 6 8 10 12 14 16 18 20
time (s) may not able to imagine the motion using parameter
plot, for example is Figure 19, but using VR the motion
Figure 19 Time history of High-arm and Joint rotation can be seen directly and can be easily understood.

Real-Time Simulation with Virtual Reality Visualization 6/7

 5 Co
oncluding Re
emarks

The real-tim
me virtual rea ality simulatio
on is presentted. Two
examples have
h been shown in thiss paper. The e results
give the advantage of using virtual rea ality for
visualization
n purpose, i.e. giving real sense e of the
motion thatt help the user undersstanding the e motion
easily. Thiss kind of visualization
v can be used
u for
educational purpose an nd design forr any dynam mic plant,
such as airccraft, roboticss and land veehicles.

References
s

[1] S. S. Wibowo,
W Virrtual Reality of Wing in Surface
Effect Craft, Masster Thesiss, Aeronautiics and
Astronautics Depa artment, Ba andung Insttitute of
Technoology, 2004.
[2] S. S. Wibowo,
W Haari Muhamma ad and S. D.D Jenie,
Simula
ation with Virt
rtual Reality Visualization
V n of Wing
in Surf
rface Effect Craft during g Takeoff Ma aneuver,
Asian Control
C Confference-2006 6, 2006.
[3] The MathWorks Inc, MA
ATLAB 7.0
0 Help
mentation, 20
Docum 004.

Singgiih Satrio Wiibowo is a teaching

t
and rresearch (T T&R) assisstant in
Departtment of Aeronauticcs and
Astrona autics ITB
B. He received
Bachellor of Eng gineering fro
om the
departmment in 20 002 and Ma aster of
Engine eering in 20 006. He is currently
c
particip
pating in somme research projects
h as WiSE, UAV
in ITB, such U and rockketry.

Hari MMuhammad is a chair of flight

physiccs group in Departm ment of
Aerona autics and Astronautics
A ITB. He
receiveed PhD from m Delft Univversity of
Technnology (TU-D Delft) in 1994. Dr Hari
Muham mmad currently become e a vice
dean of Facu ulty of Industrial
Technnology ITB. He has a lot of
industrial exxperiences especially
e in
n aircraft dessign and
flight test. He was pa articipating in
i N235 an nd N250
aircraft dessign and the eir flight tes
st program. Dr Hari
Muhammad d also partticipating in ntensively in n some
research prrojects, for eexample in WiSE-8
W crafft design
as ITB team m leader.

Real-Time Simulation
S w
with Virtual Re
eality Visualiization 7/7