2006:307 CIV

MASTER'S THESIS

Design and Implementation of a

Shared Whiteboard Application
for Industrial Environments

Daniel Lindstrm
Daniel Nilsson

Lule University of Technology

MSc Programmes in Engineering
Computer Science and Engineering
Department of Computer Science and Electrical Engineering
Division of Media Technology
2006:307 CIV - ISSN: 1402-1617 - ISRN: LTU-EX--06/307--SE

Preface
This Masters Thesis was written during 2006 at Lule
a University of Technology
as a part of the Master of Science program in Computer Science and Engineering.
The thesis work is a result of a collaboration with the Media Technology research
group at the university, which is currently working in the TI/C4 project. TI/C4
is a EU founded project which goal is to develop new concepts and methods for
Collaborative Working Environments, that is tools, technologies, services and
environments that helps individuals directly or indirectly with the interaction,
making them more productive. The university is collaborating with SSAB in
this project. This gave us the opportunity to work close with real users in their
working environment and evaluate any possibilities to develop something for the
project.
To start with we want to thank Johan Kristiansson, our supervisor, for
making this thesis possible thanks to his and the Media Technology Research
teams contact with SSAB. We also want to thank Robert Johansson and the
machine keepers from SSABs steel mill in Lule
a that gave us their help and
support to do a survey about the collaboration needed in challenging contexts.
Last, but not least, we want to thank the people we have troubled with our
tests.
Daniel Lindstrom
Daniel Nilsson

Abstract
Handheld mobile devices, such as smartphones and PDAs are becoming more
and more attractive for workers in many different environments. In some environments they are still relatively uncommon even though they could be advantageous for collaborating and problem solving. As the devices becomes better,
more possibilities arise to use them in different kinds of environments.
This thesis presents the design and development of a prototype for a collaboration tool for industrial environments. A collaboration tool is a tool that helps
the user with direct or indirect interaction, with for example a co-worker or an
expert. The prototype is a whiteboard like application that aims to improve
the interaction for machine keepers in the rough environment of SSABs steel
mill. The focus of the thesis is on what software components are needed and
how they should be designed.
This report describes how the workers at SSAB were observed, and how the
requirements on the hardware and software were set with help of interviews and
evaluations. It also goes into the different aspects of designing the application
and how to implement the various parts and components. The report shows
that the components and design presented is sufficient, but in some cases not
the best choice.

ii

I not only use all of the brains I have, but all I can borrow.
Wilson, Woodrow 28th President of the United States (1856-1924)

iii

Contents
1 Introduction
1.1 Background and Purpose
1.2 Problem Statement . . . .
1.3 Related Work . . . . . . .
1.4 Methodology . . . . . . .

.
.
.
.

1
1
1
2
2

2 Requirements
2.1 Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . .
2.2 Software Requirements . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Required Components . . . . . . . . . . . . . . . . . . . . . . . .

4
4
5
5

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

3 Theory
3.1 Networking Model . . . . . . . .
3.2 Client to Client Communication .
3.3 Finding a User . . . . . . . . . .
3.4 Synchronization . . . . . . . . . .
3.5 Vector Graphics . . . . . . . . . .
3.6 Zoom and Pan . . . . . . . . . .
3.7 Make Use of the Screen . . . . .

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

4 Hardware

7
7
8
9
10
11
11
12
14

5 Implementation
5.1 User Interface for the Whiteboard
5.2 Graphics Class Design . . . . . . .
5.3 Network Interface . . . . . . . . . .
5.4 User Database . . . . . . . . . . .
5.5 Selecting a User . . . . . . . . . . .
5.6 Synchronizing Objects . . . . . . .
5.7 Server . . . . . . . . . . . . . . . .
5.8 PC Client . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

17
17
18
19
19
20
21
21
22

6 Evaluation
6.1 Mock-up Tests . . . . . .
6.2 Single User Interface Test
6.3 Final Evaluation . . . . .
6.4 Hardware Evaluation . . .

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

23
23
25
25
26

.
.
.
.

.
.
.
.

.
.
.
.

iv

.
.
.
.

.
.
.
.

7 Discussion
27
7.1 Alternative Solutions . . . . . . . . . . . . . . . . . . . . . . . . . 27
7.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
A Protocol Specification

32

B User Guide

38

Chapter 1

Introduction
This thesis is about creating a shared whiteboard application for handheld devices in industrial environments. This first chapter introduces the background
of the work and presents the problem. It also contains a survey of related work
and a description of the methodology.

1.1

Background and Purpose

In an industrial environment, a worker sometimes has to take advice from an

expert when solving a problem. That expert may not always be able to physically come to the location where the worker is present and consequently needs to
support the worker by phone or through other communication services. Verbally
describing the problem may sometimes be too difficult. One solution to enhance
this is giving the workers the possibility to send pictures or video streams when
interacting with the expert, for example point at things of interest and draw
simple sketches. Enabling workers to collaborate better in this way is possible
since many of todays handheld devices, such as smartphones and PDAs, have
larger graphical displays and decent cameras. A whiteboard application for
such devices could help the workers in their everyday work. This master thesis
is about the software components needed to implement such an application.

1.2

Problem Statement

The research of this thesis focuses on the software components needed to implement a shared whiteboard for handheld devices. More specifically on the
following points:
What software components are needed.
How should each component be designed.
What requirements are set by an industrial environment.
Because the focus of this thesis is on the software components, some of the hardware requirements regarding the durability are ignored. The report will though
briefly touch the subject and discuss the implication the environment have on

the hardware. Furthermore, this thesis does only focus on the whiteboard part
of this collaboration solution and therefore no time or effort will be put on finding a solution for the voice communication. To make the development of the
prototype easier it is assume that there always is full network connectivity. In
real life this is of course not the fact, but this is also left outside the scope of
this thesis.

1.3

Related Work

Collaboration tools are not new, there are several existing systems. Two examples are Microsoft NetMeeting [1] and Marratech [2], which both are video
conferencing applications with functions such as shared whiteboard, video and
audio stream and chat. Most such systems, including NetMeeting and Marratech, are designed for stationary computers and lack support for handheld
clients. There is though research going on for creating a Marratech client for
mobile devices, called Pocket Pro [3].
There are though some collaboration tools for handhelds too. Belov et al. [4]
describes a collaboration system for smartphones called WICS that provides a
whiteboard like interface. WICS is designed for mathematics communication
and thus provides functions for entering formulas.
Another example is Amigo that is an Instant Messaging (IM) client, developed by Fabersjo et al. [5]. The application is made to run on a handheld
computer and what makes it different from other IM clients is that the messages
are based on images drawn with the touchscreen. Like other IM clients, amigo
is not real-time.
W. Geyer et al. [6] presents a tool for teleteaching called digital lecture board.
It is a whiteboard application that they developed to replace the teleteaching
applications at hand because of their shortcomings. W. Geyer et al. discloses
the features they had in their mind while designing the digital lecture board.
Ericsson has developed a service called IMS WeShare [7]. IMS stands for IP
Multimedia Subsystem and is an international recognized standard that specifies
interoperability and roaming for telecom operators. IMS WeShare uses this
standard, and enables a user to share different kind of media such as video,
images and files. It also supports a shared whiteboard solution that works
both with a blank background or an image that both users can draw on. IMS
WeShare currently works only with WCMDA, but will in the future also works
with GPRS/EDGE.

1.4

Methodology

The work of this thesis was started with a visit at SSAB. There an ethnographic
study was made where two machine keepers were followed at their work. During
this time information about their ways to communicate and collaborate was
gathered by interviews and observations. This method is a major research
strategy called participant observation [8]. From the information gathered at
SSAB a set of both hardware and software requirements were formed.
The next step was to conduct a prestudy where related work was searched
for to get an understanding of both the problem and how to best solve it. This

includes different aspects of developing a collaborative tool and a list of possible

solutions. The prestudy also surveys suitable wireless devices that could be used
to realize a proof of concept prototype.
For the implementation and design of the application, the software prototyping method [9] was used. The first prototype was a series of paper sketches.
Two versions of the interface were made, which were then compared to each
other. No evaluation with external test users were performed. With the help
of the results from the paper mock-up a graphical mock-up was made to more
easily illustrate what the GUI can look like. To do this a picture of a smartphone was taken and then GIMP [10] was used to draw the prototype of the
application on its screen. This way the prototype looks nicer and can more
easily be evaluated by test users, thus getting some useful ideas and feedback.
The third prototype was a single user application and with each new prototype
after that more feature were added until all requirements were met.
Each prototype were evaluated and the experience from it was used when
designing and implementing the next. Some prototypes were just discussed
internally while others were tested on users. For a overview of the prototypes,
see Table 1.1.
Prototype 1

One paper mock-up each which are compared to each

other and the best parts are used for the next prototype.

Prototype 2

A graphical mock-up created with a drawing program.

This one is evaluated with a few test users.

Prototype 3

An application that can take a picture and let the user

draw lines and point on it. No networking.

Prototype 4

Implementation of the server and a client that can connect to the server and make peer-to-peer connections,
but no synchronization of drawing objects.

Prototype 5

Like the previous but with synchronization of drawing

and pointing. It includes all the required features. The
final prototype is evaluated by user tests including users
from SSAB.

Table 1.1: A brief description of each prototype

Chapter 2

Requirements
An industrial environment such as SSABs steel mill puts lots of demands on
both hardware and software. To start with, the hardware that is going to be used
must fulfill certain requirements when it comes to robustness and usability. In
addition, lots of demands are put on the software regarding usability so that all
different kinds of users can easily understand and efficiently use the application.
The first two sections below presents and discusses these hardware and software requirements. The last section is about interface mock-ups and how some
of the functionality of the application are derived.

2.1

Hardware Requirements

Because of the type of environment the application should be used in, not just
any device can be used. To endure a rough industrial environment it needs
to be robust enough to survive dust, humidity, shock and large temperature
differences. For an environment such as a steel mill it is important to choose a
device with an appropriate IP-class [11]. Further, the device needs to be small
enough to be carried in a pocket so that the users do not loose mobility or feels
that it is in the way. Some kind of wireless connectivity must also be supported
in order to allow remote collaboration. Finally, a camera on the device is also
necessary if the user wants to take a picture of the problem at hand.
The way the users will interact with the device also influences the choice
of hardware. In rough environments it is very likely that the users want to
wear gloves, and pressing small buttons with gloves is not very practical. The
users need another way of giving input to the application. A touchscreen with
a pen is one solution that lets the user control the application with gloves on.
A touchscreen is also very common on many handheld devices today.
To be able to create an application on the device within the time frame of
a master thesis project there is one last requirement, namely the programming
possibilities. There must be tools and APIs available for developing the program,
including access to the camera and the wireless network.

2.2

Software Requirements

The industrial environment do not only put demands on the hardware, but also
on the software. Since using the buttons is unpractical, they should be avoided
and the touch screen could be used instead. Also, for fast and efficient work,
it is important to minimize the number of actions needed to do any operation,
e.g. only a couple taps on the screen should be enough to call and interact with
another user.
Since most handheld devices have got a relatively small screen resolution
(about 240x320 pixels), it needs to be used as efficiently as possible. The user
should be able to use the full screen size.

2.3

Required Components

The most obvious part in a shared whiteboard application is the whiteboard

surface. That is the surface that is used for communication by drawing and by
visualizing what other has drawn. Just like in any painting program different
kind of tools are needed for creating objects on the surface. Examples of these
tools may be a pen tool or a pointing tool. A user needs an easy and fast way
to switch between the different tools to be able to work as efficient as possible.
For this, some kind of menu could be used. For the whiteboard application to
fulfill its purpose a way to take a picture and show it on the whiteboard surface
is needed.
For any remote collaboration tool some kind of networking module is needed
and a whiteboard application is not an exception. A client must be able to
interact with another client via the whiteboard surface. This can be done either
with direct connection or via a server. For connecting to the other client an
interface is needed. With that a user can choose whom to connect to. See
table 2.1 for a summary of requirements for the whiteboard application.

Telepointer
Drawing
Zoom/pan

Erase all
Take picture
Show picture

Invite user
Buddy list
Status
Search/list users

Push document
Synchronization

Whiteboard
The user should have a way to put focus on a detail
in the image.
Some sort of drawing tool is necessary to mark details
and draw simple figures.
If the image used is larger then the screen, there
must be an easy way to zoom and pan to the area of
interest.
There should be a way to reset the session and delete
all objects drawn so far.
Some kind of camera dialog is necessary so that a
picture can be taken with the camera.
There has to be one way to render a picture, so it
can be shown to the other user.
User Management
The interface is going to need a function to find and
select someone to connect to.
A favorite list helps the user to find other users that
he often connects to.
A users status helps other users to see whether he
is offline, online or away/busy.
To be able to add a user to the favorite list there
must be a way to first find that user.
Network
The images needs to be sent over the network, so
there must be a function to push documents.
Any two clients in the same session must have their
drawn objects synchronized so that the users see the
same.
Table 2.1: Software requirements

Chapter 3

Theory
This chapter is about different aspects of designing the collaborative application.
Both system architecture, such as networking model, and user interface design
are discussed. The first two sections below discuss different network solutions
and Section 3.3 describes the interface for selecting a user to connect to. Section
3.4 is about whiteboard synchronization and after that the rest of the chapter
describes different aspects of the whiteboard user interface.

3.1

Networking Model

The networking needed for a collaborative application is in two parts. The first
part is an infrastructure for locating other users and establishing new sessions
with them. The second part is the network communication needed to synchronize the state of the clients in a session. The choices to be made for one part
is somewhat orthogonal to the choices to be made for the other. E.g. using a
server to locate users does not mean that the server has to be used in synchronization. The rest of this section is about how to locate other clients to connect
to and the next section is about the networking needed for synchronization.
For a client to connect to another client it first needs a way to find other
active clients. Using a central server that maintains a database of all clients
is one option (similar to ICQ [12]). Another option is to have peer-to-peer
connections that builds an overlay network and to use broadcasts in that to find
other online clients (similar to Waste [13]). Both solutions have their advantages
and disadvantages.
A central server always have a static address and works as a rendez-vous
point that clients connects to. This minimizes the configuration effort required
by the users. A server also makes it easier to administrate users and groups.
With a peer-to-peer solution no server at all is needed since user identity and
group membership is set up on the clients. This of course puts more load on
the clients, and more effort on the users. However, one advantage with overlay
networks is that there is no single point of failure. Even if one client fails, all
the others will still be able to work properly. If the server on the other hand
stops working, clients will not be able to find each other. However, there exists
solutions to make a server more robust and fail-safe, but that is not the primary
focus of this thesis.

3.2

Client to Client Communication

When the clients have initiated a session, they needs to synchronize the state
of the whiteboard. This can be done in two ways, either through a server
or directly with a peer-to-peer connection. Both of these solutions have their
advantages and disadvantages.
When it comes to network traffic the server solution have a drawback since
all the clients needs to send information via the server. Just small amounts of
packets might not be a big problem, but when initiating the session an image
needs to be sent. With many clients doing this at the same time the server might
become overloaded and become a bottleneck restraining the communication. A
peer-to-peer solution does obviously not have the same problem with network
traffic. Also, the response time for a peer-to-peer model is in most cases lower
since the traffic only needs to go from user A to user B, and not via the server.
However, if both clients are behind firewalls direct connections between them
is not always possible. They can usually make outgoing connections though,
so they can still communicate with each other via the server. The centralized
system has a price, the whole system is dependent on that the server is working.
A peer-to-peer solution, on the other hand, does not have a single point of failure,
so if any peer fails everything else will still work.
Setting up a new peer-to-peer connection is not so easy. First both parties
must agree to it. Then one of them must listen for a connection and the other
get the IP-address and port and make the connection. Figure 3.1 shows how
establishing a peer-to-peer connection works with the help of a central server.

User A
1
3

User B

Figure 3.1: Establishing a peer-to-peer connection. All clients (both handheld

and PCs) are connected to a central server that keeps track of them. When
user A wants to connect to user B, As client sends a connection request to the
server (1). The server then sends As address to Bs client (2). After user B has
accepted the connection request his client opens the connection to user A (3).
Another thing to consider is the number of users that can participate in a
session. The client/server model makes supporting more than two clients a lot
easier. Because they all synchronize against the server, two or more participants
is no different from the clients point of view. With the peer-to-peer model the
clients must all keep their state consistent, which can be difficult. It becomes
even harder with more than two clients. This is discussed more throughly in
Section 3.4.

3.3

Finding a User

The previous sections have discussed the networking from a system point of
view, this section will take the users view. For the user the underlying network
model is irrelevant as long as it is possible to find and connect to other users. To
facilitate this some kind of user interface is needed for displaying the available
users. One that is common and people are familiar with is to prefer. Since
instant messaging programs are so widespread and popular, a buddy list like in
those programs (see Figure 3.2) might be a good choice for connecting to other
clients. However, a problem arise when the list of users is longer than what
fits on screen. Scrolling through the list to find the target user can be tedious.
By arranging the users into groups the work is reduced because only the users
belonging to the targets group needs to be searched though. This arrangement
into groups can be presented to the users in different ways. One is to have a
tree view were groups are collapsible/expandable nodes and users are the leaves.
Another way is to have a separate list of groups and filter the user list when a
group is selected so it only shows members of that group. The advantage with
the first approach is that it is a common design in many applications and most
users know how it works. The second approach has the advantage that users can
belong to many groups and several groups can be selected to show only users
in all those groups. This is useful for example to find a mechanic in Lule
a; just
select the groups mechanic and Lule
a. Such selections would be hard to
accomplish in the tree view without an unnecessary large hierarchy were groups
and users are repeated.

Figure 3.2: Miranda is a instant messaging program with a typical buddy list.
Nevertheless, a problem can still arise when the user wants to find a friend
and both the list of users and the list of groups are long. Even though selecting
a group makes the user list shorter, the user must still search through a long list
of groups. The solution is to have a list of favorite users and a list of favorite
groups and let the user decide if only the favorites should be shown. This way
the user can switch between four states:
1. All users, all groups. Can be useful when the user needs to find someone
not normally connected to.
2. All users, favorite groups. A good mode when connecting to someone in
a certain group but when the exact user does not matter.
9

3. Favorite users, all groups. Useful when there is a small number of users
that are often connected to.
4. Favorite users, favorite filters. Useful when the number of friends is larger
and filtering on groups is needed.
Another way to find a user to connect to is to let the application choose
for you. An interesting way of doing this is to make the application contextaware, that is understanding the environment around the user like Dey et al. [14]
explains. By telling the position of the user with the help of some positioning
algorithms or sensors, the application can from that information select and
connect to a user. For example if user A stands beside a fusebox, the application
automatically finds an electrician. Another use for context-awareness is to let
the groups be dynamic, and let the application put the users in the correct
group depending on location. For example the application can place everyone
that is currently in Lule
a in the Lule
a group.
This concludes the section about the interface for user selection. When the
user has selected the one to connect to the application starts the collaboration
session. This leads to the topic of the next section.

3.4

Synchronization

When a client is connected to another client, the application continuously needs

to synchronize its state. Dewan [15] describes how this can be done in different
layers in the program as seen in Figure 3.3. Doing it in the lowest layer, the
windowing system, would require support from the OS and would result in much
traffic since every single movement of the pen is transmitted. It would also make
communication between different platforms impossible. The advantage is that
no changes needs to be done to a single user program since it will receive the
input from both users in the same way as from a single user.
Model

Model

Model

Model

View

View

View

View

Widget

Widget

Application
Program

Widget

Widget

Window

Window

Operating
System

Window

Window

User A

User B

User A

User B

Figure 3.3: Synchronizing in different layers of the application. On the left:

Synchronizing in the windowing layer requiring OS support. On the right:
Synchronizing in the model layer.
Doing the synchronization at a higher level requires modifying a single user
program so that it is aware of the the collaboration. Network load is reduced
10

because many actions from the user can be collected into just one modification
of an object, e.g. one point is added to a polyline. Having the synchronization
in a higher level also has the benefit of allowing the users to have different views,
i.e. different zoom and/or screen size.
A problem when synchronizing is to have a global ordering of all events. One
way is to have one application that is responsible for collecting all events before
sending them to the other applications. This requires something like a central
server and will introduce delays in the feedback to the users. Another solution
is to sidestep the problem by having all events commutable so their order does
not matter for the final result. This is easy if the users are only allowed to draw
new objects and never to manipulate existing ones. For manipulating objects
some kind of locking must be implemented to prohibit concurrent modification
of the same object by different users. This can be done explicitly by each object
having an owner that is the only one who can edit it. Users than first need to
request and get the ownership before manipulating the object. Newman-Wolfe
et al. [16] describes a system using implicit locking; when a user selects an object
exclusive rights to it are given and upon deselection those rights are lost.
This section has discussed how to synchronize graphical objects in a collaborative application. Nothing has been said about how these objects are
represented. The next section gives a brief introduction to the concept of vector
graphics.

3.5

Vector Graphics

Vector graphics is a way to represent geometrical objects. It does so by storing

a description of the object, such as the position and radius of a circle. This
is more compact than raster graphics that stores a picture of the object as an
array of pixels. Rosenbaum et al. [17] shows that vector graphics contains less
data and has faster load time. This makes vector graphics a better choice for
a collaborative application since less data is an advantage when transmitting
objects over the network. In addition, vector graphics can be shown in any
scale without the aliasing that occurs when scaling a raster image. There are
disadvantages too. Not all geometrical objects can easily be represented as
vector graphics, photographs is a good example of this. Furthermore, vector
graphics must be rendered, that is transformed to raster graphics when displayed
on screen. This is confirmed by Rosenbaum et al. that shows that raster
graphics are in most cases faster then vector graphics to draw on a mobile
device.

3.6

Zoom and Pan

To be able to see details in the picture the user must be able to zoom in and out
and pan around. A zoom tool can work in a number of different ways. One is to
zoom each time the user clicks, which is the way it works in many applications.
It requires one tool to zoom in and another to zoom out and only allows zooming
in predetermined discrete steps. Another way is to let the user point and drag
a rectangle and then zoom to that area. This allows arbitrary zoom levels but
how to zoom out is not obvious as the user can not make a rectangle larger than

11

the screen. A third option is have the user point and than drag in one direction
to zoom in and in another to zoom out. This has the benefit of arbitrary zoom
with continuous feedback. Drawbacks can be that users are not familiar with
this way of zooming and that the repeated redrawing needed for the feedback
is to demanding for the hardware.
There are also a few ways to make a pan tool. They can be split into two
groups, absolute and relative. Absolute pan tools are those where the user
directly controls the distance of the scrolling while relative are those where the
user controls the speed. The former offers better precision and the latter is
better for scrolling long distances. One often used method for absolute panning
is when the user points on the screen and then drags the image. An example of
relative panning is the middle mouse button scrolling available in some Windows
applications.
In a collaborative environment the users which are interacting are often
interested in the same part of the image at the same time. In this case it is
unnecessary to require both users to independently zoom and pan. One solution
to this problem is to link the views so that when one user zooms or pans the
other users view follows. This is often called WYSIWIS (What You See Is
What I See).
If one user has a much larger screen than the other, and thus has no need
to zoom himself, another solution can be utilized: A rectangular box on the
larger screen shows what the user with the smaller screen can see. The user can
then move and resize the box to update the view for other user. The other way
around, if the user with the small screen zooms or pans the rectangle is updated
to show what he is looking at.
The methods described above that only requires one of the users to zoom
and pan is an improvement but what if manual zoom and pan could be totally
eliminated? This would require some sort of automatic zoom/pan function in
the application. This function needs to understand what part of the image the
users are interested in by observing their gestures. It can be implemented as
follows: When a user starts to point or draw an interaction period is started.
The period ends when there has been no activity for a certain amount of time
(a few seconds). When an interaction period ends a bounding box of the area
used during that period is calculated. The application then zooms and pans to
that area during the time the users are inactive. When zoomed in a user can
click and hold the pointer at the edge of the screen to simultaneously zoom out
and pan in that direction.

3.7

Make Use of the Screen

Since the screen on a handheld device is so small it must be used efficiently.

This has implications for the way the user selects the active tool (line, pointer,
zoom, etc.). This selection can be done with either a menubar, a popup menu
or a toolbar.
A popup menu saves space, as it is only visible when the user enables it. The
problem is how to do that. On a PC a popup menu is often opened by right
clicking, but that is not possible on a touchscreen. It can instead be done by
holding the pen against the touchscreen for a short time. The problem with this
is that some tools requires that the pen is held against the screen and dragged,

12

and this could cause confusion. Also, since it takes some time for the popup
menu to appear, it lowers the speed and efficiency of the worker.
A menubar does not save that much space, but it is pretty easy to use
and understand. There are two drawbacks though. First, the user have to
read and figure out in what menu to look for an item. Second, two clicks are
always minimum to select a menu item. Both drawbacks make selecting a tool
more time consuming. Furthermore, the menubar as well as the popup menu
described earlier have the disadvantage that the user does not see what tool is
selected until he takes a look in the menu.
A compact toolbar at the top or bottom of the screen does not save space
compared to a normal menu, but it do save plenty of time for the user. With
just a single click, a user can select any item from the bar, and at any time he
can get feedback about what item is selected. The only drawback is that the
user needs to learn what each icon on the toolbar does.
A final point when it comes to using the screen to its full extent is that the
title bar should be hidden. This can save up to about 10% of the screen size.
This extra space can then be used for the whiteboard drawing area.

13

Chapter 4

Hardware
To develop the prototype a suitable device is needed. Since the prototype should
not be used except for usability tests and such, the robustness requirements
were not considered. The only thing looked at was the screen size, camera and
programming possibilities. This was because the focus of the thesis is on creating
the application, and therefore the device was chosen based the properties needed
for that. The devices looked upon were Qtek 9100, HP iPAQ, Symbol PPT8800,
Sony Ericsson S700i, Nokia N70 and Handheld Scandinavia M3. Below follow
a brief description on the devices.

Qtek 9100 is a smartphone with Windows Mobile 5.0 and

provides a functional development environment. It also features a 1.3 megapixel camera and a relatively large touchscreen. Unfortunately it does not meet the robustness requirements.

HP iPAQ are a series of PDAs that also run Windows

Mobile 5.0, but they do not have a camera. They are also
not suitable for industrial environments.

14

Symbol PPT8800 is a robust PDA that endures extreme

temperatures and multiple drops to concrete. It also complies to the IP54 standard for protection against dust and
water. It runs Windows CE .NET 4.1 and does not have a
camera.

Sony Ericsson S700i is a smartphone with Symbian OS.

It has a 1.3 megapixel camera. It does not have a touchscreen nor WLAN and is not designed for industrial environments.

Nokia N70 is also a smartphone with Symbian OS and is

very much like the S700i but with a 2.0 megapixel camera.

Handheld Scandinavia M3 is a smartphone designed

for though environments. It conforms to the IP64 standard
and endures extreme temperatures and multiple drops to
concrete. It runs Windows CE .NET 4.2 and has a 0.3
megapixel camera.

Because the authors are familiar with Windows programming environment

and developing with Symbian would take longer time Sony Ericsson and Nokia
phones were deliberately not chosen. The application could probably have been
developed for either OS so the authors choose the one they were most familiar
with. Because of the requirement to be able to use gloves the devices without
touchscreens are not a choice. A camera is essential for the application to be
15

useful. For the devices without an internal one there may be an external camera
available as an accessory. For this prototype though it is more convenient with
a built in camera.
This leaves us with two devices left, the Qtek 9100 and the M3. The Qtek
has a better camera and a new OS with better API, which would make the
implementation easier. The M3 on the other hand is made for rougher environments and has a slightly larger screen. The M3 will later come in a new version
with Windows Mobile 5.0, so even if the application is developed for the Qtek
it will also work on the M3. Therefore it was decided that the Qtek 9100 will
be used for the development of the prototype.

16

Chapter 5

Implementation
This chapter is about how the various parts of the application were implemented.
It also brings up some design and layout matters. A less advanced desktop-based
prototype was also implemented and will be described in the last section of this
chapter. The source code is released under the GNU General Public License
and can be downloaded from the SWAIN project on SourceForge.net [18].

5.1

User Interface for the Whiteboard

The whiteboard interface can be divided into two parts. The first is the whiteboard surface where all objects are shown, and the second is the toolbar where
the user can select which tool to use. The whiteboard surface takes up the whole
screen size, except for a small row in the bottom where the toolbar is placed,
as can be seen in Figure 5.1. The toolbar consist of a couple of icons used to
select different tools and operations, and the bar, to create contrast between
the background and the icons. When a user clicks on an icon it gets highlighted
with a white box around it. That way the user can always get feedback about
which tool is currently selected. The main reasons for choosing a toolbar instead
of a menubar or popup menu is to get feedback from the highlighting function,
and also the time saved for the users from the fast selections of tools.

Figure 5.1: Whiteboard surface and toolbar. From left the icons are: Change
color, draw lines, telepointer, zoom, pan, erase all, follow me and close session.

17

Except for the tools there are also four other items on the bar. One to
change the color, one to clear all objects, one to enable follow me mode and one
to close the session. The item used to change color works so that when clicked
on it changes the color of the icon, which consists of a circle with a color inside.
The icon cycles through a set of five predefined colors; black, white, red, green
and blue.
The draw line tool works as any normal drawing tool. The user just needs
to put the pen on the surface and draw as with a normal pen. Since probably
everyone have used a normal pen, this should feel natural. A similar technique
is used to implement the telepointer. When the user moves the pen on the
surface the point moves with it, with the point on the tip of the pen, and the
tail running after.
The zoom technique the zoom tool is implemented with is the one named in
Section 3.6, where one axis is used to zoom out, and one to zoom in. By tilting
the axis 45o the users can get a slightly more natural feeling when zooming,
since they then can drag the pen straight up/down or right/left to zoom in/out.
This is shown in Figure 5.2.

Zoom in

Zoom out

Figure 5.2: How the zoom works. Dragging the pen up and right zooms in and
dragging it down and left zooms out.
The pan tool is implemented using the grab and drag method. It is an
absolute pan tool, as described in Section 3.6. When a user selects the pan tool,
pushes the pen against the screen and drags the pen, the image will follow the
movements of the pen.
The follow me icon activates the WISIWYS functionality described in Section 3.6. If user A activates the follow me functionality, then user B is forced
to follow any zoom or pan operation. User B will always see what user A sees.

5.2

Graphics Class Design

When considering functionality of the whiteboard some classes are easily identified; the whiteboard surface itself and graphical objects such as images, pointers
and polylines. The whiteboard surface is represented by the class Canvas and the
abstract class Object is a generic graphical object with subclasses for each type
of object. Some less obvious classes that can be found are Tool and ViewPort.
Tool is an abstract class to represent what happens when the user points on the
screen, with subclasses for polylines, pointer, zoom and pan. Finally ViewPort
represents the part of the Canvas that is visible on screen. The classes and how
they relate to each other can be seen in Figure 5.3.

18

Viewport
1
1

Canvas
1
1

Tool

ZoomTool
PanTool

ObjectManager

PolylineTool

Object

PolylineObject

ArrowTool

ArrowObject
BitmapObject

Figure 5.3: Diagram showing the graphics classes and their relationships.

5.3

Network Interface

The network interface is implemented with four major classes; ConnectionHandler, PacketFactory, PacketHandler and Packet. ConnectionHandler is the class
that handles all connections, sending and receiving packets over sockets. PacketFactory is used only when receiving data, to translate the packet identifier from
the header into the correct subclass of Packet. A Packet is sort of a container
class that can read from the socket, and store the data into itself. ConnectionHandler uses the PacketFactory to create a Packet when data comes through a
socket. That packet is then sent into a PacketHandler that handles the Packet.
The socket is implemented with TCP because of the requirement of reliable
data transfer when logging in and so on. No demands about latency is made, so
TCP is an easy choice over UDP. All data sent over the socket is sent in binary
to minimize the data sent. The protocol is specified in Appendix A.

5.4

User Database

To be able to identify a client, each one needs a unique username and password.
On the server these are stored in a user database, i.e. in an instance of the
class UserDB. In the UserDB each user has its own User object, that contains
information such as name, password and Internet address. To see the connection
between UserDB and the network interface, see Figure 5.4. When a packet
containing login information arrives the PacketHandler communicates directly
with UserDB to check that the information is valid. The server is then reporting
back to the client.
The UserDB is also used when a client queries for other users and groups.
When the PacketHandler on the server receives a query from a client it hands
it to the UserDB which returns a list of results which the PacketHandler then
sends to the client. The query function in the UserDB works so that it can filter
the list of users based on whether they are in certain groups and have some
given status.

19

Group

UserDB

User

ConnectionHandler

ServerPacketHandler

ServerPacketFactory

Packet

Figure 5.4: Server class design.

5.5

Selecting a User

In the client the user needs a way to find and select the user to connect to. This
interface needs some way to select what groups to filter on, and some way to
show the results of the query.
The interface chosen consists of two lists; one showing users and the other
showing groups, see Figure 5.5 for a screenshot. When the user selects one or
more groups the user list is filtered to show only users that are members of
all the selected groups. Through a menu selection the list of users can also be
filtered by status; i.e. only show online, only show online and away or show all.
The status of a user is indicated by the color of the icon next to the name.

Figure 5.5: The user selection dialog.

To make it easier to find users and groups that are often used both lists can
be switched to only show favorites. The list of favorites are stored locally and
can be added to and removed from by selecting the appropriate alternative from
a popup that open when the pen is held down on a user or group.
To connect to someone the user can either double click on a user or select a
user and then click OK or choose connect from the menu.
In normal use the dialog should be set to only show favorite users that are
online. This makes connecting very quick and easy in most cases; just one
double click.

20

5.6

Synchronizing Objects

When two clients are connected both have an ObjectManager that keep track of
all whiteboard objects. These must be synchronized so that both clients have the
same objects and thus show the same image to the users. The synchronization
is achieved as follows: Each time a modification is made to an object a packet
is sent with a request that the other client performs the same modification.
For identifying objects across the network connection each object has a unique
id-number that is assigned when it is created.
Figure 5.6 shows what happens when the user of client A move the telepointer. The packet sent over the network contains the id-number identifying
client As arrow as well as the new position. Client B uses the id-number to
get the object from the ObjectManager and then moves the arrow to the new
position. The action for other drawing operations are analogous.
Client A
ArrowTool

mouseDragged

Client B

ConnectionHandler

ConnectionHandler

ClientPacketHandler

ArrowObject

ObjectManager

sendPacket
handle

getObject
setPoint
objectMoved

Figure 5.6: Sequence diagram showing how the telepointer is synchronized.

This concludes all parts of the implementation for the handheld device. The
next two sections describe the server interface and the desktop client.

5.7

Server

The server is a Win32 application that has only the most important functionality. It is a console application that can be controlled with a text menu. Events
on the server is logged to the console. The base configuration of the server is
made through text files, but users and groups can also be added with the help
of the menu. The server interface can be seen in figure 5.7.

Figure 5.7: The server interface.

21

5.8

PC Client

The PC client is developed for the Win32 platform and reuses much of the code
from the Windows Mobile client. This sharing of code would have been much
harder if a device with another OS had been chosen.
The PC client has mostly the same features as the handheld client. There
are some differences though:
There is no camera support in the PC client.
The GUI looks a bit different mainly because of the screen size not being such a restriction on the PC. The user selection dialog has no menu
and instead shows all options in the window. The whiteboard window
(see Figure 5.8) is resizable and can be maximized making zoom and pan
almost unnecessary.
On the PC client there is a rectangle showing what the other user currently
sees.

Figure 5.8: The PC client in a whiteboard session.

22

Chapter 6

Evaluation
This chapter describes the different tests made and presents the results. It starts
with Section 6.1 by describing the first low-fidelity prototype made to test the
interface. It continues in Section 6.2 to describe the high-fidelity prototype that
tested both the interface and some of the tools. The evaluation of the final
prototype is presented in Section 6.3. The last section in this chapter evaluate
the hardware chosen for the prototypes.

6.1

Mock-up Tests

To evaluate the overall design of the user interface some mock-ups were made
in prior to writing any code. The first one was just sketches on paper and was
not evaluated with test users. It was just discussed by the authors with the
following conclusions:
The main functions, starting a session and changing status, should be
large buttons taking up most of the screen. Secondary functions such as
changing settings should be placed in a menu.
Regarding the screen used for selecting a user, it was decided to go with
the separate group and user lists approach described in Section 3.3.
Fullscreen should be used for the whiteboard mode. There should be a
toolbar at the bottom with icons for all tools. To fit more tools into the
menu small popups with additional icons can be used when some of the
icons in the toolbar are selected. The icon for the selected tool should be
highlighted.
After the evaluation of the paper mock-ups a graphical mock-up was created
and a user test performed. A sample of the images can be seen in Figure 6.1.
Six test users were selected from the authors friends and families. They had
varying degree of computer experience ranging from those who only use computers occasionally to computer science students. During the evaluation the
users were shown pictures of the interface and were asked what they would do
in different situations:
1. Start a new session and connect to a mechanic.
23

2. Zoom and pan to a detail in the picture.

3. Draw a circle around something.
4. Direct the remote users attention to a certain spot.

Figure 6.1: Two of the images from the graphical mock-up. The buddy list on
the left and the whiteboard on the right.
At the end there were some questions about what they think of the interface
overall. The results were as follows: The main screen is easy to understand as
expected. It is just a few buttons after all. The user selection screen is a bit
harder to understand. The concept of filtering the user list when a group is
selected is not obvious. The problem can be partly due to the low fidelity of the
prototype. It was hard to show what happens in all cases because there were
not pictures to show all possible states. The reason for this is that the number
of combinations of selections is large and there were not time to make pictures
for all of them.
There was some things that the test users had problems with understanding
regarding the whiteboard:
It is hard to understand what the aim marker tool does. The tool is for
placing a permanent crosshair-like marker on the image. A user proposed
to use a flag instead, which might be more easily understood. In the
end the tool was dropped completely from further prototypes because the
telepointer or line tool can serve the same purpose.
The exclamation mark tool is also hard to understand. The purpose of
it is to flash an exclamation mark on the image a few time to bring the
attention of other user to a spot without drawing anything permanent on
the image. The decision was to remove that tool from further prototypes
because the value added by it is minor.
24

 The zoom-tool can be hard to understand from the looks of the icon. It
is still easy enough to understand once the user have tested it. It was
decided to let it be the way it is at least until the first evaluation of the
real program when the users can get a better feel for the tool.
The pointer in the prototype was just supposed to be an arrow pointing
from below and up, like a normal mouse pointer. One user gave us the idea
that the arrow should be rotatable so that it can point from any direction.
This can be done in two different ways. The arrow can be attached to a
point and be dragged to the right place and with the right direction or it
can be drawn like a vector. Point one where the screen is first touched,
point two where it is released.

6.2

Single User Interface Test

The first software prototype is a single user whiteboard without any support for
network communication. The functions supported are taking a picture with the
camera, drawing, zooming, pointing and panning. The goal with this prototype
was to evaluate how easy it is to use the functions and the current interface.
Six users, three with good and three with small to average computer knowledge
tested the prototype. The result of the evaluation is presented below:
The line and pointer tool are straight forward to understand and use.
All test users had problems with the zoom tool at first but after they were
told how it works they found it pretty easy to use. Most users first tried
to just tap on the screen expecting the application to zoom in on that
point. Implementing zoom in that way would be possible but the points
made in Section 3.6 and the fact that users learned to use the tool pretty
quickly speaks for the point and drag way of zooming.
Two users had some difficulty with the pan tool, they pointed and dragged
a small distance thinking that the viewport would start to scroll in that
direction. This way of scrolling is the relative pan method described in
Section 3.6. Because the maximum zoom in the prototype is around 400%
there is no need to scroll far and thus the better precision offered by the
absolute pan tool is still preferred.

6.3

Final Evaluation

The final prototype is a combination of the network prototype and the whiteboard prototype with some improvements. It was evaluated in a user test with
workers from SSAB. The goal is to find out if the application is something the
can be used in their work. Because the Qtek 9100 is not suitable for use in industrial environments the test was conducted in the workers office rather than
on the factory floor. The test users are told that this is just a prototype and
that it has some limitations a finished product will not have. A one page user
guide, shown in Appendix B, is given to the workers so they can see how to
use the application. After that they are presented with a Qtek running the

25

prototype and are told to try it out. After the test session is done the workers
are given a questionnaire to fill out.
During the evaluation the workers were very excited over the possibilities the
application had for making their work easier. They mentioned several situations
where they could use it. Also, a few missing features and shortcomings in the
prototype were pointed out. Here are some comments that the workers had:
The camera on the Qtek 9100 is not good enough in dim light conditions.
A camera with better light sensitivity is needed and a finished product
must have some kind of flash.
The client needs an option to save the picture to disk for future reference.
This is needed so they can document things for the workers in the next
shift.
The PC client should be able to open other types of documents than
images. This is to be able to send for example a blueprint of a machine
to someone fixing it.
This prototype was also tested at the same time as GSM, which seemed to work
without any problems. It is just a bit troublesome to make a call, since the
user needs to do that outside the application, preferably before starting the
prototype.

6.4

Hardware Evaluation

The communication services available from the Qtek 9100 was sufficient and
worked almost without trouble. It sometime had some problems to get its IPaddress when using WLAN, but that is probably due to a configuration problem,
and not the Qtek itself. Another thing that was noticed, not surprisingly, was
that the camera is insufficient for dim environments. The resolution is sufficient,
but the flash is not even close to be strong enough to compensate for the lack
of light sensitivity. The only way the Qtek 9100 could be useful in a dim
environment is to use a extern flash strong enough.
The single user prototype was tested by loading a 5 megapixel photo onto
the whiteboard, but this failed because of insufficient primary memory. The
photos taken by the camera one the Qtek 9100 is about 1 Megapixel, and load
without any problems. Maps or blueprints are examples of images that might
be to large to fit in the memory.

26

Chapter 7

Discussion
Most problems has a lot of different solutions and this thesis problem is not
an exception. The first section in this chapter discusses some of these solutions
and why those where not chosen. Section 7.2 contains the conclusions about
the problem stated in this thesis. The last section in this report discusses some
of the shortcomings that are outside the scope and briefly suggests different
solutions.

7.1

Alternative Solutions

During the development alternative solutions to the custom architecture used

in the prototype were looked at. One such alternative was to use AJAX [19]
for the client application. That would, at least in theory, have made it cross
platform compatible and easy to use from any device in any place as long as
there is a network connection. No installation is needed for the user since
the whole application is on a web page that runs in a standard browser. The
reasons AJAX was not used are threefold. First, getting an image from the
camera requires some platform specific code and that would defeat one of the
advantages. Second, the latency introduced is quite bad as can be seen in
existing AJAX whiteboards [20][21][22]. This makes real-time collaboration
hard. Third, AJAX is not so portable after all. Making an application that
works both on smartphones and PCs and in different browsers seems almost
impossible.
Another alternative that was considered was to use SIP [23] to find and
connect to other users. The main advantage would be that it is a standard
protocol put forward by Internet Engineering Task Force and therefore used by
many other collaboration tools. SIP does not have the group filtering that the
custom server of the prototype has, but that can be implemented in some other
way. The reason to not use SIP was that learning how to use it would have
taken longer time than making the ad-hoc protocol. The authors decided to
spend their time on other parts of the application instead. Looking back, this
was probably not the best choice. A lot of though and experience has been put
into SIP and that is hard to beat with a custom protocol.

27

7.2

Conclusions

The goal of this thesis was to design and develop a prototype of a collaborative
tool for industrial environments. The purpose of the tool is to help the users
solve problems more efficiently by making it easier to get help from an expert.
The evaluation with users from SSAB shows that the whiteboard prototype developed has the basic functionality needed by the workers, but a finished product
needs some more features. Chapter 6 mentioned a few of these shortcomings
and a few more are mentioned below and in the next section.
It is the authors belief that the design of the software components as described in this thesis is generally good. The time frame did not allow for more
thorough evaluation and therefore no firm conclusions can be made. The whiteboard GUI is the most tested part, as it was present in all prototypes, and all
the function of it works very well with only a small learning step for the zoom
tool. It is hard to say something about the user selection dialog without a longer
test in a more real scenario, but it does fulfill its purpose.
Even though the custom protocol used works well for the prototype it is not
sufficient for a real application. Using SIP for locating and connecting to other
clients is a better choice. For synchronizing the whiteboard the custom protocol
is not so bad, it lacks mostly in not tolerating dropped connections, which were
not considered in this thesis project.
When it comes to hardware, a finished product should probably not run
on Qtek 9100 because of the results of the evaluation in Section 6.4. If used
in a clean environment there should be no problem, but since the idea is that
it should be useful in industrial environments, a more robust device is to be
preferred. One of the most important things to actually make a product like
this work, is to choose a device with a good camera, otherwise the whole idea
with the application is lost.

7.3

Future Work

Before the application can be deployed there are some things that must be done
that are not covered in this thesis. First, the program should be able to handle
bad connectivity. It may not be possible to have 100% wireless network coverage,
so the application must reconnect automatically after loosing the connection.
A interesting thing to have would be the ability to seamlessly switch between
available network technologies; WLAN, GPRS, etc.
Second, the whole system must be made more reliable and configurable.
In the prototype these things were not a concern because of the controlled test
environment. More though must also be put into security issues. The prototype
sends everything in clear text, even passwords.
Finally, to be really useful voice communication must be integrated in the
application. Requiring the user to make a separate phone call as in the prototype
is not acceptable. The voice communication can use the normal GSM capability
of the smartphone or use voice over IP on the same network connection as the
rest of the application.
In addition, there are some features that were not in the prototype, that are
worth looking into. To start with it would be interesting to see if the automatic
zoom and pan described at the end of Section 3.6 makes the application easier

28

to use. There are also some other tools that could be implemented, for example
the flag tool that was proposed by a user in Section 6.1 as a replacement for the
crosshair tool. In this prototype there is no such marker tool, but that could
be useful. A tool for creating text such as small annotations is something that
could be good to have as well.
Using positioning or other forms of context awareness when selecting the
user to connect to is another thing to try. The position information can have
other benefits too. If the expert can see the whereabouts of a user that needs
his help, it is possible that he can for example find the correct blueprints for
the object of interest.
There should be a way to load background images based on vectors so that
maps, blueprints and other large background objects could fit in the memory
(see section 6.4. This could also be solved by dynamically loading just the
necessary parts of the background image to the primary memory and unload
the parts that are not needed.
Even though the focus of this thesis is on industrial environments the thought
has occurred that a service like the one described here can be offered to the
general public. The third generation mobile networks and todays smartphones
can easily support such an application. The requirements will of course be a bit
different than for the industry application.

29

Bibliography
[1] Microsoft NetMeeting.
http://www.microsoft.com/windows/NetMeeting/default.ASP,
13 September 2006.
[2] Marratech. http://www.marratech.com/, 26 September 2006.
[3] J. Kristiansson. Personalized and Context-aware E-meetings, 2005.
http://www.cdt.luth.se/johank/projects/emeeting/, 26 September 2006.
[4] N. Belov, I. Braude, W. Krandick, and J. Shaffer. Wireless Internet collaboration system on smartphones. In 17th International Conference on
Advanced Information Systems Engineering (CAiSE05), pages 675-689,
2005.
[5] H. Fabersjo, I. Windt, Y. Wridell, J. Sanneblad. Amigo - Wireless Image Based Instant Messaging for Handheld Computers. Viktoria Institute,
Sweden, 2003.
[6] W. Geyer, W. Effelsberg. The Digital Lecture Board - A Teaching and
Learning Tool for Remote Instruction in Higher Education. University of
Mannheim, Germany, 1998.
[7] Ericsson IMS WeShare.
http://www.ericsson.com/products/hp/IMS weShare pa.shtml,
26 September 2006.
[8] Wikipedia, Participant observation.
http://en.wikipedia.org/wiki/Participant observation, 6 September 2006.
[9] Wikipedia, Software Prototyping.
http://en.wikipedia.org/wiki/Software Prototyping, 6 September 2006.
[10] GIMP - The GNU Image Manipulation Program. http://www.gimp.org/,
6 September 2006.
[11] Degrees of Protection Provided by Enclosures (IP Code). IEC 60529, 2001.
[12] ICQ.com. http://www.icq.com/, 26 September 2006.
[13] Waste. http://waste.sourceforge.net/, 26 September 2006.
[14] A.K. Dey, and G.D. Abowd. Toward a Better Understanding of Context and
Context-Awareness. Technical Report GIT-GVU-99-22, Georgia Institute
of Technology, Atlanta, GA, USA, 1999.
30

[15] P. Dewan. Architectures for Collaborative Applications. In CSCW, Chapter

7, John Wiley and Sons, New York, 1999.
[16] R. E. Newman-Wolfe, M. L. Webb, and M. Montes. Implicit locking in
the ensemble concurrent object-oriented graphics editor. In Proceedings
of the 1992 ACM Conference on Computer-Supported Cooperative Work
(CSCW92), pages 265-272, ACM Press, New York, 1992.
[17] R. Rosenbaum, C. Tominski. Pixels vs. Vectors: Presentation of large
images on mobile devices.. In Proceedings of IMC, 2003.
[18] D. Lindstr
om, D. Nilsson. SWAIN. http://sourceforge.net/projects/swain,
25 October 2006.
[19] J. Garrett. Ajax: A New Approach to Web Applications. Adaptive Path,
2005.
http://www.adaptivepath.com/publications/essays/archives/000385.php,
26 September 2006.
[20] Virtual WhiteBoard.
http://www.virtual-whiteboard.co.uk/home.asp, 12 September 2006.
[21] Building Responsive Web Applications with AJAX and PHP: Online Edition.
http://ajaxphp.packtpub.com/ajax whiteboard.htm, 12 September 2006.
[22] Brads life - Weekend hack: Shared whiteboard in JavaScript.
http://brad.livejournal.com/2100411.html, 12 September 2006.
[23] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R.
Sparks, M. Handley and E. Schooler. SIP: Session Initiation Protocol. 2002.
IETF RFC3261.

31

Appendix A

Protocol Specification
This is the protocol used by the prototype. It is used both between a client
and the server and in peer-to-peer connections. The protocol is built on top of
TCP. The server listens on port 5982 and the clients listen on port 5983 when
waiting for a session to start.
All commands are sent as packets defined below. Every packet starts with a
four byte header containing a packet id number and a packet size. All multibyte
data is sent in network byte order.

PacketAnswer
PacketAnswer is what the server sends as answer when receiving a PacketQuery.
It contains the result from the query.
int16
int16
int32
int32[n]

Identifier for this packet. Should always be set to 3.

The size of the rest of this packet. Set to 4 + 4n, where
n is the number of results returned.
A flag that specifies what query this packet answers. It
is the same flag that was sent in the query.
Each int in this array corresponds to a user or group id,
depending on the query.

PacketArrow
PacketArrow tells another client the coordinates of a telepointer. If an object
with the unique id exists it is moved else it is created.
int16
int16
int32
float32
float32

Identifier for this packet. Should always be set to 14.

The size of the rest of this packet. Set to 12.
Unique identifier for the object.
Position of the telepointer on the X-axis.
Position of the telepointer on the Y-axis.

32

PacketBounds
PacketBounds is sent to the peer whenever the viewport changes. The packet
tell the other client the bounds of the viewport. If the follow me flag is set the
receiver should try to adjust its viewport to the same bounds. A PacketBounds
sent in response to such an adjustment must not have the follow me flag set or
a infinite loop may occur.
int16
int16
float32
float32
float32
float32
int8

Identifier for this packet. Should always be set to 17.

The size of the rest of this packet. Set to 17.
The left side of the bounds.
The top side of the bounds.
The right side of the bounds.
The bottom side of the bounds.
The follow me flag. Set to 1 for true and 0 for false.

PacketBye
PacketBye is sent to the other client when closing the peer-to-peer connection.
Upon receiving it the peer closes the connection.
int16
int16

Identifier for this packet. Should always be set to 13.

The size of the rest of this packet. Set to 0.

PacketClear
PacketClear is sent to the peer when clearing the screen.
int16
int16

Identifier for this packet. Should always be set to 16.

The size of the rest of this packet. Set to 0.

PacketConnect
PacketConnect is sent to the server when the client wishes to connect to another
client. Upon receiving it the server sends a PacketConnectTo to the other client
if it is online. If the other client is not online a PacketError is sent to the
requesting client.
int16
int16
int32

Identifier for this packet. Should always be set to 8.

The size of the rest of this packet. Set to 4.
User id of the user to connect to.

PacketConnectTo
PacketConnectTo is sent from the server to a client as response to when receiving
a PacketConnect. When the client receives PacketConnectTo, it tries to connect
to the client that the packet refers to.

33

int16
int16
int32

Identifier for this packet. Should always be set to 9.

The size of the rest of this packet. Set to 4.
The IPv4 address to connect to.

PacketError
PacketError is used when errors are sent from server to client. The error codes
used are seen below.
1
2
3

No error
Incorrect password (when logging in).
User not online.

int16
int16
int32

Identifier for this packet. Should always be set to 7.

The size of the rest of this packet. Set to 4.
The error code. See above.

PacketFile
PacketFile is used when sending a file. This is the first packet the client that
requested the peer-to-peer connection should send after receiving PacketReady
from the peer. The packet must be sent even if there is no background image,
then the file size is set to 0 and the rest of the packet is skipped. It there is an
image it should be a JPEG, GIF or BMP file. The rotation flag tells the client
if the image needs to be rotated when loaded.
int16
int16

int32
int32
char16[n]
int8
byte[s]

Identifier for this packet. Should always be set to 11.

The size of the rest of this packet. Set to 4 if there is no
file and otherwise to 9 + 2n + s, where n is the length
of the filename, and s is the size of the file.
The size of the file. Set to s.
The length of the filename. Set to n.
Array containing the filename. Not null-terminated.
Rotation flag. Set to 1 for 90o clockwise, 0 for no rotation.
The content of the file.

PacketHello
PacketHello is the first packet sent to the server after the connection is made. If
the username and password are correct the server answers with a PacketWelcome
else it sends a PacketError and closes the connection.

34

int16
int16

int32
char8[n]
int32
char8[m]

Identifier for this packet. Should always be set to 0.

Size of the rest of this packet. Set to 8 + n + m, where
n is the length of the username and m is the length of
the password.
Length of the username. Set to n.
Array containing the username. Not null-terminated.
Length of the password. Set to m.
Array containing the password. Not null-terminated.

PacketLookUp
Sent to the server to request information about a user or group. The server will
answer with PacketName and/or PacketStatus according to what was requested
in the flag field. It is a bitwise OR of the following values:
0000012
0100002
1000002
int16
int16
int32
int32

Get information about a group rather than a user.

Get the name of the user or group.
Get the status of the user or group.
Identifier for this packet. Should always be set to 4.
Size of the rest of this packet. Set to 8.
Group or user id.
Flags as described above.

PacketName
PacketName is sent by the server as a result of a PacketLookUp to report the
name of a user or a group.
int16
int16
int32
int8
int32
char8[n]

Identifier for this packet. Should always be set to 5.

Size of the rest of this packet. Set to 9 + n, where n is
the length of the name.
Group or user id.
Set to 0 if it is a user, 1 if it is a group.
Length of the name. Set to n.
Array containing the name. Not null-terminated.

PacketPolyline
Packet sent by a client to a peer to start a new or add to an existing polyline. If
there is a polyline with the unique id from the packet the given point is added
to the end of it and the color is ignored. If there is no such polyline a new one
is created with the given id, starting point and color.

35

int16
int16
int32
float32
float32
int32

Identifier for this packet. Should always be set to 15.

Size of the rest of this packet. Set to 16.
Unique identifier for the object.
X-coordinate of the new point.
Y-coordinate of the new point.
Color in packed RGB format.

PacketQuery
PacketQuery is sent to the server to get a list of users or groups. The flag field
specifies the the query. It is a bitwise OR of the following values:
000012
000102
001002
010002
100002

Use the list of groups to filter the result.

Include users or groups with online status.
Include users or groups with offline status.
Include users or groups with away status.
Get a list of groups rather than a list of users.

At least one of the status flags must be set or else the result will be empty. If
the flag for group filtering is set the server filters the list of users to only include
users that are members of every group in the supplied list.
The server will return the result in a PacketAnswer with the flag field set to
the same as in the query.
int16
int16
int32
int32
int32
int32[n]

Identifier for this packet. Should always be set to 2.

Size of the rest of this packet. Set to 12 + 4n, where n
is the number of groups.
Flags describing what is searched for, see above.
Maximum number of results the client wants.
Number of groups. Set to n.
Array with group ids.

PacketReady
When opening a peer-to-peer connection after receiving a PacketConnectTo the
client sends a PacketReady on the new connection. Upon receiving PacketReady
the other client proceeds to send a PacketFile.
int16
int16

Identifier for this packet. Should always be set to 12.

Size of the rest of this packet. Set to 0.

PacketSetStatus
PacketSetStatus is sent to the server to change the status of the client sending
it. Changing the status to offline makes the server close the socket to the client.
This is how to gracefully end the connection.

36

int16
int16
int32

Identifier for this packet. Should always be set to 10.

Size of the rest of this packet. Set to 4.
Status: 0 = online, 1 = offline, 2 = away.

PacketStatus
PacketStatus is sent by the server as a result of a PacketLookUp to report the
status of a user or a group. In the prototype groups are always reported to be
online.
int16
int16
int32
int8
int32

Identifier for this packet. Should always be set to 6.

Size of the rest of this packet. Set to 9.
Group or user id.
Set to 0 if it is a user, 1 if it is a group.
Status: 0 = online, 1 = offline, 2 = away.

PacketWelcome
PacketWelcome is sent by the server to indicate a successful login.
int16
int16

Identifier for this packet. Should always be set to 1.

Size of the rest of this packet. Set to 0.

37

Appendix B

User Guide

38