Concurrency Patterns

David Kocher
00-806-968
dk@sudo.ch
Seminar Software Architekturen
Institut f
ur Informatik der Universit
at Z
urich
Prof. H. Gall, Prof. M. Glinz, Ch. Seybold, M. Pinzger

Abstract. Developing muli-threaded applications is harder than developing sequential programs because an object can be manipulated concurrently by multiple threads.
This paper paraphrases the concurrency patterns described in [POSA2],
including the Producer-Consumer, Active Object, Monitor Object, HalfSync/ Half-Async and Leader/ Followers pattern.

Introduction

A concurrent program has multiple threads of control allowing it perform multiple computations in parallel and to control multiple external activities which
occur at the same time [MaJk99].
The reasons given why concurrent programming is useful and adds a real
benefit to the applications overall performance usually include:
1. Parallel taks can run faster than a sequence of them could, if they are e.g.
I/O bound or gain performance from multiprocessing hardware.
2. Enhanced fairness and availability if clients need not to wait for each others
task to complete first [Lea99].
3. Incread application responsiveness; high prority thread for user requests.
Putting the user interface into a separate thread from any other work makes
the application feel more responsive to the user and ensures that an unexpectedly long operation doesnt freeze the applications screen [Shira00].
4. More appropriate structure; to model the environment in a object oriented
fashion we have to control multiple activities and handle multiple events.
The hard part of the concurrency model is that the threads are usually not
really independent. They must coordinate, synchronize, and share information,
and shared resources must be managed carefully to avoid corruption and erroneous computation [Doug02].
The choice of concurrency architecture has a significant impact on the design
and performance of multi-threaded networking middleware and applications. No
single concurrency architecture is suitable for all workload conditions and hardware and software platforms [POSA2].
Concurrency as a term includes the ability to perform multiple tasks at the
same time; we generally refer to that ability as parallelism. But threaded programming is about more than parallelism: its also about simpler program design.
Historically, threading was first exploited to make certain programs easier to
write; if a program can be split into separate tasks, its often easier to program
the algorithm as separate tasks or threads [OakWo04].
The benefit of introducing concurrency into an application (by creating threads
to perform tasks) must however outweight its costs [Lea99]. This decision is not
too different from introducing other kinds of objects in object oriented programming. One has to keep in mind that increasing the number of threads can
overwhelm processing times because of scheduling and context switching.

2
2.1

Design Patterns for Concurrency

Producer-Consumer
The Producer/Consumer pattern is used to decouple processes that produce and consume data at different rates. The Producer/Consumer patterns parallel loops are broken down into two categories; those that produce data, and those that consume the data produced. Data queues are

Concurrency Patterns

used to communicate data between loops in the Producer/Consumer design

pattern. These queues offer the advantage of data buffering between producer and consumer loops.
One of the more common patterns in threaded programming is the ProducerConsumer pattern. The idea is to process data asynchronously by partitioning
requests among different groups of threads. The producer is a thread that generates requests to be processed. The consumer is a thread that takes those requests
and acts upon them. This pattern provides a clean separation that allows for
better thread design and makes development and debugging easier [OakWo04].
The producer and consumer threads are decoupled: the producer never directly calls the consumer (and vice versa). This makes it possible to interchange
different producers without affecting the consumer. It also allows us to have multiple producers serviced by a single consumer, or multiple consumers servicing
a single producer. More generally, we can vary the number of either based on
performance needs or user requirements [OakWo04].
The producer relies on the consumer to make space in the data-area so that
it may insert more information whilst at the same time, the consumer relies on
the producer to insert information into the data area so that it may remove
that information. It therefore follows that a mechanism is required to allow the
producer and consumer to communicate so that they know when it is safe to
attempt to write or read information from the data-area.
The Producer/Consumer pattern is commonly used when acquiring multiple
sets of data to be processed in order. Suppose you want to write an application that accepts data while processing them in the order they were received.
Because queuing up (producing) this data is much faster than the actual processing (consuming), the Producer/Consumer design pattern is best suited for
this application. We could conceivably put both the producer and consumer in
the same loop for this application, but the processing queue will not be able
to add any additional data until the first piece of data is done processing. The
Producer/Consumer pattern approach to this application would be to queue the
data in the producer loop, and have the actual processing done in the consumer
loop. This in effect will allow the consumer loop to process the data at its own
pace, while allowing the producer loop to queue additional data at the same
time.
2.2

Active Object
The Active Object design pattern decouples method execution from method
invocation to enhance concurrency and simplify synchronized access to
objects that reside in their own threads of control [POSA2].

This pattern is used in many applications to handle multiple client requests

simultaneously and thereby to improve their quality of service. This pattern is
called Active Object referring to the fact that the concurrent object resides in its
own thread of control independantly of the thread of control of the client that

parate threads of con-

concurrent objects to
application to handle
stead of using singlee their methods in the
ked the methods, conead of control. Howst synchronize access
cts are shared by mulf this problem, three

currently should not

revent degrading the
, if one outgoing TCP
way example becomes
ay process should still
hile waiting for flow
utgoing TCP connecould be able to send
dently of any blocked

jects should be simxample are often hard

ly use low-level synquiring and releasing
eral, methods that are
should be serialized
ed by multiple client

igned to transparailable on a hardteway example, messhould be sent in parP connections. If the

run in a single thread
lenecks cannot be alGateway on a multi-

ent execution, decou-

An active object consists of the following components.

A Proxy [5, 2] represents the interface of the object and a
Servant [6] provides the objects implementation. Both the
Proxy and the Servant run in separate threads so that method
invocation and method execution can run concurrently: the
proxy runs in the client thread, while the servant runs in a dif4 ferent thread. At run-time, the Proxy transforms the clients
method invocation into a Method Request, which is stored
invoked
one of its methods.
that theThe
method
invocation
and the
in an Activation
Queue This
by ameans
Scheduler.
Scheduler
runs
actual method execution occurs in two different threads. To the client however
continuously in the same thread as the servant, dequeueing
it appears to invoke an ordinary method.
Method
Requests
from
thejust
Activation
Queuetowhen
they because
be- of
At runtime
the proxy
cannot
pass the requests
the servant
the
requirement
theyand
bothdispatching
run in different
threads.
Instead,
the proxy
come
runnable
them
on the
Servant
that constructs
ima method request object and inserts it into an activation list. This list does not
plements the Active Object. Clients can obtain the results of
only hold the method request objects inserted by the proxy but also keeps track
methods
execution
viaexecute.
the Future
returned
by the
Proxy.
ofawhich
method
request can
The method
request
object
contains the
information to execute the desired method later; i.e. the request parameters and
any other context information.
Using the activation list, the threads of the proxy and the servant can run
concurrently. Only the activation list must be serialized to protect it from concurrent access using a concurrency control pattern such as the Monitor Object
[2.3].
The
structure
of theinActive
Objectthen
pattern
is the
illustrated
in the
A scheduler
running
its own thread
invokes
corresponding
method
onfollowing
the servantBooch
using the
information
stored
in
the
method
request
object
it takes
class diagram:
from the activation list.

7 Structure

9::0&;
&&'&<&12#=5"%"%=>-%5"%"%)*
&&.?&)'>@"1$-)**&'>2199)*
A

!"#$%
!"#"$%&'()*
!"#"$%&'+)*
!"#"$%&',)*

(6&%45"%"%)4%8&7(*

;':<78:<#9
=+)+)

,6&-./01#23)*

&'()*+,)"
-./01#23)*
%45"%"%)*

/-&-012
34
51-2.3&

%45"%"%)*
-%5"%"%)*

+6&%45"%"%)7(*

>6

6
-./-&-012
34
51-2.3&

&)"789:
'()*
'+)*
',)*

B6&'()*

>):(#* 9
?)@+)A:
@"1$-)*
2199)*

>C
>B

Fig. 1. Structure of the Active Object concurrency pattern [Schm00].

There are six key participants in the Active Object pattern:

Because the proxy cant return the result of the method invocation, a placeProxy
holder is returned instead (if the method is a two-way invocation). A future is a
mechanism which can be used to provide values that will be resolved to another

A Proxy [2, 5] provides an interface that allows clients

to invoke publically accessible methods on an Active Object using standard, strongly-typed programming language features, rather than passing looselytyped messages between threads. When a client invokes
a method defined by the Proxy, this triggers the construction and queueing of a Method Request object onto
the Schedulers Activation Queue, all of which occurs

hread than its clients,

of Method Requests
heduler decides which
t and execute on the
ethod. This schedulcriteria, such as ormethods are inserted
synchronization conrtain properties or the
ch as space becoming
ounded data structure.
synchronization conguards.

and state that is be. Servants implement

8 Dynamics

rations in the Active Object pattern:

0&,1$'**$.C&4,
4$"+,2)4,!$"

456789
:;<=>
!"#$%&

!"#$
&/0.&.&#/&123"$

42&-,&**0&,1$'
2&()&+,
2&,)2"**H),)2&

*+?F7GB57;
3"
*-?96@A96<8
+7;@A89
(B7B7

-.+.%&#$
&/0.&.&#3"$

!"+&2,**!",$
******************-4,!#-,!$"**()&)&

+41&')/!"DE
&3&4),!$"

a bounded buffer of
d by the Proxy. This
hod Requests to exehread from the servant
concurrently.

has finished, and stored the result in the future. Asking for the futures value
will immediately give the result.
2. The future has not resolved. In this case, the current thread is blocked until
the future resolves. The current thread is effectively synchronised with the
thread calculating the value. But as long as the program logic does not
The
following
figure
illustrates
thethe
three
phases
of iscollaborequire
immediate
usage
of the result,
current
thread
not blocked.

4$0:/&,!$"

s context information
on on a Proxy, such
from the Proxy to a
thread. An abstract
interface for executt. The interace also
be used to determine
ronization constraints
ct method offered by
zed access in its Serclass is subclassed to
st class. Instances of
oxy when its methods
cific context informaethod invocations and

Proxy, a Future is returned immediately

to the Patterns
client.
Concurrency
5
The Future reserves space for the invoked method to
value asynchronously.
TheWhen
proxy can
returnwants
a future
to its caller,
store its results.
a client
to obtain
theseand
re-the servant will compute the value that the future will resolve to in another thread of
it canprovides
rendezvous
thethe
Future,
blockcontrol.sults,
The future
a way towith
ask for
actual either
value. Two
situations
ing or polling until the results are computed and stored
are possible:
into the Future.
1. The future has resolved already. That is, the method calculating the value

:.6%4#$

'&()&)&**+)!,-./&
******0&,1$'**2&()&+,

4&0.&.&#3"$
478'6+59#3"$

&3&4),&

56((#$
2&,)2"**2&+)/,

!"#$

%&'()*+,*-.+.%&#$

Fig. 2. Sequence diagram of the Active Object concurrency pattern [Schm00].

1. Method Request construction and scheduling: In this

phase, the client invokes a method on the Proxy. This trigRecapitulating, the Active Object concurrency pattern consists of the followgers
the creation of a Method Request, which maintains the
ing participants [POSA2]:
argument bindings to the method, as well as any other bind A proxy
that advertises
the the
interface
of an and
Active
Objectits
visible
to the client
ings
required
to execute
method
return
results.
threads.
The
Proxy then passes the Method Request to the Scheduler,
The servant provides the implementation of the methods defined in the proxy
which
enqueues it on the Activation Queue. If the method is
interface.
A serialized
activation[6],
list a
which
holdsto
the
method request
objectstoinserted
defined
as a two-way
binding
a Future
is returned
by
the
proxy
and
enables
the
servant
to
run
concurrently.
the client that invoked the method. No Future is returned if a
method is defined as a oneway, i.e., it has no return values.
2. Method execution: In this phase, the Scheduler runs
continuously in a different thread than its clients. Within this
thread, the Scheduler monitors the Activation Queue and determines which Method Request(s) have become runnable,
e.g., when their synchronization constraints are met. When a
Method Request becomes runnable, the Scheduler dequeues

2.3

Monitor Object
The Monitor Object design pattern synchronizes concurrent method execution to ensure that only one method at a time runs within an object. It
also allows an objects methods to cooperatively schedule their execution
sequences [POSA2].

In many applications there are multiple threads invoking methods on a object

which modify its internal state. In such cases we often have to make sure to
synchronize and schedule the access to these methods [POSA2].
The Monitor Object belongs to the group of passive objects. In contrary to
an Active Object [2.2], the monitor does not have its own thread of control. Each
method is executed in the thread of the client and access is blocked until the
method returns. Also to prevent uncontrolled concurrent changes (race conditions), only one synchronized method can execute within the monitor at any
time.
The Monitor Object solves this problem implementing the following functionality [POSA2]:
1. The objects interface should define its synchronization boundaries and only
one method at a time should be active within the same object.
2. Objects should be responsible for ensuring that any of their methods that
requires synchronization are serialized transparently without requriring the
client to know anything about. Operations are invoked like ordinary method
calls but guaranteed to be mutually exclusive. The condition synchronization
is realized using wait and signal primitives (e.g. wait() and notify() in the
Java language).
3. If an objects method must block during execution (e.g. to wait for a condition to become true) it should not block the thread of control. Other clients
should be able to access the object in such a case to prevent a deadlock and
to take advantage of concurrency mechanisms avilable on hardware.
4. When a method interrupts its thread of control voluntarily, the invariants
must always hold; it must leave its object in a stable state.
To fullfill these properties, the Monitor Object synchronizes the access to its
methods. To serialize concurrent access to an objects state, each Monitor Object
contains a monitor lock. Synchronized methods can determine the circumstances
under which they suspend and resume their execution, based on one or more
monitor conditions associated with a Monitor Object.
Recapitulating, the Monitor Object concurrency pattern consists of the following participants [POSA2]:
1. The monitor object itself that exposes the methods to the clients known
to be synchronized. Each method executes in the thread of the client, the
Monitor Object itself does not have its own thread of control.
2. Synchronized methods of which only one can be execute at any point in time.
They implement the thread-safe functions exported by a Monitor Objects
interface.

upplier handler thread

into a full queue, the
e the monitor lock and
longer full, i.e., when
essage from it. A pair
can be used to implepty and not-full moni-

4. Synchronized method thread resumption: Once a

previously suspended synchronized method thread is notified, its execution can resume at the point where it waited
on the monitor condition. The monitor lock is automatically
reacquired before the notified thread reenters the monitor
Concurrency
Patterns
object and resumes executing the synchronized
method.

3. The monitor condition in conjunction with the monitor lock determines if a

The following
figure
illustrates
collaborations
in the
synchronized
method
should
suspend orthe
resume
its processing.

t pattern is illustrated

Monitor Object pattern:

!"#$%&'(
()*&+,"./0123405678
97:248""50;41<:540
="".735<>56<:540
./0123405678""97:248
:237<8"".E.F70.540

97:248""14085:540
04:5G51<:540

./0123405678""97:248
:237<8""37.E9F:540

!"#$%&'(
()*&+,"D

*+',-./01()

!"?@'%(@* !"?@'%(@* !"?@'%(@*

@BC&#(
$@#A
#@',%(%@'
!"#$%&'()

/.03.&:()
3!%,()
&'4'!5'()
5$59'6/*,-&'!/*1
*+',-./02() !"#$%&'()
/.03.&:()
6.,%78()
+',-./02() &'4'!5'()
&',$&6
&'5$+'*,-&'!/*1
&'5$+'*+',-./01()
!"#$%&'()
+',-./01()*&',$&6

/.03.&:()
&'4'!5'()

etween participants in
Fig. 3. Sequence diagram of the Monitor Object concurrency pattern [Schm00].

3
The synchronized keyword in Java Synchronization ensures that a group
of statements (a synchronized block) will execute atomically as far as all synchronized threads are concerned. Synchronization does not address the problem
of which thread executes the statements first: it is first come, first served.
The main synchronization mechanism in Java is the monitor. The Java language does not provide conventional concurrency control mechanisms, such as
mutexes and semaphores, to application developers.
Every object can have a monitor associated with it, so any object can synchronize blocks. Before a synchronized block can be entered, a thread needs to
gain ownership of the monitor for that block. Once the thread has gained ownership of the monitor, no other thread synchronized on the same monitor can gain
entry to that block (or any other block or method synchronized on the same
monitor). The thread owning the monitor gets to execute all the statements in
the block, and then automatically releases ownership of the monitor on exiting
the block. At that point, another thread waiting to enter the block can acquire
ownership of the monitor.

However, threads synchronized on different monitors can gain entry to the

same block at any time. For example, a block defined with a synchronized(this)
expression is synchronized on the monitor of the this object. If this is an object that is different for two different threads, both threads can gain ownership
of their own monitor for that block, and both can execute the block at the same
time. This wont matter if the block affects only variables specific to its thread
(such as instance variables of this), but can lead to corrupt states if the block
alters variables that are shared between the threads. [Shira00]
[Lea99] discusses the implementation of concurrency primitives such as mutexes, semaphores and readers/writers locks in Java in detail.

3
3.1

Architectural Patterns for Concurrency

Half-Sync/Half-Async
The Half-Sync/Half-Async architectural pattern decouples asynchronous
and synchronous service processing in concurrent systems, to simplify
programming without reducing performance. The pattern introduces two
intercommunicating layers, one for asynchronous and one for synchronous service processing with a queuing layer between [POSA2].

Concurrent systems often contain a mixture of both synchronous and asynchronous services. On the system level asynchronous programming is preferred
to improve the overall performance. Conversely, an application developer wants
to simplify its programming effort and therefore use synchronous processing to
reduce complexity. The need for both simplicity and high performance is often
contradictory but essential in many concurrent systems. As an example consider an operating system kernel which handles interrupts triggered by network
interfaces asynchronously but a higher-level applications services uses synchronous read() and write() system calls. The challenge is to make these services
intercommunicate nevertheless.
To provide such an architecture that enables the synchonous and asynchonous
processing services to communicate with each other the Half-Sync/Half-Async
pattern decomposes the services in the system into layers [POSA1], adding a
queuing layer between the synchronous and asynchronous layers. Both the asynchronous and synchronous layers in the Half-Sync/Half-Async pattern interact
with each other by passsing messages via this queuing layer. A real world example for such a queing layer is the socket layer in many UNIX derivates, serving
as a buffering and notification point between the synchonous application process
and the asynchonous, interrupt-driven hardware services in the kernel.
The Half-Sync/Half-Async pattern has the following participants in the UNIX
example mentioned above:
1. Tasks in the synchronous task layer (e.g. application processes) perform
high-level I/O operations that transfer data synchronously to message queuees
in the Queuing layer. Tasks in this layer are Active Objects [2.2] that have
their own thread of control while performing I/O operations.

ation and bufferus task layer and

events processed
ered in message
or subsequent red vice versa).

X kernel)

ower-level events
rces (such as netUnlike the Synynchronous layer
ve their own runhey cannot block
of events.

interfaces)

rk interfaces and
that are received
ous task layer.

on among particiwhen input events

vent processing is
nto the following

sources of events
yer via interrupts

Concurrency Patterns

!3456789
!:574((-;<6=5

&>?7=
#8>@

.5>>8A5
2<5<5

-?7=
#8>@

-+./0/1#./+-%)

-C%, 20!0!E%/ &-C%,

*D&-! *D&-!
*D&-!

gh-level I/O operously to message

Unlike the Asynhronous layer are
eir own run-time
they can block
O.

!"#!$%&'(((!)!%#
$!,)((.-/

!"#$%&'()
*+!,%)

*$+,!--((.-/

"-23"3"%&'()
!%20!0!((.-/

!"#$%&'()
B!20!0!((.-/

*+!,%)
!"!,0#!((#&-1

Fig. 4. Sequence diagram of the Half-Sync/Half-Async concurrency pattern [POSA2].

Figure 3: Collaboration between Layers in the Half2. The queuing layer

(e.g. Sockets) provides synchronization and buffering
Sync/Half-Async
Pattern
between the asynchronous and synchronous layers.
3. Tasks in the asynchronous task layer (e.g. Kernel) handle events from the
exernal event sources such as network interfaces. These are Passive Objects
messages.
key
totheir
understanding
the pattern is to recog[2.3] thatThe
do not
have
own thread of control.

nizeThe
that
Synchronous tasks
are
active
objects.
Thus, tasks
theyand
Half-Sync/Half-Async
has the
benefit
of simplified
higher-level
a single
point
for inter-layer
communication
the queueing
can
make
blocking
read
or write
calls atlayer.
any point in
accordance with their protocol. If the data is not yet avail3.2 tasks
Leader/Followers
able
implemented as active objects can sleep until the
Leader/Followers
architectural
an efficient concurdata The
arrives.
In contrast,
taskspattern
in theprovides
Asynchronous
layer
rency model where multiple threads take turns sharing a set of event
are passive
objects. Thus, they cannot block on read calls.
sources in order to detect, demultiplex, dispatch, and process service reInstead,
implemented
as passive
objects are triggered
queststasks
that occur
on the event sources
[POSA2].
by notifications
or interrupts from external sources of events.
The Leader/Followers patterns aim is to provide a solution to process multiple events concurrently such as in multi-threaded server applications. In a typical server scenario a high volume events such as CONNECT arrive simultaneously
from multiple event sources (such as multiple TCP/IP socket handles). It may
not be possible (speaking of performance) to associate a separate thread with
each single socket handle because of concurrency-related overhead such as contextHalf-Sync/Half-Async
switching and synchronizationpattern
issues. The
system
not be scalable.
The
yields
thewould
following
ben- A
key feature of the Leader/Followers pattern thefore is to demultiplex the assoefits:
ciations between threads and event sources.

7 Consequences

Higher-level tasks are simplified because they are

shielded from lower-level asynchronous I/O. Complex
concurrency control, interrupt handling, and timing issues are delegated to the Asynchronous task layer. This
layer handles the low-level details (such as interrupt
handling) of programming an asynchronous I/O system. The Asynchronous layer also manages the interac-

10

The Leader/Followers pattern structures a pool of theads to share a set

of event sources efficiently by taking turns demultiplexing events that arrive
on these event sources and synchronously dispatching the events to application
services that process them [POSA2]. Only one thread in the pool (the leader)
is allowed to wait for an event to occur while the other threads (the followers)
queue up waiting.
When
thread
anprocessing
event, itthread
first has
promotes
a follower
Rejoining
the athread
pool.detects
After the
run its event
handling
as the new leader
and then
takes
the
ofpool
a processing
thread another
dispatching
to completion,
it can
rejoin
therole
thread
and wait to process
event. A
the event to an
application-specific
event
It is important
to current
note that
processing
thread can become
the handler.
leader immediately
if there is no
leader
multiple such processing
threads
run concurrently
only one
thread. Otherwise,
the can
processing
thread returnsbut
to playing
the leader
role of athread
follower
and waits on the thread pool synchronizer until it is promoted by a leader
waits for new thread
events.
thread.
: Thread1

: Thread
Pool

: Thread2

: Concrete
Event Handler

: Handle
Set

join()
become new leader thread

handle_events()

select()

join()

notify
about
event
arrival

become new
follower
thread

event

promote_new_leader()

become new
leader thread

handle_event()

handle_events()
join()

select()
become new
follower
thread

A threads transitions between states can be visualized in the following diagram:

Fig. 5. Sequence diagram of the Leader/Followers concurrency pattern [POSA2].
New event

Processing
Processing
completed,
no current
leader

Processing completed
there is a current leader

Leading

Following
Become new leader

Douglas C. Schmidt 1998 - 2000, all rights reserved, Siemens AG 1998 - 2000, all rights reserved
19.06.2000 lf.doc

Concurrency Patterns

11

References
[POSA2] Douglas Schmidt, Michael Stal, Hans Rohnert, Frank Buschmann: PatternOriented Software Architecture, Volume 2: Patterns for Concurrent and Networked
Objects, John Wiley & Sons, 2000
[Doug02] Bruce Powel Douglass: Real-Time Design Patterns: Robust Scalable Architecture for Real-Time Systems, Addison Wesley, 2002
[Malkh04] Dahlia Malkhi: Server Architecture Models, Lecture Slides, Hebrew University, 2004
[Shira00] Jack Shirazi: Java Performance Tuning, OReilly, 2000
[Schm00] Douglas C. Schmidt: Monitor Object, An Object Behavioral Pattern for Concurrent Programming, Department of Computer Science, Washington University,
St. Louis
[Lea99] Doug Lea: Concurrent Programming in Java: Design Principles and Patterns,
Addison-Wesley, 1999.
[OakWo04] Scott Oaks, Henry Wong: Java Threads, 3rd Edition, OReilly, 2004
[MaJk99] Jeff Magee and Jeffrey Kramer: Concurrency: State Models & Java Programs, John Wiley, 1999.
[POSA1] Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad, Michael Stal, Peter Sommerlad, Michael Stal.: Pattern-Oriented Software Architecture, Volume 1: A System of Patterns, John Wiley & Sons.