Grizzly 1.

5 Architecture Overview

Jeanfrancois Arcand Senior Staff Engineer

Sun Microsystems

Agenda
Introduction Project Grizzly

JGD

> What is Project Grizzly > Classes Diagram and description. > Aynschronous Request processing > Comet

HTTP extension Performance Q&A

What is Project Grizzly

JGD

Easy-to-use high performance APIs for TCP, UDP and SSL communications Brings non-blocking sockets to the protocol processing layer Utilizes high performance buffers and buffer management Choice of several different high performance thread pools

Uses Java technology based NIO primitives and hides the complexity of programming with Java technology based NIO

Project Grizzly and its Extensions

Derby Netbeans 6 Cometd

JGD

GlassFish AsyncWeb V3

Phobos

JRuby

Jersey

Comet

GlassFish v2

Adapter Synchronous Processing Asynchronous Processing

HTTP

RealTime

Jetty

Restlet

Grizzly Framework[1] Grizzly Framework

4

Framework Classes Diagram

JGD

Controller
Main entry point when using the Grizzly Framework. A Controller is composed of
> Handlers > SelectorHandler > SelectionKeyHandler > ProtocolChainInstanceHandler > ConnectorHandler > ProtocolChain > Pipeline.

JGD

All of those components are configured by client using the Grizzly Framework.

SelectorHandler
A SelectorHandler handles all java.nio.channels.Selector operations. One or more instance of a Selector are handled by SelectorHandler. The logic for processing of SelectionKey interest (OP_ACCEPT,OP_READ, etc.) is usually defined using an instance of SelectorHandler. This is where the decision of attaching an object to SelectionKey.

JGD

SelectorHandler (Cont.)
/** * This method is garantee to always be called before operation * Selector.select(). */ public void preSelect(Context controllerCtx) throws IOException;; /** * Invoke the Selector.select() method. */ public Set<SelectionKey> select(Context controllerCtx) throws IOException; /** * This method is garantee to always be called after operation * Selector.select(). */ public void postSelect(Context controllerCtx) throws IOException;

JGD

SelectionKeyHandler
A SelectionKeyHandler is used to handle the life life cycle of a SelectionKey. Operations likes cancelling, registering or closing are handled by SelectionKeyHandler.

JGD

SelectionKeyHandler (Cont.)
/** * Expire a SelectionKey. */ public void expire(SelectionKey key); /** * Cancel a SelectionKey and close its Channel. */ public void cancel(SelectionKey key); /** * Close the SelectionKey's channel input or output, but keep alive * the SelectionKey. */ public void close(SelectionKey key);

JGD

10

ProtocolChainInstanceHandler
An ProtocolChainInstanceHandler is where one or several ProtocolChain are created and cached. An InstanceHandler decide if a stateless or statefull ProtocolChain needs to be created.

JGD

11

InstanceHandler (Cont.)
/** * Return an instance of ProtocolChain. */ public ProtocolChain poll(); /** * Pool an instance of ProtocolChain. */ public boolean offer(ProtocolChain instance);

JGD

12

Pipeline
An interface used as a wrapper around any kind of thread pool.

JGD

13

Pipeline (Cont.)
/** * Add an <code>E</code> to be processed by this <code>Pipeline</code> */ public void execute(E task) throws PipelineFullException; /** * Return a <code>E</code> object available in the pipeline. */ public E waitForIoTask() ;

JGD

14

ProtocolChain

JGD

A ProtocolChain implements the "Chain of Responsibility" pattern (for more info, take a look at the classic "Gang of Four" design patterns book). The ProtocolChain API models a computation as a series of "protocol filter" that can be combined into a "protocol chain".

15

ProtocolChain (Cont.)
/** * Add a <code>ProtocolFilter</code> to the list. <code>ProtocolFilter</code> * will be invoked in the order they have been added. */ public boolean addFilter(ProtocolFilter protocolFilter); /** * Remove the <code>ProtocolFilter</code> from this chain. */ public boolean removeFilter(ProtocolFilter theFilter); public void addFilter(int pos, ProtocolFilter protocolFilter);

JGD

16

ProtocolFilter

JGD

A ProtocolFilter encapsulates a unit of processing work to be performed, whose purpose is to examine and/or modify the state of a transaction that is represented by a ProtocolContext. Individual ProtocolFilter(s) can be assembled into a ProtocolChain.

17

ProtocolFilter (Cont.)

JGD

The API for ProtocolFilter consists of a two methods: execute(Context) postExecute(Context)

which are passed a "protocol context" containing the dynamic state of the computation

18

ProtocolFilter (Cont.)
/** * Execute a unit of processing work to be performed. This ProtocolFilter * may either complete the required processing and return false, * or delegate remaining processing to the next ProtocolFilter in a * ProtocolChain containing this ProtocolFilter by returning true. */ public boolean execute(Context ctx) throws IOException; /** * Execute any cleanup activities, such as releasing resources that were * acquired during the execute() method of this ProtocolFilter instance. */ public boolean postExecute(Context ctx) throws IOException;
19

JGD

ConnectorHandler
Non blocking IO for client communication Supports UDP, TCP and TLS Can be combined with the Server NIO side to implement two ways communication ex: the Ericsson/Sun project Sailfin uses it for supporting the SIP protocol

JGD

20

Creating a Controller

JGD

21

Request Handling

JGD

22

Worker Thread execution

JGD

23

Example 1 - TCP
By default, the Grizzly Framework bundle default implementation for TCP and UPD transport. The TCPSelectorHandler is instanciated by default. As an example, supporting the TCP protocol should only consist of adding the appropriate ProtocolFilter like:

JGD

24

Example 1 TCP (Cont.)

Controller con = new Controller(); con.setInstanceHandler(new DefaultInstanceHandler(){ public ProtocolChain poll() { ProtocolChain protocolChain = protocolChains.poll(); if (protocolChain == null){ protocolChain = new DefaultProtocolChain(); protocolChain.addFilter(new ReadFilter()); protocolChain.addFilter(new MyProtocolFilter()); } return protocolChain; } }); con.addSelectorHandler(new TCPSelectorHandler());
25

JGD

Example 2 UDP
Controller con = new Controller(); con.setInstanceHandler(new DefaultInstanceHandler(){ public ProtocolChain poll() { ProtocolChain protocolChain = protocolChains.poll(); if (protocolChain == null){ protocolChain = new DefaultProtocolChain(); protocolChain.addFilter(new ReadFilter()); protocolChain.addFilter(new MyProtocolFilter()); } return protocolChain; } }); con.addSelectorHandler(new UDPSelectorHandler());
26

JGD

Architecture HTTP layer

Derby Netbeans 6 Cometd

Glassfish AsyncWeb V3

Phobos

JRuby

SWDP Rest

Comet

GlassFish v2

Adapter Synchronous Processing Asynchronous Processing

HTTP

RealTime

Jetty

Restlet

Grizzly Framework[1] Grizzly Framework

Sun Microsystems, Inc

27

Grizzly HTTP layer

Lightweight HTTP 1.0/1.1 based server Extremely easy to embed. Small footprint. Performance is extremely good Good performance apply to both Synchronous processing and Asynchronous Processing

JGD

28

Example: Grizzly Web Server

HTTP Request s
SelectorThread HTTPProtocolFilter

JGD

ProtocolParser Grizzly Framework

StaticResourceAdapter

29

Grizzly HTTP layer

Easy to embedded. Only have to interact with one object: SelectorThread Write an implementation of com.sun.grizzly.tcp.Adapter class. The Adapter is the glue code between the HTTP layer and the program that embed Grizzly. In the following example, the default StaticResourcesAdapter is used

JGD

30

JGD

Example 1 Static Resource Web Server

SelectorThread selectorThread = new SelectorThread(); selectorThread.setPort(port); selectorThread.setAdapter( new StaticResourcesAdapter()); selectorThread.setWebAppRootPath(folder); selectorThread.initEndpoint(); selectorThread.startEndpoint();

31

JGD

Asynchronous Request Processing

Allow for parking a request; a type of continuation at the request processing level The goal is to be able to build, on top of Grizzly, a scalable ARP implementation that doesn't hold one thread per connection, and achieve as closer as possible the performance of synchronous request processing (SRP).

32

JGD

Example Asynchonous Request Processing

SelectorThread selectorThread = new SelectorThread(); selectorThread.setPort(port); selectorThread.setWebAppRootPath(folder); selectorThread.setAdapter( new StaticResourcesAdapter()); AsyncHandler asyncHandler = new DefaultAsyncHandler(); asyncHandler.addAsyncFilter(new CometAsyncFilter()); selectorThread.setAsyncHandler(asyncHandler); selectorThread.initEndpoint(); selectorThread.startEndpoint();

33

Comparing Project Grizzly to Apache MINA both running AsyncWeb

Project Grizzly Performance

What is MINA?

Apache MINA (Multipurpose Infrastructure for Network Applications) is a network application framework which helps users develop high performance and high scalability network applications easily. AsyncWeb is a high-throughput, non blocking Java platform HTTP engine - designed throughout to support asynchronous request processing. AsyncWeb is build on top of MINA.

What is AsyncWeb?

2007 JavaOneSM Conference | Session XXXX |

34

Project Grizzly Performance

How did we performance test?

JGD

Client load generated via faban Two modes of measurement

> faban.sunsource.net > Throughput > Limited # of clients > No Think Time > Scalability > Max # of clients with 90% response time metric > Think time

35

Project Grizzly versus Apache MINA both running AsyncWeb

Higher is better, normalized to Grizzly score
100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Total Throughput Scalability
Grizzly Mina

JGD

Source: Internal Benchmark Tests 36

Project Grizzly Performance

JGD

Tested against a benchmark designed to:

Measure scalability, specifically to measure how many concurrent clients can be supported with:

Average client think time of 8 seconds 90% response time within 3 seconds Error rate < 0.1%

37

Project Grizzly HTTP vs other HTTP Servers

Higher is better
3000 2750 2500
% of Traditional I/O Score

2250 2000 1750 1500 1250 1000 750 500 250 0 2 CPU 6 CPU 16 CPU

Traditional I/O C-Based Server C-Based Server Grizzly

2007 JavaOneSM Conference | Session XXXX |

38

Q&A

JGD

39

Grizzly 1.5 Architecture Overview