Weblogic Server Performance Tuning

Weblogic Server Performance Tuning Technical Document

Author : Raghava Krishna E-mail : raghavaniceboy24@gmail.com

Page 1 of 12

Weblogic Server Performance Tuning

TABLE OF CONTENTS

OVERVIEW...........................................................................................................................................................3 TUNING WEBLOGIC SERVER.........................................................................................................................3 USING WEBLOGIC SERVER PERFORMANCE PACKS ........................................................................................................3 SETTING THREAD COUNT..........................................................................................................................................3 ASSIGNING APPLICATIONS TO EXECUTE QUEUES..........................................................................................................4 ALLOCATING THREADS TO ACT AS SOCKET READER......................................................................................................4 CONNECTIONPOOLS ENHANCE PERFORMANCE..............................................................................................................4 PREPARED STATEMENT CACHE...................................................................................................................................5 TUNING JDBC CONNECTIONPOOL INITIAL CAPACITY...................................................................................................5 TUNING JDBC CONNECTIONPOOL MAXIMUM SIZE......................................................................................................5 TUNE THE CHUNK SIZE..............................................................................................................................................5 TUNING CONNECTION BACKLOG BUFFERING................................................................................................................6 TUNING PARAMETERS FOR STARTING WEBLOGIC SERVER...............................................................................................6 TUNING JAVA VIRTUAL MACHINES............................................................................................................6 JVM HEAP SIZE.....................................................................................................................................................6 GOAL FOR TUNING HEAP SIZE.....................................................................................................................................7 DETERMINE OPTIMAL HEAP SIZE...............................................................................................................................7 ANALYZE THE FOLLOWING DATA POINTS......................................................................................................................7 YOU MIGHT SEE THE FOLLOWING JAVA ERROR IF YOU ARE RUNNING OUT OF HEAP SPACE.....................................................7 TURNING ON VERBOSE GARBAGE COLLECTION............................................................................................................7 SPECIFYING HEAP SIZE VALUES.................................................................................................................................8 JAVA HEAP SIZE OPTIONS.........................................................................................................................................8 CONTROL PERMANENT GENERATION AREA SIZE EXPLICITLY ............................................................................................9 OTHERS.................................................................................................................................................................9 OPERATING SYSTEM MONITORING..............................................................................................................................9 EJBS.....................................................................................................................................................................9 PERFORMANCE TERMS AND DEFINITIONS...................................................................................................................11 CONCLUSION.....................................................................................................................................................11 REFERENCES.....................................................................................................................................................11

Page 2 of 12

Weblogic Server Performance Tuning

Overview
This document gives a brief overview of how to tune the components in a Weblogic Server Platform for which we will focus on tuning Weblogic Server and Java Virtual machines. The first step in performance tuning is isolating the hot spots. Performance bottlenecks can exist in any part of the entire system - the network, the database, the clients, or the app server. It's important to first determine which system component is contributing to the performance problem. Tuning the wrong component may even make the situation worse. There are some great profilers tools like ( OptimizeIt, JProbe, Wily Introscope ) which can help detect performance bottlenecks in the application code itself. The following sections describe how to tune WebLogic Server to match the application needs.

Tuning Weblogic Server

Using WebLogic Server Performance Packs
Major performance improvements is seen in WebLogic Server when the performance pack is used for the application platform. Performance packs use a platform-optimized (native) socket multiplexor to improve server performance. For example, the native socket reader multiplexor threads have their own execute queue and do not borrow threads from the default execute queue, which frees up default execute threads to do application work. To use a performance pack, make sure the Enable Native IO attribute of the Server element.

Environment>>Server (Select your server) >>Tuning>>Check the "Enable Native IO" Then Re-Start the weblogic server.
Setting Thread Count
The value of the ThreadCount attribute of an ExecuteQueue element in the config.xml file equals the number of simultaneous operations that can be performed by applications that use the execute queue. As work enters a WebLogic Server instance, it is placed in an execute queue. This work is then assigned to a thread that does the work on it. These threads consume resources. Note: Performance will be degraded by increasing the value unnecessarily. By default, a new WebLogic Server instance is configured with a default execute queue (named default) that contains 15 threads

Thread Count Scenarios

Thread Count < number of CPUs o This Results in an under-utilized CPU. So Increase the thread count. Thread Count == number of CPUs o Theoretically ideal, but the CPUs are under-utilized. So Increase the thread count. Thread Count > number of CPUs

Page 3 of 12

Weblogic Server Performance Tuning o Practically ideal, resulting in a moderate amount of context
switching and a high CPU utilization rate.

Assigning Applications to Execute Queues

Although the default execute queue can be configured to supply the optimal number of threads for all WebLogic Server applications, configuring multiple execute queues can provide additional control for key applications. By using multiple execute queues, we can guarantee that selected applications have access to a fixed number of execute threads, regardless of the load on WebLogic Server. Default WebLogic Server installations are configured with a default execute queue which is used by all applications running on the server instance. You may want to configure additional queues to: Optimize the performance of critical applications. Throttle the performance of nonessential applications Remedy deadlocked thread usage To use user-defined execute queues in a WebLogic Server 9.0 domain, you need to include the uy se 1 ee te xsub-element of the sre s8 l c element in the cng and reboot the server er v file oi f

Allocating threads to act as Socket Reader

To set the maximum percentage of execute threads that read messages from a socket, the attribute ThreadPoolPercentSocketReaders should be used in the config.xml file. The optimal value for this attribute is application-specific. The default is value 33, and the valid range is 1-99. Allocating execute threads to act as socket reader threads increases the speed and the ability of the server to accept client requests. It is essential to balance the number of execute threads that are devoted in reading messages from a socket and those threads that perform the actual execution of tasks in the server. When the native performance packs are not being used, execute threads must be allocated to act as socket reader threads.

Environment>>Server (Select your server) >>Tuning>> Specify Socket Readers" Then Re-Start the weblogic server.
The number of Java socket readers is computed as a percentage of the number of total execute threads (as shown in the Thread Count field for the Execute Queue).

ConnectionPools Enhance Performance

Establishing a JDBC connection with a DBMS can be very slow. If the application requires database connections that are repeatedly opened and closed, this can become a significant performance issue. WebLogic connection pools offer an efficient solution to this problem. When WebLogic Server starts, connections from the connection pools are opened and are available to all clients. When a client closes a connection from a connection pool, the connection is returned to the pool and becomes available for other clients; the connection itself is not closed. There is little cost to opening and closing pool connections.

Page 4 of 12

Weblogic Server Performance Tuning

Prepared Statement Cache
The prepared statement cache keeps compiled SQL statements in memory, thus avoiding a round-trip to the database when the same statement is used later. Set LRU from the drop down box from

Services JDBC Data Source Connection Pool Statement Cache Type

Tuning JDBC ConnectionPool Initial Capacity
During development, it is convenient to set the value of the InitialCapacity attribute to a low number. This helps the server start up faster. In production systems, consider setting InitialCapacity equal to the MaxCapacity so that all database connections are acquired during server start-up. If InitialCapacity is less than MaxCapacity, the server then needs to create additional database connections when its load is increased. When the server is under load, all resources should be working to complete requests as fast as possible, rather than creating new database connections.

Tuning JDBC ConnectionPool Maximum Size

The MaxCapacity attribute of the JDBCConnectionPool element allows to set the maximum number of physical database connections that a connection pool can contain. Different JDBC drivers and database servers might limit the number of possible physical connections. In production, it is advisable that the number of connections in the pool equal the number of concurrent client sessions that require JDBC connections. The pool capacity is independent of the number of execute threads in the server. There may be many more ongoing user sessions than there are execute threads.

Tune the chunk size

A chunk is a unit of memory that the WebLogic Server network layer, both on the client and server side, uses to read data from and write data to sockets. To reduce memory allocation costs, a server instance maintains a pool of these chunks. For applications that handle large amounts of data per request, increasing the value on both the client and server sides can boost performance. The default chunk size is about 4K. Use the following properties to tune the chunk size and the chunk pool size: weblogic.ChunksizeSets the size of a chunk (in bytes). The primary situation in which this may need to be increased is if request sizes are large. It should be set to values that are multiples of the network's maximum transfer unit (MTU), after subtracting from the value any Ethernet or TCP header sizes. Set this parameter to the same value on the client and server. weblogic.utils.io.chunkpoolsizeSets the maximum size of the chunk pool. The default value is 2048. The value may need to be increased if the server starts to allocate and discard chunks in steady state. To determine if the value needs to be increased, monitor the CPU profile or use a memory/ heap profiler for call stacks invoking the constructor weblogic.utils.io.Chunk. weblogic.PartitionSizeSets the number of pool partitions used (default is 4). The chunk pool can be a source of significant lock contention as each request to access to the pool must be synchronized. Partitioning the thread pool spreads the potential for contention over more than one partition.

Page 5 of 12

Weblogic Server Performance Tuning

Tuning Connection Backlog Buffering

The AcceptBacklog attribute of the Server element in the config.xml file allows to set the number of connection requests the server will accept before refusing additional requests. The AcceptBacklog attribute specifies how many TCP connections can be buffered in a wait queue. This fixed size queue is populated with requests for connections that the TCP stack has received, but the application has not accepted yet. The default value is 50 and the maximum value is operating system dependant.

Select EnvironmentServer Configuration Tuning from the Administration

Console to enter a value for the Accept Backlog attribute. During operations, if many connections are dropped or refused at the client, and there are no other error messages on the server, the AcceptBacklog attribute might be set too low.

Tuning Parameters for starting Weblogic Server

Clearly, the JVM's performance also affects WebLogic's performance. Most aspects of JVM tuning relate to the efficient management of the memory heap, an efficient garbage collection scheme, and the choice between two Unix threading models: green and native threads The important performance tuning parameters in these files are the JAVA_HOME parameter and the java heap size parameters: Change the value of the variable JAVA_HOME to the location of your JDK. For example: set JAVA_HOME= C:\bea\jdk150_04 For higher performance throughput, set the minimum java heap size equal to the maximum heap size. For example: "%JAVA_HOME%\bin\java" -hotspot ms512m mx512m -classpath %CLASSPATH% -

Tuning Java Virtual machines

Garbage collection is the first and foremost thing while tuning JVM. Garbage collection is the VM process of de-allocating unused java objects in the java heap. For tuning Garbage collection we will focus on: JVM heap size Generational garbage collection

JVM Heap Size

Page 6 of 12

Weblogic Server Performance Tuning

The Java heap is a repository for live objects, dead objects and free memory. The JVM heap size determines how often and how long the VM spends collecting garbage i.e. dead objects. Larger the heap size full garbage collection is slower and frequency is less. Smaller the heap size full garbage collection is faster and frequency is more.

Goal for tuning heap size

To minimize the time spent doing garbage collection while maximizing number of clients to be handled at a time. To ensure maximum performance during benchmarking the heap size values should be set in such a way that garbage collection does not occur during the entire run of the benchmark.

Determine Optimal Heap Size

Use the -option to turn on verbose garbage collection output for your JVM and redirect vr eo b both the standard error and standard output to a log file. Make sure that the heap size is not larger than the available free RAM on the system.

Analyze the following data points

How often is garbage collection taking place? In the weblogic.log file, compare the time stamps around the garbage collection. How long is garbage collection taking? Full garbage collection should not take longer than 3 to 5 seconds. Lower heap if major GC time is greater. What is your average memory footprint? In other words, what does the heap settle back down to after each full garbage collection? If the heap always settles to 85 percent free, you might set the heap size smaller.

You might see the following Java error if you are running out of heap space

Turning on Verbose Garbage Collection

Turn on verbose garbage collection output for Java VM when WebLogic Server is started, as shown in the example. Redirect both the standard error and standard output to a log file.

Page 7 of 12

Weblogic Server Performance Tuning

This places thread dump information in the proper context with WebLogic Server informational and error messages, and provides a more useful log for diagnostic purposes. Example java -ms64m -mx64m -verbosegc -classpath $CLASSPATH -Dweblogic.domain =mydomain -Dweblogic.Name=clusterServer1 -Djava.security.policy==/bea/weblogic91/server/lib/weblogic.policy -Dweblogic.management.server =192.168.0.101:7001 -Dweblogic.management.username =system -Dweblogic.management.password =systemPassword weblogic.Server >> logfile.txt

Specifying Heap Size Values

Java heap size values are specified when starting the WebLogic Administration Server from the Java command line.

Example:

java ... -XX:NewSize =128m -XX:MaxNewSize=128m -XX:SurvivorRatio=8 -Xms512m -Xmx512m

The default size for these values is measured in bytes. Append the letter k or K to the value to indicate kilobytes, m or M to indicate megabytes, and g or G to indicate gigabytes.

Java Heap Size Options

-XX:NewSize: This option is used to set the New generation Java heap size. Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set this option to be one-fourth the size of the maximum heap size. Increase the value of this option for larger numbers of short-lived objects. Make sure that the New generation is increased as the number of processors are increased. Memory allocation can be parallel, but garbage collection is not parallel. -XX:MaxNewSize This option is used to set the maximum New generation Java heap size. Set this value to a multiple of 1024 that is greater than 1MB. -XX:SurvivorRatio This option is used to configure the ratio of the Eden/survivor space size. Try setting this value to 8 and then monitor your garbage collection. -Xms

Page 8 of 12

Weblogic Server Performance Tuning

This option is used to set the minimum size of the memory allocation pool. Set this value to a multiple of 1024 that is greater than 1MB. As a general rule, set minimum heap size (-Xms) equal to the maximum heap size (-Xmx). -Xmx This option is used to set the maximum Java heap size. Set this value to a multiple of 1024 that is greater than 1MB.

Control Permanent Generation area size explicitly

Set initial heap size = max heap size Permanent Generation area defaults: o 4MB min, 32MB max o To increase, use: o -XX:PermSize=32m .XX:MaxPermSize=64m

Use less than ~80% available RAM on the machine

o 32-bit limit is 2GB; 64-bit is 16GB

Others
Operating System Monitoring
Unix provides many tools for monitoring system sar: system activity mpstat: per-processor statistics vmstat: virtual memory statistics netstat: network statistics iostat: Input/Output statistics

EJBs

Page 9 of 12

Weblogic Server Performance Tuning

Enterprise Java Beans (EJB) are fundamentally a Java specification that allows for reuse of services. Typically, they act as bridges between the Web user and the backend, non-Web components that make up the application. The nature of an EJB is to be as portable and as flexible as possible so a developer need not necessarily include code to make the service transactional. Freeing the developer from concerns centered on transactional behavior, security, and lifecycle management, allows them to concentrate on implementing and improving the business logic of the application. Making changes to applications that employ EJBs is much simpler and faster because the developer can make attribute changes, such as altering the transactional behavior without touching the business logic. Pooling and caching are the main features that the EJB container provides to boost performance for session and entity beans. However, not all of these are applicable to all types of beans. The downside is that memory requirements are higher, though this isn't a major issue. Pooling is available for stateless session beans (SLSB), message-driven beans (MDB), and entity beans Local Interfaces Use local-interfaces or set call-by-reference to true to avoid the overhead of serialization when one EJB calls another or an EJB is called by a servlet/JSP in the same application. Note the following: o In release default. For releases off by default. Older do not explicitly turn o prior to WebLogic Server 8.1, call-by-reference is turned on by of WebLogic Server 8.1 and higher, call-by-reference is turned applications migrating to WebLogic Server 8.1 and higher that on call-by-reference may experience a drop in performance.

This optimization does not apply to calls across different applications Transaction Avoid using the Rtransaction parameter. Using Rcauses the EJB container to start a e e new transaction after suspending any current transactions. This means additional resources, including a separate data base connection are allocated IdleBeanCount The number of EJBs currently idle within a pool that is managed by the Bean Container, which provides services to EJBs, including creating bean instances, managing pools of instances, and destroying instances. It also handles threading, transaction support, data storage, and retrieval for the beans. While a steady count of Idle Beans is normal, a climbing graph can be a strong indicator that the container needs to destroy beans at a faster rate. A possible reason for an increased bean count is that a new application may have been deployed, or an existing one modified so that the EJBs are stateful (contain session information across session restarts; a valid application method but not always necessary). This parameter should be monitored closely for anomalous conditions. Caching Caching is one of the ways the WLS improves response time for the application and the end user. It places user session information in memory in the form of EJBs. These session beans containing the users session information are quickly and readily retrieved the next time the user visits that website. The administrator has control over how much session information is stored in the cache and needs to be mindful of striking a balance.

Page 10 of 12

Weblogic Server Performance Tuning

From a user perspective, it would be ideal if their session information was always found in the cache, ensuring a rapid response time when they visited the website. Isnt this the goal of IT managers? Perhaps it is also the managers goal to optimize the use of WebLogic and ensure that the software is tuned to deliver the best service to all users. This means removing old session beans from the cache and properly configuring the cache size so that it does not consume too much memory of the JVM. HTTP Sessions In terms of pure performance, in-memory session persistence is a better overall choice when compared to JDBC-based persistence for session state. If you use a single large attribute that contains all the session data and only 10% of that data changes, the entire attribute has to be replicated. This causes unnecessary serialization/deserialization and network overhead. You should move the 10% of the session data that changes into a separate attribute. So during developing the application need to take care the above fact. For the cluster environment make sure that the objects which are participating in the session need to be serialized.

Performance Terms and Definitions

Performance can be defined as: how the systems response time and throughput
are affected by adding load. Capacity can be defined as: what the maximum threshold for the system is under a given set of conditions. Scalability can be defined as: how well the system responds to increasing load by adding additional resources. Most commonly we use the response time of the system and the throughput of the system as measurements for these terms.

Conclusion
The above are only some of the various ways that the server can be tuned. Bear in mind, however, that a poorly designed, poorly written application will usually have poor performance, regardless of tuning. Performance must always be a key consideration throughout the stages of the application development life cycle - from design to deployment. It happens too often that performance takes a back seat to functionality, and problems are found later that are difficult to fix.

References
http://edocs.bea.com/wls/docs91/perform/index.html http://www.javaperformancetuning.com/tips/appservers.shtml http://whitepapers.techrepublic.com.com/whitepaper.aspx?docid=110328 http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=forum&f=40

Page 11 of 12

Weblogic Server Performance Tuning

WebLogic: The Definitive Guide, by Jon Mountjoy, Avinash Chugh

Page 12 of 12