1

Configuring & Tuning the

PeopleSoft Application
Server
2
Agenda

Inside the PeopleSoft Internet
Architecture (PIA)
In Depth Coverage of the Application
Server's Role in PIA

Installation Issues, Architecture
Considerations
Monitoring, Tuning, and Automating
Tasks
High Availability and Clustering
Questions/Answers
3
Part I: Inside the PeopleSoft Internet
Architecture (PIA)
4
Architecture History
Application
server
DB
server
Window
s
RPC
SQL
DB
server
Windo
ws
SQL
Application
server
DB
server
Brows
er
HTTP/HTTPS Jolt/Tuxedo
SQL
Web
server
Rendering
Code
Application
Code
5
PeopleSoft Internet Architecture
PeopleSoft Application Server
Controls all application logic/HTML/XML
generation
Jolt/Tuxedo interface to servlet/web server
LDAP authentication optional (Active
Directory)
Application
server
DB
server
Web
server
Brows
er
HTTP/HTTPS Jolt/Tuxedo
SQL
LDAP
6
PeopleTools Development
Environment
PeopleSoft Designer (IDE)
IDE talks to DB Server, or Application Server
Runs in Windows-only environment
Most objects are stored in the database
Application
server
DB
server
Web
server
Brows
er
Jolt/Tuxedo
SQL
Wi
n
SQL
Jolt/Tuxedo
HTTP/HTTPS
7
Runtime
HTML/XML Generation
Occurs on Application Server
Page definitions written in a proprietary code
is stored in database
Developer does not code HTML; does not
need to know web programming
HTML automatically generated from page
definition
Page design is somewhat limited
Pages have a standard look and feel
8
PIA Web Applications/Servlets
Page: end users connection to PeopleSoft apps
Portal: gathers content from multiple sources
Report Distribution: users interface to batch
reports
Application Messaging Gateway: transmits
publish/subscribe (ie XML) messages between
nodes
XML Link: used for Business Interlinks
Integration Broker: next generation application
messaging/XML link
Sync Gateway: used for mobile connectivity
Web
9
PIA Web Applications/Servlets
Integration Broker
Messaging (sync/async) across the
enterprise
Web services based hub that handles
system to system interactions
External
Systems
Application
server
DB
server
Web
server
Brows
er
Jolt/Tuxedo
SQL
HTTP/HTTPS
XML
10
In Depth Coverage of the Application Server's
Role in PIA
11
Application Server Domains
Built around Tuxedo domains
Tuxedo developed by Bell Labs in 1983
BEA buys Tuxedo from Novell in 1995
Provides many benefits
Scalability (multiple instances of app server
processes, fewer connections to database)
Restartability/fault tolerance of app server
processes
Load balancing of app server processes
12
Application Server Domains
Domains consist of two items
Sub-directory within $PS_HOME/appserv
Set of server processes
Relationships
Domains to Machines: Many-to-1
Domains to Databases: Many-to-1
Databases to Domains: 1-to-1
13
Application Server Domains
Simple domain on one application server
machine
BROWSER
WEB
SERVER
APPLICATION
SERVER
HR
database
HR
Domain
14
Application Server Domains
Multiple domains per application server -
each domain associated with a different
URL
BROWSER
WEB
SERVER
APPLICATION
SERVER
HR
database
HR
Domain
FS
Domain
FS
database
15
Application Server Domains
Fault resilient multiple machines
BROWSER
WEB
SERVER
APPLICATION
SERVER
HR
database HR
Domain
FS
Domain
FS
database
APPLICATION
SERVER
FS
Domain
HR
Domain
WEB
SERVER
16
APPLICATION SERVER
Application Server Domains
BBL
Workstation
Handler
Workstation
Handler
Workstation
Listener
1
st
Request
Request
Request
Request
PSAPPSRV
PSAPPSRV
PSAPPSRV
PSQRYSRV
PSQRYSRV
APPSRV Q
QRYSRV Q
17
BEA Tuxedo Communications
Queue
A process used when an application needs to
communicate in a time-independent fashion
Transaction
Groups logical actions together and allows the
operation to be undone
Event
A server is notified when the information it
wants is available or when something has
happened
18
TUXEDO Communication Models
Synchronous Request/Response: tpcall(), TxRPC
Asynchronous Request/Response: tpacall(),
tpgetrply()
Conversation: tpconnect(), tpsend(), tprecv()
Unsolicited Notification: tpnotify(),
tpbroadcast()
Event: tppost(), tpsubscribe()
Queued Request/Response: tpenqueue(),
tpdequeue()
19
PeopleSoft Servers
PSAPPSRV
Performs panel builds and panel save
Caches object definitions & business logic in memory to speed
processing
PSQCKSRV (optional)
Performs a subset of fast, non-transactional, read-only requests
PSSAMSRV
Performs transactional SQL requests
PSQRYSRV
Handles SQL request from PS Query
Messenger Services
Publishing Broker: PSBRKDSP, PSBRKHND
Publishing Contractor: PSPUBDSP, PSPUBHND
Subscription Contractor: PSSUBDSP, PSSUBHND
20
Application Server Domains
PSADMIN
Command line utility to create and administer
domains
Domain status
For use by the Application Server administrator
psappsrv.cfg
Contains domain specific settings
For use by the Application Server administrator
Many other files behind the scenes
Process Scheduler domains use same procedure
21
Installation Issues and Architecture
Considerations
22
Two Main Discussion Points
AppServer Sizing
Simple Install





23
AppServer Sizing
Measure against a known workload
TPC/C Ratings (http://www.tpc.org)
Compare the TPC/C ratings
Compare the different models
Physical or Logical Servers?
Are the Servers, OS, etc. supported by
PeopleSoft?
Upgrade options?
24
AppServer Sizing
PeopleSoft Benchmarks
Rule-of-Thumb: Application Server needs 6
times more processing power than a web server
Estimate the concurrent usage patterns
Categorize the users into light, medium and
heavy
25
AppServer Sizing
Estimate the number of Light, Medium & Power
users
Estimated load by user type
Light User TPC/C = 60
Medium User TPC/C = 100
Power User TPC/C = 30
For Example Calculation, Assume
500 light users
100 medium users
10 power users
26
App Server Sizing
Calculating the TPC figures
For 500 Light Users, 60 X 500 = 30,000
For 100 Med Users, 100 X 100 = 10,000
For 10 Power Users, 300 X 10 = 3,000
Thus Total Appserv TPC units = 43,000
You have to find a server that would deliver at
least this many TPC units
Add any additional work load
Memory???? PS recommends 512MB per 25
users
Add 1.5MB per each additional user
27
Appserver Sizing: Remember!
Highly CPU bound
Large memory foot print in psappsrv
Memory growth may result in
fragmentation of main memory
Watch for the recycle count to recycle the
appserv at periodic intervals.
Caching is good; Queuing is bad
28
Comes right off the Tuxedo CD
Install.sh or setup.exe
UNIX/AIX ftp
Tuxedo 6.5/Jolt 1.2 certified Tools 7.05 8.4x
Max install approx. 160MB
Software
Docs
WebGUI Admin
Upgrading Tuxedo?
Win - Must uninstall old version!
UNIX Tuxedo 6.3 vs 6.5?
Simple Install
29
Simple Install
AppServer Administrator
Existing vs. New User
Environment Variables
TUXDIR
PATH
NLSPATH
Validate Install
$TUXDIR/udataobj/lic.txt
PeopleSoft Edition Built on BEA March
21, 2000
30
Simple Install
Problems with the Install?
Right CD?
Disk Space?
Windows - uninstall complete?
UNIX/AIX /var/tmp sizing?
31
Simple Install
Configuration
PSADMIN
Command line utility to create and administer
domains
Domain status
For use by the application server administrator
New Install
Domains?
Sub-directory within $PS_HOME/appserv
Set of server processes
You can have more than one domain per
machine
Each domain only connects to one database
Multiple domains can connect to the same
database
32
Simple Install
Configuration
Upgrades
Recreate Domain!!!!!!!!!
Tuxedo may not change but PIA does!
Creating the Domain (PSADMIN)
Updates psappsrv.cfg, generates psappsrv.ubb, creates
pstuxcfg
Appserver Size Templates
IDs
Determine ports in advance (tracking is a good idea)
WSL (7000)
WSH Random assignment
JSL (9000)
JSH = JSL + Max Handler!
JRAD (9100)
33
Simple Install
Problems?
Bad ID
Wrong Database
No 2-Tier access
Unavailable Port
No BBL
Add to Path?
34
Monitoring, Tuning, and Automating Tasks
35
Most Common Performance
Problems - Web & App. Server
S/W
http://www.newport-group-inc.com/
36
Typical PeopleSoft Daily
Monitoring
Database components
System logs
Process Scheduler
Application Server(s)
Web Server(s)
Report Server
File Server
Network operation
Customer Connection for patches and
code updates
37
Monitor Application Server
The PeopleSoft Administrator can check
the status of each application server
domain from the PSADMIN program
Log files
For each domain should be checked at least
daily
File size should also be checked; they can
grow quite large, especially if the trace
options are enabled
Check for queuing at peak and normal
times
38
Automated Monitoring
echo pq | tmadmin -r 2> /dev/null >
domainlog
while test 1 -eq 1
do
echo pq | tmadmin -r 2> /dev/null >>
domainlog
sleep 5
done

39
Automated Monitoring
Quest - Foglight
Tivoli
BMC Patrol
40
Troubleshooting
Validate that all servers and services are
still running
Check whether or not some additional
servers need to be loaded (according to
the workload)
Check the ULOG files for error messages
Check the status of the database the
application is working with
41
Troubleshooting (contd)
Check whether the limit of IPC resources
has been reached
Check the free space status of each
involved directory
Check for network problems
Check whether there are enough BEA
TUXEDO licenses

42
Tuning
How do you know you have a problem?
Record normal baseline performance
metrics
Change a configuration value and then
test
Use a load tester (i.e. Mercury
LoadRunner)
Monitor transactions per min, CPU,
memory, and disk utilization
Compare results to baseline metrics
Readjust performance metric if necessary
43
PSAPPSEVR Memory
Application Memory Use
Users Per
Process
CRM 100-150 MB 50 Users
HRMS 100-150 MB 30 Users
Supply Chain 300-500 MB 10-20 Users
Financials 150-300 MB 20-30 Users
* PeopleSoft recommends using these values for estimating memory usage
44
PSAPPSERVR Memory
Windows 2000
Add up the Memory Usage and Virtual Memory
Usage columns
This is not an absolute value because some processes
are shared
Should not exceed 70% of real memory
Check page/sec for page swapping
Should be below 100 hard page swapping / sec
UNIX
Use ps_chk_domain (search for Chistopher
Che)
Use vmstat and check po and pi columns for
paging
45
Recycle Count
The AppServer caches panel objects to
memory
If memory usage gets too high swapping
occurs
Set Recycle Count = 2000
Check for memory swapping
To minimize the cost of re-caching, set
EnableServerCaching=2
46
Timeouts
Timeout during a PIA transaction
Transaction fails and must be resubmitted
Values should be staged
Timeout between PIA transaction
User has been idle too long
Timeout that reserves a resource until
expiration time
47
Application Server Tips
Adjust timeout parameters in order
Apache Jserv Jolt Servlet PIA App
service
httpd.conf
zone.properties
psappsrv.cfg
configuration.properties
pstools.properties

psappsrv.cfg
48
Application Server Tips
tmadmin
PrintQueue (pq)


PrintServer (psr)
49
Application Server Tips
Tuxedo settings
Message size, queue, segment, semaphore
Win2k: Control Panel-> Bea Tux-> IPC Resources
UNIX: kernel prefix with msg*,sem*,shm*
50
Application Server Tips
OS tuning - NT/Win2K
TcpTimedWaitDelay, MaxUserPort
OS tuning - UNIX
UNIX: kernel prefix with tcp*
Process priority, increase load page size
Jolt handler
Max handlers * Max Clients per Handler >= # of
active users
51
High Availability (HA)
and
Clustering
52
HA and Clustering Topics
PeopleSoft Internet Architecture (PIA) and
Clustering
Infrastructure
Webserver Cluster
WebLogic
WebSphere
Application Server Cluster
Database Server Cluster
Example Clustered Environment
53
Terms
High Availability
A system designed to avoid loss of service
Continuous Availability
Non-stop service
Fault Tolerance
Method for achieving very high levels of
availability

54
Downtime
Is 99.9% good enough?
20,000 incorrect drug prescriptions per year
500 incorrect surgical operations each week
3,500 US babies dropped at delivery each
year
22,000 checks deducted from the wrong
bank accounts each hour
Your heart fails to beat 32,000 times each
year
55
Clustering and HA
Set target for Quality of Service (QOS)
Scalability (i.e hundreds of thousands of
users)
99.9% availability
Clustering method
Load balance
Replication
OS clustering
OS clustering
Used only in DB Tier
56
Application
server
DB
server
Brows
er
HTTP/HTTPS Jolt/Tuxedo
SQL
Web
server
PIA Clustering
Also consider
Scalability
Availability
Security

Sticky Sessions Load
Balance
Load Balance OS Cluster or
Replication
57
DMZ
Internet
VLAN1
VLAN3
VLAN2
Router Router
Lan1
Lan2
UID
Ext
Int
hp proliant DL360g3
0 1
Lan1
Lan2
UID
Ext
Int
hp proliant DL360g3
0 1 Clustered
VIP Address
Load
Balancer
VLAN4
Lan1
Lan2
UID
Ext
Int
hp proliant DL360g3
0 1
Lan1
Lan2
UID
Ext
Int
hp proliant DL360g3
0 1
Web Servers
Web Servers
Clustered
VIP Address
hp proliant DL560
hp proliant DL560
hp proliant DL740 1 2 3 4 5
Router
Corporate
Net
App
Server1
App
Server2
Database
Server
Load
Balancer
Firewall
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
OPEN
RAID
Storage
Redundant Architecture
58
Infrastructure
Internet Access
BGP 4 (Border Gateway Protocol)
FatPipe
Routers
Enable Packet Filtering, Port Blocking, etc for security
Firewalls - Load balanced
Remove routing and forwarding if firewall is multi-homed
Hardware Load Balancer for scalability
Reverse Proxy Server
Microsoft ISA
Storage
RAID 0+1 (striping and mirror)
RAID 5
AutoRAID
59
Webserver Cluster
Use hardware load balancer
Cisco LocalDirector
Enable sticky generic
Sticky timeout greater than PeopleSoft
session
Create a http probe
Monitor syslog for failure
Instances can use different ports
Maximum scalability
Minimum upgrade issues
60
Application Server Cluster
Multiple Processes per domain
Load balance to domain random
Load balance to process round robin
Edit configuration.properties
Comma separated list of JOLT hosts:port
psserver=AppHost1:9000,AppHost1:9100,App
Host2:9000,AppHost2:9100
61
Database Cluster
Replication
Lots of choices
Database clustering
Very complicated and expensive
Requires O/S Lock Manager
OS Clustering
AIX HACMP
Compaq TruCluster
HP Service Guard
Solaris Sun Cluster
Windows MSCS
62
HA Key Points
Plan redundant infrastructure on paper
and then test in a lab
Justify each redundant item
Keep it simple
Some equipment may not require redundancy
Avoid O/S clusters
Continue to make backups
Learn how troubleshoot the system
Use monitoring tools
63
Questions / Discussion