dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

Project: db FEEDS
Stage: Release 1
Phase: Design
Work stream: Interfaces
Document: Performance Tuning Guide
Subject: Informatica – Performance tuning

Prepared by: Rajesh Narayanan

Date: 31/December/2004
Reference:
Version: Draft

Page 1 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

Version History:
Version Date By Changes

0.1 28/December RN First draft

0.2 29/December RN Added UNIX portion (section 5)
0.3 07/Jan RN Merged section2.2/2.3 from development
standards.

Contributors:
Name Role Location Remarks

Approval:
Name Role Location Remarks
Sanjay De Costa
Joseph Houben

Reference Documents:
Name Author Version Date
Unix system (OS) tuning Informatica

Page 2 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

CONTENTS

Page 3 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

1 Document Description
This document describes the practices that can be followed by the ETL development
team, in order to get the best of Informatica PowerCenter (ETL). This document
mainly concentrates on optimising the performance of core ETL. In order to make
ETL to achieve the optimal performance; it is imperative to strike a good balance in
hardware, OS, RDBMS & Informatica PowerCenter 7.1.1. This document can be
used as reference by the development team & administration team.

2 Document Organisation
This document is divided into following parts
o Primary guidelines - Necessary for ETL to perform optimally,
fundamental approach for ETL design with Informatica PC 7.1.1
o Advanced guidelines - Guidelines can be applied on case-to-case basis,
Can be followed based on the problem scenario / environment
o Optimising Unix system – Performance tuning the OS (Unix/Linux
system)

3 Informatica PC Primary guidelines

3.1 Database utilisation

Utilise database for significant data handling operations, staging tables can be a
real benefit for parallelism in operations. It reduces the amount of processing time
by a significant amount.

3.2 Localisation
Try to localise the relational objects as far as possible. Try not to use synonyms for
remote database. Usage of remote links for data processing & loading certainly
slow the things down.

3.3 Removal of Database driven Sequence Generators

Usage of database oriented sequence generators proves to be a costly decision.
As this requires wrapper function / store procedure call, which might degrade the
performance by 3 times. Also it is not so easy to determine the bottleneck caused
by database sequence generators. If it is must to use database sequence
generators, then have a shared sequence generator & build a staging table from

Page 4 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

the flat file, add a SEQ_ID column & call a POST TARGET LOAD procedure to
populate this column.

This requires a wrapper function / stored procedure call. Utilizing these stored
procedures has caused performance to drop by a factor of 3 times. This slowness is
not easily debugged - it can only be spotted in the Write Throughput column. Copy
the map, replace the stored proc call with an internal sequence generator for a test
run - this is how fast you COULD run your map. If we must use a database
generated sequence number, then follow the instructions for the staging table
usage. If we're dealing with GIG's or Terabytes of information - this should save you
lot's of hours tuning. IF YOU MUST - have a shared sequence generator, then build
a staging table from the flat file, add a SEQUENCE ID column, and call a POST
TARGET LOAD stored procedure to populate that column. Place the post target
load procedure in to the flat file to staging table load map. A single call to inside the
database, followed by a batch operation to assign sequences is the fastest method
for utilizing shared sequence generators.

3.4 Switch off the “Collect performance statistics”

This has an impact though it is minimal; removing this operation reduces reliance
on the flat file operations. However it may be useful to have this option switch ON
during tuning exercise.

3.5 Switch off the verbose logging

The session log has a tremendous impact on the overall performance of a session.
Over-ride the session log to NORMAL logging mode. In informatica logging
mechanism is not parallel; it is embedded into the operations. Also, this prevents
informatica metadata table from growing. Also, it is good idea to perform some
amount of automated house keeping which truncates the log from Informatica
metadata at regular intervals.

3.6 Utilise staging

If the source is flat file utilise a staging table. This way you can use SQL Loader,
Bulk-Load utility. Keep the basic logic in source load map; eliminate all lookups
from the code. At this juncture if the reader is slow, then check for following
o If there is an item in configuration file which sets a value to throttle reader,
it will limit the read throughput.
o Move flat file to local disk; don’t read from network or from RAID.

Page 5 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

3.7 Eliminate non-cached lookups
Usage of non-cached lookups will hamper the performance significantly. Especially
if the lookup table is “growing” or “updated” target table. This show the indexes are
changing during the operation and optimizer looses the track of index & its
statistics. If possible use staging table - this allows using joiner also which can
increase the performance to large extent.

3.8 Tune the database

Estimate for small, medium and large source data set sizes, in terms of number of
rows / average bytes per row. Also estimate the throughput for each and
turnaround time for load. DBA should be provided with this information, along with
tables that are expected to be high read / write .DBA should assign the right table
to the right disk space that could make difference.

3.9 Availability of SWAP & TEMP space on PMSERVER

Having less disk space for SWAP & TEMP could potentially slow down the
performance of entire server. To monitor this one need to watch the disk space
while sessions are running. Without monitoring, it would be difficult to assess the
reason, especially if mapping contains Aggregates, or lookups that uses disk
cache or Joiner with heterogeneous sources.

3.10 Session Settings

Major chunk of tuning can be done in session. By switching on the “Collect

performance statistics” one will come to know the parameters to be set in session
level, or at least what has to be changed in database. Basically one should try to
achieve OPTIMAL READ, OPTIMAL THROUGHPUT and OPTIMAL WRITE. Over-
tuning one of these pieces can ultimately slow down the sessions.

Index Cache and Data cache are dynamically allocated first. As soon as the
session is initialised, the memory for data and index caches are setup. Their sizes
depend upon session settings

The Reader DTM also based on dynamic allocation algorithm, it uses the memory
available in chunks. Size of the chunk would be determined by the session setting
“Default Buffer block size”

Page 6 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

Read the session throughput, then tune for the reader, see what the settings are,
and send the write output to a flat file for less contention. Check the Throttle reader
setting; increase the default buffer size by a factor of 64K each shot. If the reader
still appears to increase during the session, then stabilize, and then try increasing
Shared Session Memory from 12 MB to 24 MB. Check the writer throughput
performance statistics to make sure there is NO writer bottleneck. If you have slow
writer, change the map to single target table at a time to see which target is
causing the slowness and tune it.

NOTE: if the reader session to flat file just doesn't ever "get fast", then we have got
some basic map tuning to do. Try to merge expression objects, set the lookups to
unconnected (for re-use if possible), check the Index and Data cache settings if we
have aggregation, or lookups being performed. Etc... If we have a slow writer,
change the map to a single target table at a time - see which target is causing the
"slowness" and tune it. Make copies of the original map, and break down the
copies. Once the "slower" of the N targets is discovered, talk to DBA about
partitioning the table, updating statistics, removing indexes during load, etc...
There are many database things you can do here.
Remember the TIMING is affected by READER/TRANSFORMER/WRITER
threads. With complex mappings, don't forget that each ELEMENT (field) must be
weighed - in this light a firm understanding of how to read performance statistics
generated by Informatica becomes important. In other words - if the reader is
slow, then the rest of the threads suffer, if the writer is slow, same effect. A pipe is
only as big as its smallest diameter.... A chain is only as strong as its weakest
link. Sorry for the metaphors, but it should make sense.

3.11 Remove all other applications on PMServer

Except the database staging, PMServer plays well with RDBMS & its engine, but
doesn’t play well with application servers, in particularly JAVA Virtual Machines,
Web Servers, Security Servers, applications and Report Servers. All of these items
should be broken out to other machines; this is critical to improve performance on
PMServer machine.

3.12 Removal external registered modules

As far as possible, try to avoid the API’s which calls external objects, as this has
been proven slow. External modules might exhibit speed problems, instead try
using pre-processing / post processing with SED, AWK or GREP.

Page 7 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

4 Informatica PC Advanced guidelines

4.1 Filter Expressions :

Create the filter (TRUE / FALSE) inside the port expression upstream. Complex
filter expressions slow down the mapping. However it acts faster in Expression
transformation with an output port for the result. Place the expression in
EXPRESSION Transformation upstream from filter. Compute a single numerical
flag: 1 for TRUE 0 for FALSE as output port. Push this data into the filter. This will
have positive impact on performance.

Use the Filter transformation early in the mapping.

To maximize session performance, keep the Filter transformation as close as
possible to the sources in the mapping. Rather than passing rows that you plan to
discard through the mapping, you can filter out unwanted data early in the flow of
data from sources to targets.
Use the Source Qualifier to filter
The Source Qualifier transformation provides an alternate way to filter rows. Rather
than filtering rows from within a mapping, the Source Qualifier transformation filters
rows when read from a source. The main difference is that the source qualifier limits
the row set extracted from a source, while the Filter transformation limits the row set
sent to a target. Since a source qualifier reduces the number of rows used
throughout the mapping, it provides better performance.

4.2 Remove Default’s:

Having a default value including “ERROR” slows down the session. It causes
unnecessary evaluation of values for every data element in the mapping. Best
method of allotting default value is to have variable in expression, which returns the
expected value on the condition. This will be faster than assigning default value.

4.3 Output port instead of Variable port

Variables are good for static and state driven, but slow down the performance time
as they are allocated each time a row passes through expression object. Try to use
Output port instead of variable port.

Page 8 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

4.4 Datatype conversion:
Avoid performing implicit conversion of datatypes by connecting an Integer to string
or vice versa. Instead use the function that converts the data explicitly, this avoids
PMServer to decide on datatype conversion at run time.

4.5 String Functions:

String functions are costly on performance. E.g. ltrim, rtrim etc., as there involves
allocate & re-allocate of memory within READER thread. Also it would be imperative
to perform the string operations on the data, in which case following can be
considered.
Use varchar/varchar2 datatypes in database sources, if source is file then make it
delimited one. Try to use LTRIM/RTRIM functions on the data coming in from a
database SQL; this would be much faster than performing in ETL.

4.6 IIF Conditions caveat

As far as possible, make a logic that goes away from IIF, as IIF conditions are costly
in any language. IIF creates multiple path logic inside the application & uses the
decision to navigate. This might have an implication on performance as well. Other
option is to use Oracle DECODE in source qualifier.

4.7 Expressions
Expressions like IS_SPACES, ISNUMBER etc. affects the performance, as this is
the data validation expression that has to scan the entire string to determine the
result. Try to avoid using these expressions unless there is absolute requirement for
its usage.

4.8 Update Expressions for session :

In session if the option Update Else Insert is ON, then definitely performance will
slow down. As, Informatica has to performs 2 operations for each rows update w.r.t
PK, then if it returns 0 rows then perform Insert. As an alternative, Update Strategy
can be used where rows would be marked using DD_UPDATE or DD_INSERT
inside the mapping. In this case session settings can be INSERT & UPDATE AS
UPDATE or UPDATE AS INSERT.

4.9 Multiple targets / sources are too slow :

Mappings with Multiple targets can eat up the performance some time. If the
architecture permits then make one map per target. If the sources are from different

Page 9 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

ftp locations & they are flat file, then ideal choice would be FTPing the file to source
to the ETL server & then process it.

4.10 Aggregator
If the mapping contains more than one aggregators, then the session will run slow,
unless the cache dir is fast & disk drive access speed is high. Placing aggregator
towards the end might be another option; however this will also bring down the
performance. As all the I/O activity would be a bottleneck in informatica.
Maplets are good source for replicating data logic, but if a maplet contains
aggregator still the performance of the mapping (that contains maplet) will affect.
Reduce the number of aggregators in the entire mapping to 1(if can), if possible,
split the mapping to several mappings for breaking down the logics.
Sorted input to aggregator will increase the performance to large extent, however if
the sorted input is enabled & the data passing to aggregator is not sorted, Session
will fail. Set the cache size to calculated amount using below mentioned formulae.
Index size = (sum of column size in group-by ports + 17) X number of groups
Data size = (sum of column size of output ports + 7) X number of groups

4.11 Joiner
Perform joins in a database. Performing a join in a database is faster than
performing a join in the session.
Use one of the following options:
o Create a pre-session stored procedure to join the tables in a database.

o Use the Source Qualifier transformation to perform the join.

Designate as the master source the source with the smaller number of records.
For optimal performance and disk storage, designate the master source as the
source with the lower number of rows. With a smaller master source, the data cache
is smaller, and the search time is shorter. Set the cache size to calculated amount
using below mentioned formulae.
Index size = (sum of master column size in join condition +16) X number of rows in
master table
Data size = (sum of master column size NOT in join condition but on output ports +
8) x number of rows in master table

Page 10 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

4.12 Lookups
When caching is enabled, the PowerCenter Server caches the lookup table and
queries the lookup cache during the session. When this option is not enabled, the
PowerCenter Server queries the lookup table on a row-by-row basis.
Eliminate too many lookups. More the lookups means, the DTM
reader/writer/Transform threads are not left with enough memory to be able to run
efficiently (as it can). With too many lookups one need to trade in Memory
contention for Disk contention. The memory contention might be worse than disk
contention, as the OS ends up swapping in and out of TEMP/SWAP disk space, with
small block sizes to try to locate the lookup row, and as the row goes from lookup to
lookup, swapping becomes worse.
Both lookups and aggregators require memory space & each of them requires Index
& Data cache, ideally they share from the same heap segments. Hence, care should
be taken while designing mapping that consumes the memory.
In the case where a lookup uses more than one lookup condition, set the conditions
with an equal sign first in order to optimize lookup performance. In the case of a
cached lookup, an ORDER BY condition is issued in the SQL statement used to
create the cache. Columns used in the ORDER BY condition should be indexed.
The session log will contain the ORDER BY statement.
Tips on Caches:
Cache small lookup tables
Improve session performance by caching small lookup tables. The result of the
lookup query and processing is the same, whether or not you cache the lookup
table.
Use a persistent lookup cache for static lookup tables:
If the lookup table does not change between sessions, configure the Lookup
transformation to use a persistent lookup cache. The Informatica Server then saves
and reuses cache files from session to session, eliminating the time required to read
the lookup table. If the lookup table does not change between sessions, configure
the Lookup transformation to use a persistent lookup cache. The Informatica Server
then saves and reuses cache files from session to session, eliminating the time
required to read the lookup table.
Override the ORDER BY statement for cached lookups:
By default, the Informatica Server generates an ORDER BY statement for a cached
lookup that contains all lookup ports. To increase performance, you can suppress

Page 11 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

the default ORDER BY statement and enter an override ORDER BY with fewer
columns
Place conditions with an equality operator (=) first.
If a Lookup transformation specifies several conditions, you can improve lookup
performance by placing all the conditions that use the equality operator first in the
list of conditions that appear under the Condition tab
Consider following for calculating caches for lookup

Attributes Method

Minimum Index Cache 200 * [( S column size) + 16] over all condition ports

Maximum Index Cache # rows in lookup table [(S column size) + 16] * 2
over all condition ports

Minimum Data Cache # rows in lookup table [(S column size) + 8] over all
outputports (not condition port)

Maximum Data Cache 2 * minimum data cache

4.12.1 Lookups & Aggregators Fight.

The lookups and the aggregators fight for memory space as discussed above. Each
requires Index Cache, and Data Cache and they "share" the same HEAP segments
inside the core. Particularly in the 4.6 / 1.6 products and prior - these memory areas
become critical, and when dealing with many rows - the session is almost certain to
cause the server to "thrash" memory in and out of the OS Swap space. If possible,
separate the maps - perform the lookups in the first section of the maps, position the
data in an intermediate target table - then a second map reads the target table and
performs the aggregation (also provides the option for a group by to be done within the
database)... Another speed improvement...

4.13 Maplets for complex Logic

It’s good idea to break the complex logic into maplets. This allows managing the
mapping in much better & efficient way of breaking down the business logics.
Always remember shorter the distance between source and Target, better the
performance.
With complex mappings READER,WRITER & TRANSFORM threads affects by
timing, i.e. if the reader is slow, then rest of the threads suffer,similarily f he writer is
slow same is the effect.

Page 12 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

4.14 Database IPC settings & Priorities
If PMServer & Oracle instance are running on the same server, use IPC connection
instead of TCP/IP connection. Try to change the protocol in the TNSNames.ORA
and Listener.ORA files, and restart the listener on the server .However this can be
used only locally, at the same time the speed increases between 2x and 5x .Another
option one can think is prioritizing the database login that informatica uses to
execute its task. These tasks when login to the database would override others. This
would be particularly helpful in increasing the performance especially when bulk
loader or SQL Loader is used.

4.15 Loading
Make sure indexes and constraints are removed before loading into relational
targets & this can be created as soon as the load is completed. This would help in
boost up the performance in bulk data loads.

Lesser the commit interval more the time for session to complete, set the
appropriate commit interval, anything above 50K is good. Partioning the data while
loading is another wise option. Following are the partitions Informatica provides
o Key Range

o Hash Key

o Round Robin

o Pass Through

When partitioning the individual transformation it is advisable to go for the following

o Aggregator Cache Use hash auto key

o Lookup Cache Use hash auto key partition type, equality condition

o Sorter Cache Use Hash auto key or Pass-through or Key range

4.16 Memory Settings

Session Shared Memory Size controls the total amount of memory used to buffer
rows internally by the reader and writer. Set session shared memory between 12MB
and 25MB, remember increasing the shared memory beyond this doesn’t guarantee
increase in performance rather it does decrease the performance.
Buffer Block Size controls the size of the blocks that move in the pipeline. Set
shared buffer block size around 128K, This would be used by informatica for
handling block of rows.

Page 13 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

If the server has RAM over 12 GIG’s, then shared memory can be increased
between 1 and 2 GIG’s. Also the shared buffer block size should set relative to
shared memory settings.
The Informatica Server moves data from sources to targets based on workflow and
mapping metadata stored in a repository. A workflow is a set of instructions that
describes how and when to run tasks related to extracting, transforming, and
loading data. The Informatica Server runs workflow tasks according to the
conditional links connecting the tasks.

4.17 Reduce Number of OBJETS in a map

Frequently, the idea of these tools is to make the "data translation map" as easy as
possible. All too often, that means creating "an" (1) expression for each
throughput/translation (taking it to an extreme of course). Each object adds
computational overhead to the session and timings may suffer. Sometimes if
performance is an issue / goal, integrate several expressions in to one expression
object, thus reducing the "object" overhead. In doing so – it could speed up the map.

4.18 Slow Sources - Flat Files

If we've got slow sources, and these sources are flat files, we can look at some of the
following possibilities. If the sources reside on a different machine, and we've opened a
named pipe to get them across the network - then we've opened (potentially) a can of
worms. We’ve introduced the network speed as a variable on the speed of the flat file
source. Try to compress the source file, FTP PUT it on the local machine (local to
PMServer), decompress it, and then utilize it as a source. If you're reaching across the
network to a relational table - and the session is pulling many rows (over 10,000) then
the source system itself may be slow. We may be better off using a source system
extract program to dump it to file first, and then follow the above instructions. However,
there is something your SA's and Network Ops folks could do (if necessary) - this is
covered in detail in the advanced section. They could backbone the two servers
together with a dedicated network line (no hubs, routers, or other items in between the
two machines). At the very least, they could put the two machines on the same sub-net.
Now, if the file is local to PMServer but is still slow, examine the location of the file (which
device is it on). If it's not on an INTERNAL DISK then it will be slower than if it were on
an internal disk (C drive for you folks on NT). This doesn't mean a UNIX file LINK exists
locally, and the file is remote - it means the actual file is local.

4.19 Break the mappings out

Page 14 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

One per target. If necessary, 1 per source per target. Why does this work? Well -
eliminating multiple targets in a single mapping can greatly increase speed... Basically
it's like this: one session per map/target. Each session establishes its own database
connection. Because of the unique database connection, the DBMS server can now
handle the insert/update/delete requests in parallel against multiple targets. It also helps
to allow each session to be specified for its intended purpose (no longer mixing a data
driven session with INSERTS only to a single target). Each session can then be placed
in to a batch marked "CONCURRENT" if preferences allow. Once this is done,
parallelism of mappings and sessions become obvious. A study of parallel processing
has shown again and again, that the operations can be completed sometimes in half the
time of their original counterparts merely by streaming them at the same time. With
multiple targets in the same mapping, a single database connection to handle multiplies
diverse database statements - sometimes hitting this target, other times hitting that
target. Think - in this situation it's extremely difficult for Informatica (or any other tool for
that matter) to build BULK operations... even though "bulk" is specified in the session.
Remember that "BULK" means this is your preference, and that the tool will revert to
NORMAL load if it can't provide a BULK operation on a series of consecutive rows.
Obviously, data driven then forces the tool down several other layers of internal code
before the data actually can reach the database.

4.19.1 Keep the mappings as simple as possible

Bury complex logic (if you must) in to a maplet. If you can avoid complex logic all
together - then that would be the key. The old rule of thumb applies here (common
sense) the straighter the path between two points, the shorter the distance... Translated
as: the shorter the distance between the source qualifier and the target - the faster the
data loads

4.20 READER/TRANSFORMER/WRITER threads affect the TIMING

With complex mappings, don't forget that each ELEMENT (field) must be weighed - in
this light a firm understanding of how to read performance statistics generated by
Informatica becomes important. In other words - if the reader is slow, then the rest of
the threads suffer, if the writer is slow, same effect. A pipe is only as big as its smallest
diameter.... A chain is only as strong as its weakest link.

4.21 Sorting – performance issues

We can improve Aggregator transformation performance by using the Sorted Input
option. When the Sorted Input option is selected, the Informatica Server assumes all
data is sorted by group. As the Informatica Server reads rows for a group, it performs

Page 15 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

aggregate calculations as it reads. When necessary, it stores group information in
memory. To use the Sorted Input option, we must pass sorted data to the Aggregator
transformation. We can gain added performance with sorted ports when we partition the
session.
When Sorted Input is not selected, the Informatica Server performs aggregate
calculations as it reads. However, since data is not sorted, the Informatica Server stores
data for each group until it reads the entire source to ensure all aggregate calculations
are accurate.
For example, one Aggregator has the STORE_ID and ITEM Group By ports, with the
Sorted Input option selected. When you pass the following data through the Aggregator,
the Informatica Server performs an aggregation for the three records in the 101/battery
group as soon as it finds the new group, 201/battery:

STORE_ID ITEM QTY PRICE

101 ‘battery’ 3 2.99
101 ‘battery’ 1 3.19
101 ‘battery’ 2 2.59
201 ‘battery’ 4 1.59
201 ‘battery’ 1 1.99
If you use the Sorted Input option and do not presort data correctly, the session fails.

4.21.1 Sorted Input Conditions

Do not use the Sorted Input option if any of the following conditions are true:
• The aggregate expression uses nested aggregate functions.
• The session uses incremental aggregation.
• Input data is data-driven. You choose to treat source data as data driven in the
session properties, or the Update Strategy transformation appears before the
Aggregator transformation in the mapping.
• If we use the Sorted Input option under these circumstances, the Informatica
Server reverts to default aggregate behaviour, reading all values before
performing aggregate calculations.

4.21.2 Pre-Sorting Data

To use the Sorted input option, you pass sorted data through the Aggregator.
Data must be sorted as follows:
• By the Aggregator group by ports, in the order they appear in the Aggregator
transformation.
• Using the same sort order configured for the session.
If data is not in strict ascending or descending order based on the session sort order, the
Informatica Server fails the session. For example, if you configure a session to use a

Page 16 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

French sort order, data passing into the Aggregator transformation must be sorted using
the French sort order.
If the session uses file sources, you can use an external utility to sort file data before
starting the session. If the session uses relational sources, we can use the Number of
Sorted Ports option in the Source Qualifier transformation to sort group by columns in
the source database. Group By columns must be in the exact same order in both the
Aggregator and Source Qualifier transformations.

4.22 Workflow Manager

The Informatica Server moves data from sources to targets based on workflow and
mapping metadata stored in a repository. A workflow is a set of instructions that
describes how and when to run tasks related to extracting, transforming, and loading
data. The Informatica Server runs workflow tasks according to the conditional links
connecting the tasks.

• Monitor, add, edit, delete Informatica Server info in the repository

• Stop the informatica Server
• Configure database, external loader, and FTP connections
• Manage sessions and batches - create, edit, delete, copy/move within a folder,
start/stop, abort sessions, view session logs, details, and session performance
details.

Source Server Target

Source Data Transformed

data

Page 17 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

Instructions from Metadata

`
Repository

4.22.1 Monitoring and Running a Session:

When the Informatica Server runs a Session task, the Workflow Monitor creates session
details that provide load statistics for each target in the mapping. We can view session
details when the session runs or after the session completes.
Create a worklet to reuse a set of workflow logic in several workflows. Use the Worklet
Designer to create and edit worklets.

4.22.2 Informatica suggests that each session takes roughly 1 to 1 1/2

CPU's.
In keeping with this - Informatica play's well with RDBMS engines on the same machine,
but does NOT get along (performance wise) with ANY other engine (reporting engine,
java engine, OLAP engine, java virtual machine, etc...)

4.22.3 Place some good server load monitoring tools on the PM Server in
development
Watch it closely to understand how the resources are being utilized, and where the hot
spots are. Try to follow the recommendations - it may mean upgrading the hardware to
achieve throughput. Look in to EMC' s disk storage array - while expensive, it appears
to be extremely fast, it may improve the performance in some cases by up to 50%

4.23 Change Database Priorities for the PMServer Database User

Prioritizing the database login that any of the connections use (setup in Server Manager)
can assist in changing the priority given to the Informatica executing tasks. These tasks
when logged in to the database then can over-ride others. Sizing memory for these
tasks (in shared global areas, and server settings) must be done if priorities are to be
changed. If BCP or SQL*Loader or some other bulk-load facility is utilized, these
priorities must also be set. This can greatly improve performance. Again, it's only
suggested as a last resort method, and doesn't substitute for tuning the database, or the

Page 18 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

mapping processes. It should only be utilized when all other methods have been
exhausted (tuned). Keep in mind that this should only be relegated to the production
machines and only in certain instances where the Load cycle that Informatica is utilizing
is NOT impeding other users.

4.24 Change the Unix User Priority

In order to gain speed, the Informatica UNIX User must be given a higher priority. The
UNIX SA should understand what it takes to rank the UNIX logins, and grant priorities to
particular tasks. Or - simply have the pmserver executed under a super user (SU)
command; this will take care of reprioritizing Informatica's core process. This should
only be used as a last resort - once all other tuning avenues have been exhausted, or if
we have a dedicated UNIX machine on which Informatica is running.

4.25 Try not to load across the network

If at all possible, try to co-locate PMServer executable with a local database. Not having
the database local means: 1) the repository is across the network (slow), 2) the sources /
targets are across the network, also potentially slow. If we have to load across the
network, at least try to localize the repository on a database instance on the same
machine as the server. The other thing is: try to co-locate the two machines (pmserver
and Target database server) on the same sub-net, even the same hub if possible. This
eliminates unnecessary routing of packets all over the network. Having a localized
database also allows us to setup a target table locally - which you can then "dump"
following a load, ftp to the target server, and bulk-load in to the target table. This works
extremely well for situations where append or complete refresh is taking place.

4.26 Balance between Informatica and the power of SQL and the database
Try to utilize the DBMS for what it was built for: reading/writing/sorting/grouping/filtering
data en-masse. Use Informatica for the more complex logic, outside joins, data
integration, multiple source feeds, etc... The balancing act is difficult without DBA
knowledge. In order to achieve a balance, we must be able to recognize what
operations are best in the database, and which ones are best in Informatica. This does
not degrade from the use of the ETL tool, rather it enhances it - it's a MUST if you are
performance tuning for high-volume throughput.

5 Performance tuning the UNIX OS

Page 19 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

5.1 Process check
ps-axu:
Run to check for the following items:
o Are there any processes waiting for disk access or for paging, if so check
the I/O and memory subsystems.
o Processes that are using most of the CPU and Processes are using most
of the memory. This may help you distribute the workload better.

5.2 Identifiying & resolving memory problem

Run vmstat S 5 confirms memory problems and check for the following:
o Pages-outs occurring consistently? If so, you are short of memory.

o Are there a high number of address translation faults? (System V only)

This suggests a memory shortage.
o Are swap-outs occurring consistently? If so, you are extremely short of
memory. Occasional swap-outs are normal; BSD systems swap-out
inactive jobs. Long bursts of swap-outs mean that active jobs are
probably falling victim and indicate extreme memory shortage. If you don’t
have vmstat -S, look at the w and de fields of vmstat. These should
ALWAYS be zero.

If memory seems to be the bottleneck of the system, try following remedial steps:

o Reduce the size of the buffer cache, if your system has one, by
decreasing BUFPAGES. The buffer cache is not used in system V.4 and
SunOS 4.X systems. Making the buffer cache smaller will hurt disk I/O
performance.
o If you have statically allocated STREAMS buffers, reduce the number of
large (2048- and 4096-byte) buffers. This may reduce network
performance, but netstat-m should give you an idea of how many buffers
you really need.
o Reduce the size of your kernels tables. This may limit the systems
capacity (number of files, number of processes, etc.).
o Try running jobs requiring a lot of memory at night. This may not help the
memory problems, but you may not care about them as much.

Page 20 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

o Try running jobs requiring a lot of memory in a batch queue. If only one
memory-intensive job is running at a time, your system may perform
satisfactorily.
o Try to limit the time spent running sendmail, which is a memory hog.

o If you don’t see any significant improvement, add more memory.

5.3 Identifying and Resolving Disk I/O Issues

Use iostat to check i/o load and utilization, as well as CPU load. Iostat can be used
to monitor the I/O load on the disks on the UNIX server. Using iostat permits
monitoring the load on specific disks. Take notice of how fairly disk activity is
distributed among the system disks. If it is not, are the most active disks also the
fastest disks
Following might help to rectify the problem due to I/O
o Reorganize your file systems and disks to distribute I/O activity as evenly
as possible.
o Using symbolic links helps to keep the directory structure the same
throughout while still moving the data files that are causing I/O contention.
o Use your fastest disk drive and controller for your root file system; this will
almost certainly have the heaviest activity. Alternatively, if single-file
throughput is important, put performance-critical files into one file system
and use the fastest drive for that file system.
o Put performance-critical files on a file system with a large block size:
16KB or 32KB (BSD).
o Increase the size of the buffer cache by increasing BUFPAGES (BSD).
This may hurt your systems memory performance.
o Rebuild your file systems periodically to eliminate fragmentation (backup,
build a new file system, and restore).
o If you are using NFS and using remote files, look at your network
situation. You don’t have local disk I/O problems.
o Check memory statistics again by running vmstat 5 (sar-rwpg). If your
system is paging or swapping consistently, you have memory problems,
fix memory problem first. Swapping makes performance worse.

Page 21 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

If system has disk capacity problem and is constantly running out of disk
space, try the following actions:

o Write a find script that detects old core dumps, editor backup and auto-
save files, and other trash and deletes it automatically. Run the script
through croon.
o Use a smaller block size on file systems that are mostly small files (e.g.,
source code files, object modules, and small data files).

5.4 Identifying and Resolving CPU Overload Issues

Use sar u to check for CPU loading. This provides the %usr (user), %sys (system),
%wio (waiting on I/O), and %idle (% of idle time). A target goal should be %usr +
%sys= 80 and %wio = 10 leaving %idle at 10. If %wio is higher, the disk and I/O
contention should be investigated to eliminate I/O bottleneck on the UNIX server. If
the system shows a heavy load of %sys, and %usr has a high %idle, this is
indicative of memory and contention of swapping/paging problems. In this case, it is
necessary to make memory changes to reduce the load on the system server.
When you run iostat 5 above, also observe for CPU idle time. Is the idle time
always 0, without letup? It is good for the CPU to be busy, but if it is always busy
100 percent of the time, work must be piling up somewhere. These points to CPU
overload.
o Eliminate unnecessary daemon processes. rwhod and routed are
particularly likely to be performance problems, but any savings will help.
o Get users to run jobs at night with any queuing system that’s available
always for help. You may not care if the CPU (or the memory or I/O
system) is overloaded at night, provided the work is done in the morning.
o Use nice to lower the priority of CPU-bound jobs will improve interactive
performance. Also, using nice to raise the priority of CPU-bound jobs will
expedite them but will hurt interactive performance. In general though,
using nice is really only a temporary solution. If your workload grows, it
will soon become insufficient. Consider upgrading your system, replacing
it, or buying another system to share the load.

Page 22 of 23
dbFEEDS Project

Performance Tuning Guidelines– Informatica PowerCenter

5.5 Identifying and Resolving Network Issues
One can suspect problems with network capacity or with data integrity if users
experience slow performance when they are using rlogin or when they are
accessing files via NFS.
Look at netsat-I. If the number of collisions is large, suspect an overloaded network.
If the number of input or output errors is large, suspect hardware problems. A large
number of input errors indicate problems somewhere on the network. A large
number of output errors suggest problems with your system and its interface to the
network.
If collisions and network hardware are not a problem, figure out which system
appears to be slow. Use spray to send a large burst of packets to the slow system. If
the number of dropped packets is large, the remote system most likely cannot
respond to incoming data fast enough. Look to see if there are CPU, memory or disk
I/O problems on the remote system. If not, the system may just not be able to
tolerate heavy network workloads. Try to reorganize the network so that this system
isn’t a file server.
A large number of dropped packets may also indicate data corruption. Run netstat-
s on the remote system, then spray the remote system from the local system and
run netstat-s again. If the increase of UDP socket full drops (as indicated by
netstat) is equal to or greater than the number of drop packets that spray reports,
the remote system is slow network server If the increase of socket full drops is less
than the number of dropped packets, look for network errors.
Run nfsstat and look at the client RPC data. If the retransfield is more than 5
percent of calls, the network or an NFS server is overloaded. If timeout is high, at
least one NFS server is overloaded, the network may be faulty, or one or more
servers may have crashed. If badmixis roughly equal to timeout, at least one NFS
server is overloaded. If timeout and retrans are high, but badxidis low, some part of
the network between the NFS client and server is overloaded and dropping packets.
Try to prevent users from running me /O- intensive programs across the network.
The greputility is a good example of an I/O intensive program. Instead, have users
log into the remote system to do their work.
Reorganize the computers and disks on your network so that as many users as
possible can do as much work as possible on a local system. Use systems with
good network performance as file servers. If you are short of STREAMS data buffers
and are running Sun OS 4.0 or System V.3 (or earlier), reconfigure the kernel with
more buffers.

Page 23 of 23