IBM InfoSphere DataStage Data Flow and Job Design

InfoSphere DataStage Parallel Framework Standard Practices

Information Server: Installation and Configuration Guide
Section 1 - Configuration (6%)
Describe how to properly configure DataStage v8.5
Identify tasks required to create and configure a project to be used for v8.5 jobs
Given a configuration file, identify its components and its overall intended purpose
Section 2 - Metadata (6%)
1. Demonstrate knowledge of Orchestrate schema
2. Identify the method of importing, sharing, and managing metadata
3. Demonstrate knowledge of runtime column propagation
Section 3 - Persistent Storage (10.5%)
Explain the process of importing/exporting data to/from framework (e.g., sequential file, external
source/target)
Describe proper use of a sequential file
1. Describe proper usage of FileSets and DataSets
2. Describe use of FTP stage for remote data
3. Describe use of restructure stages (e.g., column import/export)
4. Identify importing/exporting of XML data
5. Section 4 - Parallel Architecture (9%)
1. Demonstrate proper use of data partitioning and collecting
2. Demonstrate knowledge of parallel execution
3. Section 5 - Datatbases (9%)
1. Demonstrate proper selection of database stages and database specific stage properties
2. Identify source database options
3. Demonstrate knowledge of target database options
4. Section 6 - Data Transformation (12%)
1. Demonstrate knowledge of default type conversions, output mappings, and associated
warnings
2. Demonstrate proper selections of Transformer stage vs. other stages
3. Describe Transformer stage capabilities (including: stage variables, link variables,
DataStage macros, constraints, system variables, link ordering, @PART NUM, functions
4. Demonstrate the use of Transformer stage variables (e.g., to identify key grouping
boundaries on incoming data)
5. Identify process to add functionality not provided by existing DataStage stages. (e.g.,
wrapper, BuildOps, user def functions/routines)
6. Demonstrate proper use of SCD stage
7. Demonstrate job design knowledge of using RCP (modify, filter, dynamic transformer)
8. Demonstrate knowledge of Transformer Stage input and output loop processing (e.g.,
LastRecord(), LastRowInGroup(), SaveRecord(), etc.)
Section 7 - Job Components (12%)
1. Demonstrate knowledge of Join, Lookup and Merge stages
2. Demonstrate knowledge of SORT stage
3. Demonstrate understanding of Aggregator stage
4. Describe proper usage of change capture/change apply
5. Demonstrate knowledge of Real-time components
Section 8 - Job Design (9%)
1. Demonstrate knowledge of shared containers
2. Describe how to minimize Sorts and repartitions
3. Demonstrate knowledge of creating restart points and methodologies
4. Demonstrate proper use of standards
5. Explain the process necessary to run multiple copies of the source (job multi-instance)
Section 9 - Monitor and Troubleshoot (7%)
Demonstrate knowledge of parallel job score
1. Identify and define environment variables that control DataStage v8.5 with regard to
added functionality and reporting
2. Given a process list, identify conductor, section leader, and player process
3. Identify areas that may improve performance (e.g., buffer size, repartitioning, config
files, operator combination, etc.)
4. Demonstrate knowledge of runtime metadata analysis and performance monitoring
Section 10 - Job Management and Deployment (10.5%)
Demonstrate knowledge of advanced find
Demonstrate knowledge and the purpose of impact analysis
Demonstrate knowledge and purpose of job compare
Articulate the change control process
Source Code Control Integration
Section 11 - Job Control and Runtime Management (6%)
1. Demonstrate knowledge of message handlers
2. Identify the use of dsjob command line utility
3. Demonstrate ability to use job sequencers (e.g., exception hunting, re-startable,
dependencies, passing return value from routing, parameter passing and job status)