Generate IIDR Replication Mappings as

Lineage in Information Governance Catalog

Elmer Cecilio
ececilio@ca.ibm.com

Last Update: April 5, 2016

© 2015 IBM Corporation
Background

Step by step guide on how to add CDC/IIDR subscription mappings into Information
Governance Catalog

Tested on IIDR 11.3.3 and InfoServer 11.5, 11.3 FP1 on Linux. Tested on Windows with
modifications required to Path and File locations.

Uses XML export of IIDR Subscription Mapping from Management Console

Requires the use of DataStage to run jobs that will create the Extension Mappings

This guide currently works with an IIDR setup of database source and target. Need to
investigate further for flat files and JMS targets.

Supports regular column mapping, Derived Columns, Derived Expressions, Default Value,
Journal Control columns

Link to sample subscription xml export and database tables in isx format in Appendix A.

Instructions on how to use the steps with a Sequence job in Appendix B.

Latest version of this slide and other assets found on this link:
https://ibm.box.com/s/09yuentvrems5shuzqwf382a21znez3m

2 © 2015 International Business Machines Corporation

Step 1: Export IIDR Subscription Mapping

Open IIDR Management Console

Export the IIDR Subscription Mapping stored in XML format.

3 © 2015 International Business Machines Corporation

Step 2: Create Server folders/directories

On the IS server, using a telnet utility like Putty, login as dsadm.

Create the folders.

• Create a high level folder. Used CDC in my example.
• Create 3 subfolders called temp, sub_export and mappings under CDC folder.

These folders are used extensively in the DataStage jobs.

4 © 2015 International Business Machines Corporation

Step 3: FTP/Copy XML Export into server

FTP/Copy the IIDR XML export into the sub_export folder

5 © 2015 International Business Machines Corporation

Step 4: Import DataStage DSX jobs

From DataStage Designer Import the LineageBackup.dsx

file

Here is a Box download link to DSX file -

https://ibm.box.com/s/hkciievjvdu0ahbz2kkjj9hvj13dxwf3

Will import 4 DataStage jobs. 3 Parallel jobs,

A01_GetDerivedColumns, B01_ProcessIIDRXML,
C01_Restructure. 1 Sequence job.

Jobs will be created under a folder called CDC.

Note: imported jobs need to be compiled prior to use.

Sequence job will automate the running of the jobs.

Check Appendix B for additional setup.

6 © 2015 International Business Machines Corporation

Step 5: Import XSD into DataStage Shema Library (Optional)

This step is optional since the Heirarchical stage of imported DataStage job will
contain the XML schema.

Import the IIDRExport.xsd into DataStage Schema Library.

I created a schema folder called CDC then Imported the XSD using Import New
Resource.

Box download link to XSD -

https://ibm.box.com/s/eo4gn4vbm0heo4aai1lg42ruz5kqsjnm

7 © 2015 International Business Machines Corporation

Step 6: Run A01_GetDerivedColumns

Run A01_GetDerivedColumns

This job will parse XML and get all the Derived Columns with their expressions.

Output will be used as Lookup for the next job.

Needs 2 parameters, full path to subscription XML, target path which is the
main folder you created in Step 4.

8 © 2015 International Business Machines Corporation

Step 7: Run B01_ProcessIIDR_XML

Run B01_ProcessIIDR_XML job.

This job will take the XML and create a temporary mapping file (used as input to next job) and a dsjob
script.

A temporary mapping file is created for each source to target mapping.

Will lookup for any Derived Columns output from previous job.

Uses additional parameter for CDC Subscription Name and DataStage project name.

DataStage project name will be used on script generation.

9 © 2015 International Business Machines Corporation

Step 7: (cont)

First output of B01_ProcessIIDR_XML job is a temporary mapping file for each
table in an IIDR subscription. Temp files stored in the temp folder.

Second output is a script called dsjob_script.sh that will automate the calling of
the 3rd job C01_Restructure.

10 © 2015 International Business Machines Corporation

Step 8: Run dsjob_script.sh

Script will be located in the main folder as defined in the Target_Path

parameter when you ran B01_ProcessIIDR_XML.

While logged in as dsadm in Putty, cd to $DSHOME/bin.

For first time run, you may need to make the script executable using similar
command “chmod +x /opt/IBM/home/dsadm/demo/CDC/dsjob_script.sh”

Run the dsjob_script.sh as shown in screen print below. Your path may vary.

11 © 2015 International Business Machines Corporation

Step 9: Verify Mapping output

The C01_Restruture job will take the temporary mapping files in the temp folder
and create the Extension Mapping ready file in the mappings folder.

Sample image of the Extension Mapping

12 © 2015 International Business Machines Corporation

Step 10: Import table metadata into IGC

Import the source and target database metadata information into IGC.

You can use either InfoSphere Metadata Asset Manager (IMAM) or the Import
Table Definition – Start Connector Import Wizard in DataStage.

Link to IMAM import -

https://www.ibm.com/support/knowledgecenter/SSZJPZ_11.5.0/com.ibm.swg.i
m.iis.mmi.doc/topics/t_importing_and_sharing.html?lang=en

13 © 2015 International Business Machines Corporation

Step 11: Verify import in IGC

Open IGC and Search/Browse for the tables you imported.

Take note of the Host, Database and Schema information. You will need this
info during the import of the extension mapping.

14 © 2015 International Business Machines Corporation

Step 13: Import Extension Mapping

Open IGC from a browser. If using a client machine, you may need to FTP the
Extension Mappings from server into your client machine.

Under Information Assets, Import Extension Mapping Document

15 © 2015 International Business Machines Corporation

Step 13: Import Extension Mapping (cont)

Browse for File – click on Next

You can select all the mapping

files and import all at once.

Provided they share the same

source and target context.

Specify the context for source

and target assets. This is the
host, database and schema
you took note of in Step 11.

16 © 2015 International Business Machines Corporation

Step 13: Import Extension Mapping (cont)

Add any default properties.

Click Import

Import message.

17 © 2015 International Business Machines Corporation

Step 14: Verify Lineage

Open the Lineage for your table.

Success!

18 © 2015 International Business Machines Corporation

Step 15: Check mapping expressions (Optional)

Optional step to verify the column mappings.

Open details of your Extension Mapping

You should see a Mapping for each column.

When you hover to column you will see the derived expressions, default
values, journal control columns.

19 © 2015 International Business Machines Corporation

Step 15: Check mapping expressions (cont)

Regular Mapping

Journal Control Column

20 © 2015 International Business Machines Corporation

Step 15: Check mapping expressions (cont)

Derived Expression

Default Value goes to Rule. I

also specify Default Value under
Description

21 © 2015 International Business Machines Corporation

Step 15: Check mapping expressions (cont)

Derived Column. Derived Column

Expression goes to Function.

Added Derived Column under

Description

22 © 2015 International Business Machines Corporation

Thank you

You can email me at ececilio@ca.ibm.com for any comment/feedback on how it

works for your environment.

The IIDR output XSD used in this guide was for 11.3.3 version.

DataStage jobs tested on Information Server 11.5 and 11.3 FP1

The extension mapping format has not changed, so the steps outlined should
still work previous IS versions.

23 © 2015 International Business Machines Corporation

Appendix A: Sample assets for testing

If you want to test this yourself, you can download an export of a sample
subscription as well as the an isx archive format of the metadata assets from
this link:

https://ibm.box.com/s/09yuentvrems5shuzqwf382a21znez3m

24 © 2015 International Business Machines Corporation

Appendix B: Job Sequence

Added a job sequence to automate the steps. I suggest running the jobs at
least once as instructed, to ensure that the jobs run on their own.

The last stage in the Sequence job calls an external script that calls the
dsjob_script.sh

You may have to modify the script to match the DSHOME/bin folder and path
location.

25 © 2015 International Business Machines Corporation

© Copyright IBM Corporation 2016. All rights reserved. The information contained in these materials is provided for informational purposes
only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use
of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any
warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement
governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in
all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole
discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any
way. IBM, the IBM logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United
States,
26 other countries or both. Other company, product, or service names may be trademarks or service marks
© 2015 of others.
International Business Machines Corporation