ODI Advanced

Advanced Topics in ODI-EE: Hands on Lab
Advanced Topics in Oracle Data

Integrator: Hands-on Lab

Page 1 of 26
7/9/2010 6:08:00 PM (version 1.0)
Oracle, All rights reserved
Supported Version: Oracle Data Integrator 11.1.1.3

Objectives:
This lab will highlight to Developers, DBAs and Architects some of the best practices for
implementing techniques to improve performance for real-time data warehousing using Oracle
Data Integrator Enterprise Edition (ODI-EE).
In this lab participants will walk through the steps to enable change data capture on a source
relational table, then deploy and manage these implementations to load and transform records to
a relational target.
Prerequisite:
Basic knowledge of Oracle Data Integrator, Installed Oracle Data Integrator software and Oracle
Database instance.
S308461 - Using ODI-EE and Oracle BI for Real-Time Insight:

Hands on Lab

Page 2 of 26
7/9/2010 6:08:00 PM (version 1.0)
Table of Contents
Part 1: Configuring a Model for CDC ........................................................................... 4
Part 2: Creating a Package to Set Up CDC ................................................................. 8
Part 3: Creating the Integration Flows to Consume the Changes ............................... 13
Part 4: Run the CDC Set Up and Consumption Processes in Production................... 22
Summary .................................................................................................................. 26

Page 3 of 26
7/9/2010 6:08:00 PM (version 1.0)
Part 1: Configuring a Model for CDC

In this exercise you will configure an existing source Oracle data model to support ODI CDC
feature. The CDC mode used will be Consistent-Set CDC. Oracle Log Miner will be used to track
the changes.
1. Double-click the icon Reset_ODI.bat from the desktop. This will reset the files for all Handson Labs to the initial state.
Warning: If you have custom ODI projects in the repository on this machine, they will be
deleted.
2.
3.
Open the Oracle Data Integrator Studio using the Start Menu > Oracle > Oracle Data Integrator > ODI
Studio menu.
Go to the Designer Navigator and press Connect to Repository
4.
Select HOL Advanced from the drop-down list of logins and press OK.
5.
Open the Designer navigator menu and make sure that the Default Execution Context is set to the
Development context (not Production).

Page 4 of 26
7/9/2010 6:08:00 PM (version 1.0)
6.
Open the Models accordion in the Designer navigator by clicking on the label.
7. Right-click on the Oracle CDC Source model and select Changed Data Capture > Add to
CDC.
Click Yes to add all the tables to the CDC.
8. Double-click on the Oracle CDC Source model.

The editor for this model opens.

Page 5 of 26
7/9/2010 6:08:00 PM (version 1.0)
9. Go to the Journalizing tab.

10. Select the Consistent Set journalizing mode, click OK to close the popup window then the
JKM Oracle 10g Consistent (Log Miner) knowledge module. Set the
AUTO_CONFIGURATION KM option to Yes.
11. Go to the Journalized Tables tab, select the SRC_REGION table and click on the Remove
from CDC
button.
12. Click on the Reorganize button

to order the tables according to their foreign-keys.
Press yes on all confirmation dialogs.
The datastores are automatically organized as shown below.

Page 6 of 26
7/9/2010 6:08:00 PM (version 1.0)
Note: Consistent Set Journalizing tracks changes to a group of the model's datastores, taking into
account the referential integrity between these datastores. The group of datastores journalized in
this mode is called a Consistent Set.
13. Click Save(
) on the toolbar to save the Model.

Page 7 of 26
7/9/2010 6:08:00 PM (version 1.0)
Part 2: Creating a Package to Set Up CDC

In this exercise you will create a package that sets up the CDC infrastructure in a given context.
1. In the Designer navigator, Click on the Projects accordion.
2. Expand the Advanced Topics in ODI-EE project and then expand the First Folder folder.
3. Select the Packages node, right-click and select New Package.

A package editor opens.
4. Enter the following package name: Tutorial Part 2 - CDC Setup
5. Go to the Diagram tab.
6. Drag and drop into the diagram the Oracle CDC Source data model from the Designers
Models tree view. A new step appears in the diagram, named after your data model.

Page 8 of 26
7/9/2010 6:08:00 PM (version 1.0)

Page 9 of 26
7/9/2010 6:08:00 PM (version 1.0)
7. Click on this step. In the Properties panel:
Select Journalizing Model in the Type drop-down list
Click the Start and Add Subscribers checkboxes.

Enter CONSUMER_1 in the Subscribers field, and then click Add.
8. Click Save(
) on the toolbar to save the package.
9. Click on the Execute(
) button on the toolbar.
10. In the Execution window, select Development for the context and then press OK.

Page 10 of 26
7/9/2010 6:08:00 PM (version 1.0)
11. Press OK when the Session Started window appears.

12. Open the Operator navigator.
13. In the Operator, select the Session List tab, and expand the All Executions node.

Page 11 of 26
7/9/2010 6:08:00 PM (version 1.0)
14. Check that the last session ran correctly. You can review the steps and tasks that have
activated the CDC process.

Page 12 of 26
7/9/2010 6:08:00 PM (version 1.0)
Part 3: Creating the Integration Flows to Consume the Changes

In this exercise you will create the integration flows that will consume the changes captured using
CDC.
1. In the Designer navigator, open the Projects accordion.
3. Expand the Procedures node.
4. Right-click on the CDC Changes procedure and select Execute.

7. The procedure did the following:
1 insert in SRC_CITY.
1 insert in SRC_CUSTOMER.
1 update in SRC_CITY.
2 updates in SRC_CUSTOMER.

Page 13 of 26
7/9/2010 6:08:00 PM (version 1.0)
8. Expand the Interfaces node and double-click on Load SRC_CITY CDC.
9. Go to the Mapping tab.

10. Click on the source datastore SRC_CITY and check the Journalized Data only checkbox.
This ensures that ODI will load only the changed data (latest inserts, updates or deletes) from
the source instead of the entire source dataset.
11. Click on the filter

and modify the filter expression from JRN_SUBSCRIBER =
'SUNOPSIS' to JRN_SUBSCRIBER = 'CONSUMER_1'

Page 14 of 26
7/9/2010 6:08:00 PM (version 1.0)
12. Right-click on the filter
and select Data to review the changed data.
13. Save and close the interface.

14. Double-click on Load SRC_CUSTOMER CDC in the Interfaces node.
15. Go to the Mapping tab.
16. Click on the source datastore SRC_CUSTOMER and check the Journalized Data only
checkbox.

Page 15 of 26
7/9/2010 6:08:00 PM (version 1.0)
17. Click on the filter

and modify the filter expression from JRN_SUBSCRIBER =
'SUNOPSIS' to JRN_SUBSCRIBER = 'CONSUMER_1'.
18. Right-click on the filter
and select Data to review the changed data.
14. Click on the Models tab in the tree view.

Page 16 of 26
7/9/2010 6:08:00 PM (version 1.0)
19. Expand the Flat Files Sources model.
20. Expand the Flat Files Sources model and drag and drop the SRC_SALES_PERSON.txt
datastore from the Designers Models tree view into the source panel of the interface.
21. Create a join between the SRC_CUSTOMER table and the SRC_SALES_PERSON.txt file.
To create it, drag and drop the SALES_PERS_ID column from the SRC_CUSTOMER table
into the SALES_ID column from the SRC_SALES_PERSON.txt file.
Note: The join expression is:
SRC_CUSTOMER.SALES_PERS_ID = SRC.SALES_ID

Page 17 of 26
7/9/2010 6:08:00 PM (version 1.0)
22. We now need to map the remaining target column:

- SALES_PERS: map FIRST_NAME and LAST_NAME from the SRC_SALES_PERS.txt
file into it. The two source columns should be concatenated with an extra space between
them then we use the LTRM and RTRIM to remove any whitespaces.
Note: The mapping expression is
LTRIM(RTRIM(SRC.FIRST_NAME)) || ' ' || LTRIM(RTRIM(SRC.LAST_NAME))
23. Save and close the interface.
25. Select the Packages node, right-click and select New Package.
A package editor opens.

26. Enter the following package name: Tutorial Part 3 - Load CDC Data
27. Go to the Diagram tab.
29. Click this step. In the Properties panel:

Select Journalizing Model in the Type drop-down list.
Click the Extend Window and Lock Subscribers checkboxes.

Page 18 of 26
7/9/2010 6:08:00 PM (version 1.0)
30. Drag and drop the Load SRC_CITY CDC and Load SRC_CUSTOMER CDC interfaces from
the Designers Projects tree view.
32. Click this step. In the Properties panel:

Select Journalizing Model in the Type drop-down list.
Click the Purge Journal and Unlock Subscribers checkboxes.
33. Link the package steps together using green arrows.
34. Click Save(
) on the toolbar to save the package.

Page 19 of 26
7/9/2010 6:08:00 PM (version 1.0)
35. Click on the Execute(
) button on the toolbar.
38. Open the Operator navigator.
40. Check that the last session ran correctly. There should be 2 inserts and 3 updates.
We can see that the changed data for CONSUMER_1 has been propagated correctly.
41. Open the Models navigator and expand the Oracle CDC Source data model.
42. Right-click on SRC_CITY, select Changed Data Capture > Journal Data

Page 20 of 26
7/9/2010 6:08:00 PM (version 1.0)
Since we used CONSUMER_1 in our package, the changes for CONSUMER_2 have not been
propagated. Having multiple subscribers enables us to execute multiple processes loading the
same changed data without any interference.

Page 21 of 26
7/9/2010 6:08:00 PM (version 1.0)
Part 4: Run the CDC Set Up and Consumption Processes in Production

In this exercise you will run in a production environment the two packages you have created to
set up CDC, and then consume the changes captured by the CDC process.
ODI is already configured to connect to the production environment. A context called Production
has been created and it uses our Oracle production schema (CUST_DW_PROD) instead of the
development schema (CUST_DW_DEV).
1. In the Designer navigator, open the Projects accordion.
3. Expand the Packages node.
4. Right-click on Tutorial Part 2 - CDC Setup and select Generate Scenario. Press OK in the
New Scenario dialog.
Note: A scenario is a frozen version of an ODI component, it is used to put that component
into production.
In this lab we have simplified the deployment process and we can simply run our scenarios in
production using the Production context.
Typically we would have exported a scenario from the development repository and then
imported it into the production repository.
5. Expand the package and the Scenarios node then right-click on
TUTORIAL_PART_2__CDC_SETUP and select Execute.

Page 22 of 26
7/9/2010 6:08:00 PM (version 1.0)
6. In the Execution window, set the Context to Production and press OK.

8. Go to Operator and make sure the session ran successfully.
9. In Designer, go to the Projects tree view and expand the Procedures node in the Advanced
Topics in ODI / First Folder folder.
10. Right-click on the CDC Changes procedure and select Execute.

Page 23 of 26
7/9/2010 6:08:00 PM (version 1.0)
13. Click on the Operator
button in the toolbar.
15. Validate that the procedure ran correctly, 2 inserts and 3 updates should have been
performed.
16. Under Packages, right-click on Tutorial Part 3 Load CDC Data and select Generate
Scenario then click OK.
17. Expand the package and the Scenarios node then right-click on
TUTORIAL_PART_3__LOAD_CDC_DATA and select Execute.

Page 24 of 26
7/9/2010 6:08:00 PM (version 1.0)

20. Open the Operator navigator and select the Session List tab, and expand the All
Executions node.
21. Validate that the last session ran correctly.

Page 25 of 26
7/9/2010 6:08:00 PM (version 1.0)
Summary
At the end of these exercises the participants understand how to:
- Enable Change Data Capture.
- Design Integration Flows to consume the changes captured by the
CDC process.
- Run their processes in a Development environment as well as in a
Production environment.
Questions & Clarifications:
If you have any comments or need additional information, please communicate through the
Oracle Data Integrator forum at: http://forums.oracle.com/forums/forum.jspa?forumID=374

Page 26 of 26
7/9/2010 6:08:00 PM (version 1.0)

ODI Advanced

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ODI Advanced

Uploaded by

Copyright:

Available Formats

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in Oracle Data

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Supported Version: Oracle Data Integrator 11.1.1.3

S308461 - Using ODI-EE and Oracle BI for Real-Time Insight:

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Part 1: Configuring a Model for CDC

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

8. Double-click on the Oracle CDC Source model.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

9. Go to the Journalizing tab.

12. Click on the Reorganize button

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

) on the toolbar to save the Model.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Part 2: Creating a Package to Set Up CDC

3. Select the Packages node, right-click and select New Package.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

7. Click on this step. In the Properties panel:

Select Journalizing Model in the Type drop-down list

Click the Start and Add Subscribers checkboxes.

) on the toolbar to save the package.

9. Click on the Execute(

) button on the toolbar.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

11. Press OK when the Session Started window appears.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

Part 3: Creating the Integration Flows to Consume the Changes

6. Press OK when the Session Started window appears.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

8. Expand the Interfaces node and double-click on Load SRC_CITY CDC.

9. Go to the Mapping tab.

11. Click on the filter

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

12. Right-click on the filter

and select Data to review the changed data.

13. Save and close the interface.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

17. Click on the filter

18. Right-click on the filter

and select Data to review the changed data.

14. Click on the Models tab in the tree view.

Advanced Topics in ODI-EE: Hands on Lab

Advanced Topics in ODI-EE: Hands on Lab

19. Expand the Flat Files Sources model.

Advanced Topics in ODI-EE: Hands on Lab