Name of Solution:

Simulate Real Time Processing of Flat File Sources

Business Requirement:
Near Real-time processing of flat files from the directory by continuously creating file lists for newly added files.

Solution URL:
https://community.informatica.com/solutions/2101

Supported Versions:
PowerCenter 9.1 and 9.5

Description:
In many project flat files written by an external into a directory are treated as source files. These files have to be processed continuously as and when they are created by a continuously running workflow. You can download a workflow in this mapping that demonstrates one of the ways how this can achieved using PowerCenter. The automated process built using PowerCenter performs the following tasks: 1. Reads all the filenames in the directory along with the timestamp. 2. Check each filename against the list of filenames that are already processed. Timestamp is also part of the lookup condition. 3. If the filename is already processed, then it should ignore. Otherwise it is written into a file list. 4. The file list will be used by the session performing the business transformation as the source. 5. Once the processing is done, the workflow is restated again.

Download file contents:

1. Workflow 2. Source File 3. Script

Steps to implement the solution:

1. Execute the script to create tables and copy source files with folder in

$PMSourceFileDir folder. 2. Import workflow using Repository Manager. Select the appropriate folder from repository and resolve the conflicts by choosing suitable option.

Please rate this solution and share your feedback on Marketplace Website.

3. In PowerCenter Designer, observe the expression transformation as below.

TO_DATE functions format has to be update according to systems time format otherwise output will not be generated. 4. Observe the expression transformation of mapping m_create_file_list_test.

5. Open the workflow in Workflow Manager. Assign the integration service in Workflow > Edit -> Integration Service 6. Edit session and assign valid connection object for the source and target. 7. For more details for importing object please visit our YouTube link. 8. In workflow manager, observe the command editor of command task.

Please rate this solution and share your feedback on Marketplace Website.

9. The session s_m_create_file_list filters the files according to time span. If it finds any file then it executes the last session.

10. The workflow has to be scheduled to "Run Continuously".Execute the workflow as schedule workflow and observe the target tables. Create the one more file EMP7.txt observes the workflow log and session log.

11. The key logic of the workflow process is s_m_create_file_list. This session processes the directory list and creates a file list. It also inserts the processed files in a table so that they are not included again in the file list during subsequent run of the workflow.

YouTube Video on Importing and Configuring Workflows:

http://www.youtube.com/playlist?list=PLLRreK2jjjgWBQ4NPfp0QWTxYDvlnEqSJ

Other Useful links:

Mapping Bundles Workflow Bundles Informatica Tools Bundles Informatica for Social Media Bundles Debugging Tools Bundles Visio Templates Bundles B2B Templates and Projects Bundles

Please rate this solution and share your feedback on Marketplace Website.

Data Quality Packs and Plans Bundles Script Bundles

Please rate this solution and share your feedback on Marketplace Website.