Professional Documents
Culture Documents
Configuration Standardization
Most of the connection strings used in the SSIS components as well as other change-prone properties of any task in SSIS must be configured to enable an SSIS job that is transferable and maintainable. Challenges in Configuration Management
During SDLC of any SSIS, the tool must be moved to work in different environments (DEV, TEST and PROD predominantly). This results in changing SSIS/Config files during every move, which is always a time-consuming process.
Post-production, any expiring user ID passwords, server migrations, database migrations and resource movements necessitate a configuration change of a related SSIS package. Most servers/job rights would be held under a DBA resulting in frequent requests to update an SSIS configuration which leads to an actual delay of the change and also appears as overhead for IT and the support function.
failed. If the IT/support has access to the server, the job history can be viewed for the actual error message. If not, then there is a dependency on the DBA for the error message to proceed with the actual resolution of the error. The package should be designed in such a way that it has tasks embedded to notify the target audience of the failure of the job with the error message. This will improve failure resolution efficiency. If required per the business, notifications should also be configured in the package to send success of the job process to the desired audience. This can also be done for monitoring highly critical jobs. The package should have logging enabled and appropriate log type chosen such as log file or event viewer.
If required per the business, notifications should also be configured in the package to send success of the job process to the desired audience. This can also be done for monitoring highly critical jobs.
To overcome the above challenges, rather than hard coding a package configuration or having the configuration in a file, we suggest holding all the configurations in database tables and just use one single configuration to connect to that database in a configuration file that will be a common configuration file across all SSIS packages for an application or customers using the software.
If no error handling is embedded in the job, the failure may exist unnoticed, resulting in critical business impact for the customer. The SQL server agent can be configured to send an alert on failure, but it is just an alert with the name of the job and the particular step that
Configuring SSIS
APPCong DB
6 Tables 1 SP
Figure 1
The ConfiguredValue column of the below table can be stored encrypted if the customer security policies require encryption of the sensitive data in the configuration. See section Securing Configurations for a detailed description of this approach. Table Name Column Name
ConfigurationFilter ConfiguredValue PackagePath ConfiguredValueType
Remarks
Holds the value that links the table and the package. Value of the parameter like DB name, password, etc. Configuration path details. Data type.
This table will be used by the SSIS at runtime to configure its various properties. With any changes due to server migrations, database migrations and/or password changes, an update of that particular row of this table ends the change and activates the package for the new configuration. Error Handling and Monitoring For every successful job, there will be a set of people within different roles who should receive these e-mails; there also may or may not be a different set of people for the failure e-mail. This also varies across job areas and with applications. Role group, application group, people group, status group, success content and failure content, in order, should be configured in interlinked database tables that can be dynamically picked by the job using a custom stored procedure per job needs. To incorporate the aforementioned requirements, five additional database tables and a stored procedure need to be created (as shown below). The new database AppConfig will have the below components: 1. APPLICATION_HEADER Table 2. EMAIL_CONTENT_DETAIL Table 3. EMAIL_CONTENT_HEADER Table 4. EMAIL_SERVICE_DETAIL Table 5. EMAIL_SERVICE_HEADER Table 6. SSIS_CONFIGURATIONS Table (See above Configuration Standardization section for details) 7. SP_EMAIL_CONFIGURATIONS Stored Procedure
Any changes due to resource movements, additions and role changes can be handled by updating the above table records. 1. APPLICATION_HEADER: This table holds the key and the name of each application. Column Name
App_Key App_Description Mod_Date Mod_User
Data Type
numeric varchar datetime varchar
Remarks
Holds the key for each application. Application name. Date field. Modified user details.
2. EMAIL_CONTENT_DETAIL: This table holds the e-mail content for success and failure run of the package. Column Name
EMAIL_KEY EMAIL_CODE DESCRIPTION MOD_DATE MOD_USER
Data Type
numeric varchar varchar datetime varchar E-mail key.
Remarks
E-mail code to differentiate the email sending option. Description of e-mail receiver. Modified user details. Modified date.
3. EMAIL_CONTENT_HEADER: This table holds the header details of the email such as subject and footer notes. Column Name
EMAIL_KEY JOB_CODE EMAIL_SUCCESS_SUBJECT EMAIL_FAIL_SUBJECT EMAIL_FOOTER MOD_USER MOD_DATE
Data Type
numeric varchar varchar varchar varchar varchar datetime E-mail key.
Remarks
Job name which must be the same as package name. Holds e-mail subject for success run. Holds e-mail subject for failure run. Holds e-mail footer. Modified user details. Modified date.
4. EMAIL_SERVICE_DETAIL: This table holds the e-mail IDs of the personnel for whom the e-mail notifications are to be sent. Column Name
LOGIN EMAILID FULLNAME APP_KEY EMAIL_KEY MOD_DATE MOD_USER
Data Type
varchar varchar varchar numeric numeric datetime varchar
Remarks
Holds UPIN ID. E-mail address of the person. Name of the associate. Application key. E-mail key. Modified date. Modified user details.
5. EMAIL_SERVICE_HEADER: This table holds the different roles of the users who will be receiving e-mail notifications on job success/failure.. Column Name
EMAIL_KEY EMAIL_CODE DESCRIPTION MOD_DATE MOD_USER
Data Type
numeric varchar varchar datetime varchar E-mail key.
Remarks
E-mail code to differentiate the email sending option. Description of e-mail receiver. Modified user details. Modified date.
7. SP_EMAIL_CONFIGURATIONS: Retrieves the e-mail ID based on the input parameters passed from the SSIS package. The input parameter to the SP will be set based upon the e-mail notification requirement of the job such as success/failure, customer/IT, etc.
Create a view SSISConfigurations that has all the fields from the table SSIS_Configurations with the ConfiguredValue column decrypted. Create three triggers on the view for insert, delete and update. The triggers should facilitate the corresponding action in the configuration table SSIS_Configurations with the plain value in the view encrypted by the triggers and stored in the table.
Change column ConfiguredValue in the table SSIS_Configurations to VarBinary: datatype and add an identity ID column to the table. cognizant 20-20 insights
The encryption and decryption is by a password. Also, the view should have restricted access for reading its DDL. All SSIS packages should use the created view for SQL server configurations. To implement the security concept alone for the standardized package, just renaming the configuration table with encryption column and
creating a view (with decryption) by the name of the old configuration table and creation of triggers would suffice, requiring no changes at the SSIS packages front. Hence, for SSIS the view will retrieve decrypted plain text configurations and at the database end it will appear encrypted (see screen capture, below).
Attached below is a text document containing sample create scripts for the table, view and triggers.
Increased ease of maintenance. Any changes to the intended users (business or IT) can be made easily application-wise or in any grouping applied. Removing any inactive person from the notification list is a one-place activity rather than searching all the job configuration files (see screen capture, below).
Benefits
Key advantages of this approach include:
Simplified configurations lookup for a specific job (see screen capture, below).
Simplified lookup for a specific configuration across jobs. This helps find the usage of any server/DB/ file location, which is very useful during server migrations and DB changes. Even lookup by username is possible, which will make username and password changes much easier (see screen captures, below).
Well-monitored jobs increase database availability. Decreased down times. Decreased change implementation time. Decreased dependency on DBA/other teams for support.
Increased resolution efficiency due to data (error cases) availability for the desired people. Secured configurations. Security can be incorporated without change to the existing packages.
References
About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the worlds leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50 delivery centers worldwide and approximately 145,200 employees as of June 30, 2012, Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on Twitter: Cognizant.
World Headquarters
500 Frank W. Burr Blvd. Teaneck, NJ 07666 USA Phone: +1 201 801 0233 Fax: +1 201 801 0243 Toll Free: +1 888 937 3277 Email: inquiry@cognizant.com
European Headquarters
1 Kingdom Street Paddington Central London W2 6BD Phone: +44 (0) 20 7297 7600 Fax: +44 (0) 20 7121 0102 Email: infouk@cognizant.com
Copyright 2012, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.