Understanding Document

For
UNIX Procedure Automation

117
 DOCUMENT CONTROL

Document Name STC Unix Procedure Automation Understanding

Document

Abstract This document is useful to gain brief understanding for

the scripts involved in procedure automation..

Document Reference

VERSION HISTORY

Versio Date Author(s) Reviewed Approved Reason for

n By by Changes
11th Feburary
0.1 2016 Somesh Sharma Tarun Makhija Initial draft version.
Functional Excel Sheet
Tarun Attached as per
23rd feburary Makhija/Shreya changes suggested for
0.2 2016 Somesh Sharma Karan draft version
 CONTENTS
1 INTRODUCTION TO UNIX_AUTOMATION..........................................................................................................
2 SCOPE.........................................................................................................................................................................
3 TARGET AUDIENCE.................................................................................................................................................
4 FUNCTIONAL DESCRIPTION................................................................................................................................
5 TECHNICAL DESCRIPTION...................................................................................................................................
6 OPERATIONAL DESCRIPTION..............................................................................................................................
7 PROCESS AND TOOLS USED................................................................................................................................
8 PROCESS FLOW......................................................................................................................................................
9 PROCESS FLOW DESCRIPTION..........................................................................................................................
 1 Introduction to UNIX Automation of Procedures (Goals)

The Aim for this activity was to automate Operational KPI’S procedures and to send the
required report in email, which was done before manually by team members.

This has reduced lots of Team effort and thus providing an efficient method for the above
activities.

2 Scope
This document is prepared based on Cognizant’s understanding of the project based on the
knowledge gained in past. This document will be a live document and will be continuously
updated.

 Functional – This Includes functionality of how we are scheduling our Operational

KPI’S and also sending an email for the same automatically if the threshold is
breached.

 Technical –This Includes UNIX Scripts which has been developed for achieving the
required goal.

 Operational –This includes location, logs, monitoring etc. for all functional and
technical aspects.

 Processes and Tools Used – This includes standard processes followed to implement
and deploy the code and various tools used.

3 Target Audience
Target audience for this document is
 Data quality team
 EDW DQ Team
 ODS DQ Team
 CLDM DQ validation team

4 Functional Description
 Backend Description and Motive
 The KPIs implemented are meant to analyze any anomaly in trend of the daily “Stage
Tables Loading” (ICMS/CRM), Revenue reconciliation (IN_SDP Revenue), Stage-target
Consistency and the Target-target Consistency.

 If any deviation is found and the predefined threshold is breached then, an email will be
triggered for those KPIs.
 Unix Automation

 It includes various Unix Scripts for automation of above procedures and email, if any
deviation is found.

5 Technical Description
 Windows Base BAT File:-

 “KPI_OPS” is a Wrapper BAT file which calls all the Unix Scripts and is scheduled by
Windows Scheduler.

 Unix Shell Scripting:-

 Shell Scripts are developed to carry out the desired functionality, which includes:-
Dynamic generation of a BTEQ, from the predefined queries present in the
database.
To analyze the result of queries executed, and to generate the report with the
KPIs breaching the threshold.
To generate a HTML file, to design the content and view of email which has to
be send.
To configure and send an email to DQ team which consist the report.

 Various Unix Scripts Included are:-

 “Wrapper_Deviation_Calculation.bash”
 “Check_Threshold.bash”
 “Export_Data_Upd.bash”
 “File_To_Reports.bash”
 “Create_Bat_For_PowerShell.bash”

6 Operational Description:-
 All the Logs which are generated from above Scripts can be found in below directory path:-
E:\cygwin\log
Or
In Cygwin Environment:-
Cygdrive/E/Cygwin/Log

7 Processes and Tools Used:-

 Tools Used:-
Cygwin, Teradata.
 Technical Process:-
UNIX Shell Scripting
 8 Process Flow:-
KPI OPS (BAT
FILE) Wrapper Bat File

Argument Passed While Calling:-

Fetch_Deviation_Queries.txt

Wrapper_Deviation
Argument Passed While Calling:-
_Calculation.bash
Fetch_Deviation_Queries.txt

Export_Data_Upd.bash Check_Threshold.bash CallsThreshold_Queries.txt

Calls Logon.txt
Creates B.TEQ and Run it

CHECK_THRESHOLD.BASH PICKS QUERY FROM THRESHOLD_QUERIES.TXT AND CHECKS WHETHER

THERE IS ANY DEVIATION IN THE RESULTS.

IF DEVIATION FOUND “EXPORT_DATA_UPD.BASH “IS CALLED TO FETCH EMAIL RELATED

PARAMETERS (EMAIL ID ETC.).

Check_Threshold.bash

No
Deviation
Deviation STOP
STOP
Found?
Found?

Yes

Export_Data_Upd.bash Fetches Email Related Parameters

File_To_Reports.bash Create_Bat_For_Powers
To Generate HTML File for Email hell.bash
Body

THEN AGAIN KPI_OPS CALLS FINALLY CREATED BAT FILE OF POWERSHELL TO SEND AN EMAIL TO
CONCERNED TEAM MEMBERS.
 KPI OPS (BAT Dynamically Created
FILE) Bat File for Power
shell
Calls

To Send Email

9 Processes Flow Description:-

 KPI OPS (BAT FILE):-
Wrapper Bat File which is called after DQF_MASTER procedure is called. It is necessary to call this
BAT after Revenue_KPI procedure is finished.

 Wrapper_Deviation_Calculation.Bash:-
This is main Unix Script Which first Calls “Export_Data_Upd.bash” and also Passes Path Of
Fetch_Deviation_Queries.txt, which includes Main Procedures to analyze Deviation in Trend.

 Export_Data_Upd.Bash:-
It Fetches the queries from Fetch_Deviation_Queries.txt and creates a BTEQ and executes it.

 Check_Threshold.Bash:-
Then After Above Process:-
This script is called by Wrapper Bat file and path of Threshold_Query.txt is passed which includes
query to check whether the threshold is breached or not?

If Threshold is breached then Check_Threshold.Bash calls Export_Data_Upd.Bash to fetch

Email_Id’s and Email Text.

Then File_To_Reports.bash is called to dynamically generate a HTML/JAVASCRIPT file to

represent overall email and it calls Create_Bat_For_Powershell.bash and passes the above
HTML file as an argument.

 Create_Bat_For_Powershell.bash:-
It creates a bat file for Powershell for the purpose of sending an email.

Again KPI OPS Wrapper Script Calls The above bat file to send an email.

 Please find below the description of all the scripts.

Scripts_Function.xlsx