Data Warehousing

L.RAMANATHAN Asst. Prof. SCSE VIT University Room No. MB 206 E

PLANNING AND PROJECT MANAGEMENT

Planning Your Data warehouse

Key Issues

Value and Expectations- To identify whether a data warehouse is needed for the company. If needed, what are the expectations you wish to achieve. Risk Assessment- Planners have to assess the risk factors. If the project fails how much will be the loss, whether direct or indirect etc.

Planning Your Data warehouse

Key Issues Contd..

Top-Down or Bottom-Up

Once we have decided to build a data warehouse, we should decide on the approach.

Top-down From enterprise level move down to data marts Bottom-up - From several data marts move up to a single large data warehouse. Each has its own merits and demerits. Depending upon our requirements and resources available ,we should decide on either one of these approaches.

Planning Your Data warehouse

Key Issues Contd.. Build or Buy

Develop the DW as an in house project or buy the tools that are available in the market Large companies will have a blend of these two Need to find the balance between in-house and vendor software.

Planning Your Data warehouse

Key Issues Contd..

Single Vendor or Best of breed- Either use the products of a single vendor or use products from more than one vendor.

Advantages of Single vendor

High Level integration Seamless cooperation among components Centrally Managed information exchange Overall price Customized environment Select products best suited for the function No need to compromise between database and support tools

Advantages of Multiple Vendors

Planning Your Data warehouse

Key Issues Contd Business requirements Not

Need to focus on what information is needed and not on how to provide information Conduct a preliminary survey of requirements Prioritize and determine the plan for individual data marts

Top Management Support

The Project should have the support of the management right from day one Whenever the project encounters major setbacks, the official is expected to resolve the issues.

top top

Planning Your Data warehouse

Key Issues Contd

Justifying the data warehouse- Justifying the total cost required for developing a DW by balancing the risks against the benefits is not that easy. If the top management of the company is able to understand the benefits of having a DW then there is no need for justification. But if the top management doesnt understand its importance then a formal justification is needed. One such approach is to find the ratio between current costs and proposed costs and highlighting that to the management.

Planning Your Data warehouse

Key Issues Contd

The Overall Plan- An overall plan sets the direction, tone and goals of the project.

Outline for Overall Plan Introduction Mission Statement Scope Goals and Objectives Key Issues and Options Values and Expectations Justification Executive Sponsorship Implementation Strategy Tentative Schedule Project Authorization

THE DATA WARE HOUSE PROJECTDW DIFFERENT FROM OLTP DATA DATA PROJECT INFORMATION
ACQUISITION STORAGE DELIVERY
Large number of sources Storage of large data Several user types Different Computing Platforms Outside sources Huge initial load Data Replication Complex data integration and transformation Parallel processing Data storage in staging area Several index files Multiple query types Multidimensional analysis OLAP functionality

Storage of newer data Meta data management types Archival of old data and new tool compatibility Interface to DSS and feed to data mining

THE DATA WARE HOUSE PROJECTTHE LIFE-CYCLE APPROACH

A data warehouse life cycle is entirely different from that of a SDLC. A DW life cycle should include repetitive tasks going through cycles of refinement. This is because of the complexity and broad scope of the project. The functional components of a DW are data acquisition, data storage and information delivery. The phases of the development should be based in these functional components.

THE DATA WARE HOUSE PROJECT

THE DEVELOPMENT PHASE

DATA ACQUISITION D E S I G N ARCHITECTURE INFRASTRUCTURE R E Q. D E F N C O N S T R U C T I O N D E P L O Y M E N T M A I N T e I N A N C E

P R O J. P L A N

DATA STORAGE

INFORMATION DELIVERY

THE DATA WARE HOUSE PROJECTTHE PROJECT TEAM

Success of a project depends on the project team. The project team minimizes the complexity by assigning the right task to the right person. Each person in a team is assigned with a specific role and responsibility. Organizing the Project Team

Putting right person in the right area List all the project challenges and specialized skills that are needed Prepare a list of team roles needed to develop the project Assign the roles to individual or group as per the requirement. Skills, experience and Knowledge are important for team members.

THE DATA WARE HOUSE PROJECT

THE PROJECT TEAM - Roles and Responsibilities
Team roles are designed to perform one or more tasks A basic set of team roles are: related

Roles
Executive Sponsor Project Manager User Liaison Manager Infrastructure Specialist Business Analyst Lead Architect

Responsibilities
Direction,support,Arbitration Assignments,Monitoring,Control Coordination with user groups Infra. Design or Construction Requirements Definition Architecture Design

THE DATA WARE HOUSE PROJECT

THE PROJECT TEAM- ROLES Contd
Roles
Data Modeler DW administrator Data Transformation Specialist Quality Assurance Analyst Testing Coordinator End User appln. specialist Development Programmer Lead Trainer

Responsibilities
Relational and Dimensional modeling DBA functions Data Extraction, Integration and Transformation Quality Control Program, System & tools testing Confirm data meaning/relation In house projects Coordination of user and team training

THE DATA WARE HOUSE PROJECT

THE PROJECT TEAM- SKILLS & EXPERIENCE
Roles Executive Sponsor Project Manager User Liaison Manager Lead Architect Infrastructure Specialist Business Analyst Data Modeler Skill set and experience levels In-depth knowledge of the business Project Management experience Knowledge of system from user point of view Analytical skills, knowledge of DW concepts Specialist in H/W, OS etc. Analytical skills, interaction with users Expertise in Relational and Dimensional modeling

THE DATA WARE HOUSE PROJECTTHE PROJECT TEAM- SKILLS & EXPERIENCE
Roles
DW administrator Data Transformation Specialist Quality Assurance Analyst Testing Coordinator End User appln specialist Development Programmer Lead Trainer

Skill set and experience levels

Expert in physical database design and implementation Knowledge of data structures Knowledge of Quality Control techniques Familiarity with testing methods and tools In-depth knowledge of source application Programming and analysis skills Experience in IT training

THE DATA WARE HOUSE PROJECT

User Participation

THE PROJECT TEAM- USER PARTICIPATION

Project Planning

Provide goals, objectives, expectations etc. Provides top management support. Identify all source systems Define metrics for measuring business success Define information needed from DW. Review dimensional data model Data extraction and transformation Review architectural design

Requirements Definition

Design

THE DATA WARE HOUSE PROJECT

User Participation Contd.

THE PROJECT TEAM- USER PARTICIPATION

Construction

Participate in user acceptance testing Validate data extraction and transformation functions Test usage of meta data Verify audit trails Confirm initial data load Arrange and participate in user training Provide input for enhancements Test and accept enhancements

Deployment

Maintenance