Module 3: Introduction to Business Intelligence

Development
Presentation:60 minutes
Table of contents
Module 3: Introduction to Business Intelligence Development.............................................................. 1
Determine Business Intelligence development requirements and implement a Business
Intelligence development project. ............................................................................................... 3
Lesson 1: Overview of Business Intelligence Development............................................................. 5
Lesson 1: Overview of Business Intelligence Development............................................................. 5
Identify requirements and resources for implementing and managing a Business Intelligence
development project.................................................................................................................... 5
Considerations for Business Intelligence Development.................................................................. 7
Principle: Describe considerations for Business Intelligence development projects. ................. 7
Guidelines for Effective Business Intelligence Development....................................................... 11
Principle: Use techniques that ensure that Business Intelligence development is effective and
efficient. .................................................................................................................................... 11
Considerations for Assembling a Business Intelligence Development Team............................... 15
Principle: Describe the factors to consider when assembling a Business Intelligence
development project team. ........................................................................................................ 15
Demonstration: Using Source Control in a Business Intelligence Project .................................... 19
Procedure: Use Visual SourceSafe to show how source control does and does not work in a
Business Intelligence environment. .......................................................................................... 19
Lesson 2: Managing Business Intelligence Development............................................................... 22
Manage a Business Intelligence development project. ............................................................. 22
Considerations for Planning the Development Process ................................................................ 23
Principle: Describe considerations for planning a Business Intelligence development project.23
Guidelines for Testing a Business Intelligence Solution............................................................... 27
Principle: Identify the processes and infrastructure required to properly test a Business
Intelligence solution.................................................................................................................. 27
Guidelines for Deploying a Business Intelligence Solution.......................................................... 31
Principle: Describe guidelines for deploying a Business Intelligence solution......................... 31
Lesson 3: Determining Data Management Processes ...................................................................... 33
Determine effective data management processes...................................................................... 33
Guidelines for Managing Data Issues ........................................................................................... 35
 Principle: Describe how to manage data issues effectively. ..................................................... 35
Considerations for ETL................................................................................................................. 39
Principle: Describe considerations for ETL. ............................................................................. 39
Considerations for Data Validation............................................................................................... 42
Principle: Describe considerations for validating data.............................................................. 42
Guidelines for Managing Lineage................................................................................................. 45
Principle: Determine lineage requirements and considerations for implementing lineage
support....................................................................................................................................... 45
Guidelines for Reporting............................................................................................................... 47
Principle: Determine reporting requirements, volumes, and supporting infrastructure required.
.................................................................................................................................................. 47
Lab: Business Intelligence Development.......................................................................................... 50
Exercise 1: Identifying Team Requirements................................................................................ 52
Exercise 2: Determining Naming Standards ................................................................................. 54
Module objective
After completing this module you will be able to:

Determine Business Intelligence development requirements and implement a Business Intelligence

development project.
Instructor note
Ensure that students understand that this module is an overview of requirements and that many other
courses deal with the specific details of methods, technologies, and techniques. Also, emphasize that a
number of writers, including Kimball and Inmon, have written multiple books and articles about how
Business Intelligence projects should be implemented.

This module contains references to external sources of further information. You can find links to all
referenced resources on the student CD. Because these external sites are beyond Microsoft control,
you should test all links prior to teaching this module.
Introduction
A wide range of skills and knowledge is required for the successful development of any Business
Intelligence solution. The purpose of the discussion in this module is to provide you with the essential
knowledge to manage such a project and to provide an introduction to key concepts that you should
know and use.
This module presents an overview of the issues that you must consider and deal with in order to have a
successful project. You should use the information in this module as a means for establishing with
your organization and your development team a solid understanding of the magnitude of the effort
required for successful Business Intelligence solution development, and some of the ways in which
you can optimize the development effort in a BI project.
Lesson 1: Overview of Business Intelligence Development

Lesson objective:
After completing this lesson, you will be able to:

Identify requirements and resources for implementing and managing a Business Intelligence
development project.
Instructor note
Ensure that students understand the nature of this topic. Its purpose is to give them a good introduction
to factors that those running a Business Intelligence solution development project should consider.
There are many other resources that they should become familiar with and use, ranging from various
product-specific information to more general materials about tools, techniques, and methodologies.

Identify which students have already run such projects and ask what lessons they have learned about
each of the topics covered in this module.

Introduction
Managing a Business Intelligence development project is much like running any other major project.
However, Business Intelligence projects have some special issues that you should consider in terms of
both development processes themselves and the nature of the solution requirements. Business
Intelligence projects are inherently wide ranging and complex. This means that managers of Business
Intelligence solution development projects must have a broad set of skills; their development team
members also must possess a wide set of skills and knowledge.
This lesson is designed to provide a solid foundation in the critical elements that are required to
develop successful Business Intelligence solutions.
Considerations for Business Intelligence Development

Principle: Describe considerations for Business Intelligence development projects.

Instructor note
Ask students to talk about their own experiences with the various factors that influence technology
choices, and encourage them to assess the importance of each type of factor in reaching a satisfactory
outcome for the project.

Ask students how they would deal with each type of factor if there were disagreement in the
organization about which way the technology decisions should go.

Relate the technology choice decision to Microsoft software and technology factors such as ease of
use, robustness, capacity, scalability, availability, integration, and comprehensiveness of solution
technologies for end-to-end Business Intelligence solutions.

Introduction
Among the many things that you should consider when planning and implementing a Business
Intelligence solution are three key topics: the nature of the systems to be used, the nature of the
application requirements, and the selection of specific technologies.
Every Business Intelligence project involves these factors. System requirements range broadly from
network issues and clustering to Web farms and load balancing. For application development, the
potential list of considerations is extremely large, encompassing virtually any technology that is
available for use in modern application development. The determination of application requirements is
crucial, then, so time, money, and effort are not wasted.
Today, the range of technological possibilities is wide. For Business Intelligence application
development, various server technologies, client technologies, Web technologies, and development
environments—in addition to any management tools or technologies—are used to guide, manage, or
control the project development activities.
System assessment
In the current computing environment, the concept of a system is very broad. In a Business
Intelligence project environment, it is possible that virtually every aspect of your computer systems,
infrastructure, and network will be involved. You should document the infrastructure (or use existing
documentation) to access its current state. This documentation usually consists of a combination of
network diagrams and text descriptions. While not exhaustive, the following is a list of things that you
should check to determine the role that they will play in your particular solution scenario, as well as
the degree to which they are critical to the solution’s success and whether there are any issues to be
resolved:
• Network types, speed, and bandwidth
• Network protocols in use
• Firewalls and TCP/IP ports settings
• Server infrastructure
• Client/workstation infrastructure
• Server and client maintenance routines
• Server application configurations
• Domain structure and Microsoft® Active Directory® directory service features in use
• Security model for your corporate environment and systems
• Legacy systems
• Physical location of data sources and speed of connections
• Internet and intranet facilities, architecture, and bandwidth
• Display capabilities
• Software in use—versions, capabilities, integration, and collaboration capabilities
• Service pack and support update infrastructure
Each of these parts of your system can contribute to the Business Intelligence solution as well as to
problems that might arise in the development and deployment of the project. It is important that you as
a project manager consider and assess how each of these affects your particular project. Use this list to
take stock of your system and system parameters. Be sure to compare your system components with
Microsoft’s product specifications, white papers, and best practices where those are available.

Tip
One feature of modern systems that is often overlooked is the software life cycle of commercial
products. Microsoft has a published list of when particular products will expire in terms of support and
updates. It is strongly recommended that you determine what the limits are on all key products. Use
the search words [Microsoft will no longer support] on the Microsoft Web site.
Application requirements
Application development for Business Intelligence is a very broad topic. There are several categories
of applications and a variety of meanings of the term application. There are client-side applications
and server-side applications. As with the server infrastructure, you should identify the applications
that are present at the start of the BI project and document their current state. Categories of
applications can be broken into several broad groups, and these can also overlap one another:
• Data entry and editing applications
• Data analysis applications
• Data browsing and reporting applications
• Web services applications
• Microsoft Windows® forms applications
• Web forms applications
A number of Business Intelligence application types fall within these categories, specifically, data
entry, analysis, and reporting. Additionally, there are specialized Business Intelligence applications for
data mining and drilldown/drill-up interactive querying and analysis of data warehouse cubes. For
specific development of these applications in your Business Intelligence project, requirements will
vary from case to case depending on the following factors:
• Purpose of the application—data entry, browsing, reporting, analysis, data mining
• Type of application—for example Web-based, Windows-based, or mobile
• User characteristics and requirements
• Number of users
• Workflow patterns
• Maintenance requirements
• Interface requirements
• Location
• Platform—for example, workstation, mobile computer, Microsoft Tablet PC, Personal Digital
Assistant (PDA), or smart phone
• Performance
• Environment—server availability and scalability
• Deployment—integration with existing systems and additional requirements
• Security
• Access rights and rights management
Each of these topics can help to determine the precise style and characteristics of your application.
Clearly, a thorough analysis and design process are required to properly assess these issues. The
design process that builds on this information is a time-consuming one for which you must allow in
your project plan.
Choice of technologies
The choices about application design and features will also be affected by the available technologies.
Many technologies are available today, and the market for Business Intelligence technologies has been
growing significantly over the past decade. In addition to server technologies such as Microsoft SQL
Server™ Analysis Server, there are numerous client technologies ranging from dedicated Business
Intelligence client software to add-ins that extend the functionality of more generic products such as
Microsoft Office Excel®. It is critical that both the server-side and client-side technologies be
evaluated and selected near the beginning of the project.
Each project will need to assess which Microsoft and third-party products it wants or needs to employ
in order to deliver the solution that addresses the requirements for data handling, usability, scalability,
and accessibility along with any other requirements that the design process has identified.
Technologies fall into the following broad categories:
• Server technologies, such as SQL Server, SQL Server Analysis Server, SQL Server Reporting
Services, Microsoft SharePoint® Server, and other third-party server applications
• Generic client-side technologies such as Microsoft Office and Web browsers
• Specialist client-side technologies such as third-party Business Intelligence analysis clients
• Web services
For your Business Intelligence project, it is important to determine the mix of technologies that will
best address the business and stakeholder requirements. To do this, the project team may need to do
evaluation and testing of sample software, seek vendor information, use Internet search engines to
assess product experiences of others, and determine what similar projects have done by examining
case studies.
Like other processes already described, the actual determination of which technologies to use can be a
time-consuming and resource-consuming activity. It may also require budget commitments to pay for
testing software or infrastructure on which to run such tests.
Investment in careful technology assessment will contribute to the overall success of the project and is
not an activity that should be either ignored or treated lightly.
Guidelines for Effective Business Intelligence Development

Principle: Use techniques that ensure that Business Intelligence development is effective and efficient.
Instructor note
The factors described in this topic are common to all development projects, but make sure to point out
how Business Intelligence projects are slightly different in terms of both their tools and complexity.

Ask students to discuss the ways in which the complexity of Business Intelligence projects can be best
managed, particularly with regard to source control.

Encourage students to discuss whether they have had such issues and how they have dealt with them
in their organizations.

Introduction
The topics in this section are not unique to Business Intelligence development projects. However,
there are characteristics of Business Intelligence projects that are somewhat different from standard
application development requirements in terms of team skills required, technological knowledge
needed, and project management issues. The topics discussed provide a context in which to determine
whether your team has the appropriate skills, the right resources, and the project management
capabilities.
Use these guidelines to verify that your Business Intelligence project will be well supported by
qualified developers, competent project managers, appropriate technologies, and good source control
mechanisms.
Identify key skills required
In the course of a Business Intelligence project, your team will need a wide range of skills, particularly
if it is a full enterprise-scale solution. It is important to properly identify the critical skills required by
the project at the outset so that, as previously discussed, staff can be trained, retrained, or hired to
provide the right development skill set to complete the project properly. It is common in BI projects to
provide additional training to existing staff or to hire consultants for portions of the project.
Key skills should be identified for at least the following areas of the project:
• System analysis
• Data handling and Extract Transform and Load (ETL) operations
• Application design—client and server
• Web development
• SQL Server administration
• SQL Server development
• SQL Server Analysis Services development, including OLAP cube development, MDX
queries, and Data Mining
• SQL Server Reporting Services development
• SQL Server Integration Services development.
• Microsoft Visual Studio® development—ADO.NET, ASP.NET, and Microsoft Visual
SourceSafe®
• Microsoft Office products, including Microsoft Office Excel, Microsoft Office Visio®, and
Microsoft Office Project
• Microsoft Windows Server® 2003 and Active Directory management
• Infrastructure design and implementation—hardware and software
• Application testing
• Deployment
• System monitoring—Windows and SQL Server
Other skills might well be identified as the project is defined and planned, but this list covers virtually
all the major skills required to design, develop, test, and deploy a Business Intelligence solution.
Select technologies to be used in the project
On the basis of the information collected, the project team needs to select the appropriate technologies
to use in the delivery of the project. Because technologies change with new products and versions and
corrections to existing versions, there is always a need to review the current options that are available.
It is important to select a set of technologies that will work well for the life of the project, be well
supported by venders, and work well together both from a user standpoint and a technology
integration standpoint. Technologies to be selected will depend on project particulars but fall into the
following categories:
• ETL
• Database
• Analysis
• Development
• Storage
• Disaster recovery
• System management
• Project management
• Presentation
• Delivery
• Collaboration
For each of the categories to be used in your particular Business Intelligence project, you should
review the available choices, select a short list, do comparative review and evaluation, and make a
final selection. It is usually best of have a first and second choice in the event that some unforeseen
event occurs (such as a company takeover) that changes your potential selection options.
Another important consideration is in selection the edition of SQL Server 2005 that you will use in
your solution. There is a significant difference in feature sets for BI between the standard and
enterprise editions and you should identify the features you require so you can base your budget and
infrastructure planning accordingly.
Establish project management processes
A great many projects fail because of the lack of understanding good project management practices
and processes. Ensure that you determine a method of managing your Business Intelligence project
that enables accurate tracking of tasks, resources, critical paths, conflicts, and times. Establish who the
project managers are, who reports to whom, and what communication methods are to be used for task
verification, resource substitution, or other adjustments to the project (such as putting extra resources
on critical tasks to ensure that they are completed on time).
You should ensure that tools are in place to support the project management strategy. Products such as
Microsoft Office Project and Microsoft Project Server are examples, but there are many others as well.
To use such tools effectively requires proper training, dedicated time for the project managers to
maintain information, and clear definitions of roles and responsibilities. If the project team is
geographically distributed, use of a portal solution, such as SharePoint Portal Server or Windows
SharePoint Services, can help facilitate collaboration.
Implementation of a project management process that has proper use of project management tools can
reduce stress on staff, help to anticipate project bottlenecks, enable smooth allocation of resources,
and track project costs easily.
Match technologies to requirements
After the requirements have been assessed through the surveys of stakeholders and other methods, the
project team must determine which technologies to use for the solution.
Use the following list to review which technologies (software and hardware) are required for each of
the components of the solution:
• Networking—local area network (LAN), wide area network (WAN), intranet, and Internet
• Data connectivity and ETL
• Application development—client and server
• Web site—presentation, load balancing, and data handling
• Data storage
• Data manipulation
• Security
• Server management
• Application testing
• Business analysis
• Reporting
• Presentation
Each of these requirements can be met by a range of technologies, each with their own implications
for the project. For example, security in BI solution that is delivered to end users through a web site
might require the use of SSL, which would require a certificate for IIS. You should try to use the
smallest number of the right technologies to achieve the required outcomes for the project. The actual
selection in each case can be difficult because many choices are available in most of these areas. The
task of the project team is to identify the best fit in terms of factors of concern to the organization,
which are likely to include ease of use, richness of features, currency of version, scalability,
performance, security, and cost.
Establish source control mechanisms
Particularly in complex projects involving multiple development teams and developers, it is crucial to
have proper source control processes and tools in place. The project management team should identify
the best tools for this at the outset of the project and use them throughout the life of the project,
including subsequent revisions or modifications of the solution.
Many tools on the market offer source control. Within the Microsoft solution environment, many
organizations use Visual SourceSafe (VSS) as the primary method of tracking source control. You
need to assess whether VSS provides the features that your team requires along with any other
candidates that the project is considering. The advantage of VSS is that it is specifically designed to
integrate well with other Microsoft technologies. While this does not make it an automatic choice, it is
nonetheless an important consideration.
Ensure that source control cannot be bypassed
Source control cannot be achieved simply by software mechanisms. It is also important to put in place
policies and procedures that ensure that all members of the development team adhere to systematic
methods of change control, documentation, and version control.
You should familiarize yourself with source control software, understand how to manage source
control, and establish a culture of source control within your project group.
Considerations for Assembling a Business Intelligence Development Team

Principle: Describe the factors to consider when assembling a Business Intelligence development
project team.
Instructor note
Explain the importance of obtaining the correct skill sets along with the right technologies. Explain
that a poor implementation of a good technology can be disastrous in terms of project success.

Explain proper consideration of these factors and that correct mapping of solution components to team
members and skills can reduce development time, enable parallel development efforts, and minimize
both risks and errors.

Explain and give examples of how proper identification of training requirements for both skills
acquisition and reskilling can benefit the project and help to ensure success. Discuss the ways in
which not placing enough emphasis on the training endangers the solution and increases the likelihood
that stakeholders will accept it.

Introduction
Experienced project managers know that the mixture of personnel, skills, technologies, and training
can make or break a project. It is important to consider the following areas when assembling your
Business Intelligence project team.
Many potentially good projects end up wasting considerable amounts of time and money because of:
• Inappropriate or out-of-date skills.
• Untrained or poorly trained personnel.
• Incorrect or insufficient technologies.
• Poor project management skills and processes.
• Poor or incorrect task allocation.
• Failure to invest in appropriate training ahead of project requirements and deadlines.
Projects can fail or become problematic for many other reasons besides those listed. However, if you
address the major topics discussed here, there is a good chance that you can deliver your project in a
timely manner and on budget.
What skills do team members have?
Any Business Intelligence solution will involve development solutions that range across the entire IT
domain—networking, hardware, software, Web, Internet, intranet, desktop applications, server
applications, and many more. The project manager who keeps this in mind from the outset is far less
likely to be surprised partway through the project that the project is missing particular skills or people.
When considering the issue of team member skills, it is helpful to create a task/skills matrix to identify
clearly what skills are required for each part of the Business Intelligence solution. This analysis needs
to be sufficiently detailed to determine whether developers have the very particular skills needed to
pursue their specific development tasks. For example, listing Microsoft Visual Basic® as a skill is not
sufficiently detailed enough to know whether the skills are for Visual Basic 3.0, Visual Basic 6.0, or
Visual Basic .NET.
To assess team skills, you should, in addition to reviewing team member skill lists and resumés,
survey team members to identify what they themselves perceive are their skill strengths and
weaknesses. By combining information from these sources along with manager assessments (where
possible), you should be able to create a summary workbook in Microsoft Office Excel that lists
project components and the skills required to complete them. A second matrix can similarly
summarize the mix of people and skills. These matrices can then be used for project planning and
training needs analysis.
For a Business Intelligence project using primarily Microsoft technologies, you will almost certainly
need developers with the following areas of skills or knowledge:
• SQL Server 2005—particularly SQL Server Integration Services, SQL Server Analysis
Services, and SQL Server Reporting Services
• Visual Studio and the Microsoft .NET Framework—including ADO.NET and ASP.NET
• Multidimensional Expressions (MDX) query language
• XML and Web services in a .NET Framework context
• Internet protocols and Web development
• Activity Directory and security
• Open Database Connectivity (ODBC) and OLE DB
• Microsoft Office Business Scorecard Manager
• Microsoft Visual Studio Tools for Office
• Microsoft Office Excel 2003 or later, Microsoft PivotTable® and Microsoft PivotChart®, and
add-ins for Business Intelligence
 • Microsoft Windows 2003 infrastructure design, Internet Information Services (IIS)
configuration and design of Web infrastructure, and other required servers (for example,
Microsoft Exchange Server, SharePoint Server, and Microsoft Project Server)
• Networking specialist support – subnet design, security, protocols, and TCP/IP port
assignments
Many other technological skills and additional knowledge might be relevant depending on the solution
architecture under consideration. You will need to do a detailed assessment in the specific context of
your proposed solution and develop a precise list of requirements and personnel.
Which team members will be responsible for each project component?
For a successful Business Intelligence solution to be developed, you will need to carefully allocate the
development tasks to appropriate staff or contractors. Consider using a matrix approach as discussed
in the preceding section, using either Office Excel or Office Project as the tool for recording the
information.
After components are listed, along with their detailed technical specifications, team members must be
associated with them based on skills, knowledge, availability, and ability to deliver the required result.
This task allocation also requires that you work out scheduling and resource allocation in some detail.
Goals, tasks, and time frames must be determined in a realistic manner.
At a minimum, you should consider the following when matching component requirements with
developer or other skills:
• Task details—description, purpose, design, security requirements, inputs, outputs, and
processes
• Required technologies and development features
• Skill domain—for example, networking, server infrastructure, coding, query languages,
connectivity, data sharing, ETL, presentation, or reporting
Consideration of all of these factors should ensure that your project has a good chance of being
delivered on time and on budget.
What training is required?
Frequently, projects do not succeed as a result of the failure of organizations to invest in proper
training (for example, in new technologies, including XML and Web services) or retraining (for
example, upgrading development skills from Visual Basic 6.0 to Visual Basic .NET). It is almost
never the case that training expenditure is wasted when it comes to improving and refining
development skills. With Business Intelligence development in particular, it is important to appreciate
that specialized skills beyond what is needed for standard application development are required.

Note
Microsoft recently revised its certification streams to reflect these concerns and has refined the
understanding of developers to take account of special areas of skill, such as Business Intelligence
development. See the Microsoft Learning Web site for further details.

You must deliberately and carefully assess training needs, preferably through a formal training needs
analysis process or other similar process, to determine existing skills and knowledge as compared with
required skills and knowledge.
After you have determined the training needs, you must properly budget for them and allow for them
in terms of project time. Training needs should be anticipated as far in advance of the real project as
possible. Typically, it takes several months for a developer or other specialist to become well versed
in a new technology. Project managers often find that the lead time has been insufficient and
improperly planned, usually in response to budget or other management pressures.
It is particularly important in a Business Intelligence project for training assessment to be performed
by one or more people with sufficient knowledge of the requirements and skills required to effectively
assess the real training need of team members. Staff are not always willing to acknowledge that they
lack skills that they need, believing that they can acquire them in a casual manner during the project
(from books, Web sites, colleagues, or articles). Such learning is usually incomplete, insufficient, and
expensive in real project terms. By contrast, taking specific time out to do particular training is often
efficient, cost effective, and rewarding for both the staff and the project.
Demonstration: Using Source Control in a Business Intelligence Project

Procedure: Use Visual SourceSafe to show how source control does and does not work in a Business
Intelligence environment.
Instructor note
Students may or may not be familiar with Microsoft Visual SourceSafe® and development processes.
Determine which students are. Ask whether anyone has had trouble using it to manage control of
development.

This demonstration is designed to show students how it is possible to bypass the source control
process by using a different means to connect to Analysis Services. It can also be run as a Practice,
depending on the group and your sense of the participants' skills.

In this demonstration, you will open the Adventure Works UDM project in Visual SourceSafe, make
a change to it, note the tracking by Visual SourceSafe, and then deploy it. You will then make another
change to the same project, but this time through SQL Server Management Studio. You will then
show how the change made in SQL Server Management Studio is not tracked by the VSS control
process. Discuss the implications of this for how Business Intelligence projects should be run and
managed.

Describe the goals of Project REAL and mention the aim of sharing information about best practices
and guidelines that project members have discovered about creating BI solutions with SQL Server
2005.

Encourage students to become familiar with the various resource papers from the Project REAL Web
site and point out that there are additions to the Project from time to time.
Introduction
In SQL Server 2005, there are several different ways of working with the Analysis Services
component of the product. For daily management and small changes (for example, to change a
dimension), you can usually use SQL Server Management Studio. However, this is not a development
environment and has limited support for source control. In the SQL Server Business Intelligence
Development Studio, which provides a rich, full-featured development environment, work is usually
done in disconnected mode (the default—detached from Analysis Services) and then deployed. In this
mode, you can use proper source control procedures and tools to ensure that developers do not make
conflicting changes to code and design components. You can also work in the SQL Server Business
Intelligence Development Studio in connected mode, in which it is possible to work against the live
system and bypass both deployment and checkout/checkin procedures.
The consequence of such a range of choices is that, if processes and procedures are not clearly
specified and followed, it is relatively easy to bypass the source control tools and make changes that
conflict with changes that other project developers or database administrators (DBAs) make.
Preparation
Ensure that the virtual machine 2794-MIA-SQL-03 is running and that you are logged on as Student.
If a virtual machine has not been started, perform the following steps:
1. Close any other running virtual machines.
2. Start the virtual machine, and then use the Virtual Machine Remote Control Client to connect
to it and log on by using the user name Student and the password Pa$$w0rd.
Using Microsoft Visual SourceSafe
You must perform the following steps to create a change in SQL Server Business Intelligence
Development Studio that is tracked by Visual SourceSafe:
1. Click Start, point to All Programs, point to Microsoft SQL Server 2005, and then click
SQL Server Business Intelligence Development Studio.
2. On the File menu, point to Open, and then click Project/Solution.
3. In the Open Project dialog box, click SourceSafe (LAN), and then double-click the VSS
database. When prompted, log on to Visual SourceSafe with the user name Student and no
password.
4. Open the Adventure Works UDM.root folder, and then open the Adventure Works UDM
folder and double-click the Adventure Works UDM.sln solution. If you are prompted to
choose between a previously downloaded local version and the server version, click Server
Version.
5. In Solution Explorer, right-click Roles, and then click New Role.
6. Click the new role in Solution Explorer, and in the Properties window, set the File Name
property to TestRole.role. When prompted, click Yes to change the object name as well.
Then save the role. Note that the role has a plus sign (+) next to it in Solution Explorer.
7. In Solution Explorer, right-click TestRole, and then click Check In. Then, in the Check In
dialog box, click Check In. Note that the role now has a padlock symbol next to it in Solution
Explorer.
8. On the File menu, click Save All, then, right click Adventure Works UDM at the root of the
tree in Solution Explorer and click Check In. In the Check In dialog box, click Check In.
9. On the Build menu, click Deploy Adventure Works UDM. Click Yes if you are prompted to
overwrite the existing database, and then close Business Intelligence Development Studio
when deployment is complete.
Bypassing Visual SourceSafe
You must perform the following steps to create a change that is not tracked by Visual SourceSafe:
1. Click Start, point to All Programs, point to Microsoft SQL Server 2005, and then click
SQL Server Management Studio. When prompted, connect to the Analysis Services
instance on MIAMI.
2. In Object Explorer, expand Databases, and then expand Adventure Works UDM.
3. Click the Roles folder, and note that TestRole is listed.
4. Right-click the Roles folder, and then click New Role.
5. In the Create Role dialog box, type TestRole2 in the Role name box, and then click OK.
Then close SQL Server Management Studio.
6. Open Business Intelligence Development Studio, reopen the Adventure Works UDM
project from Visual SourceSafe, and note that the changes made in SQL Server Management
Studio are not tracked in the source control system.
7. Close SQL Server Business Intelligence Development Studio.
Discussion
Use the following questions to discuss the importance of this sequence of events.
Q How could you resynchronize the source-controlled version of the database with the deployed
database?
A Answers will vary. You can open a project directly from the Analysis Services instance;
so as long as no other changes have been made to the local version of the project in
Visual SourceSafe, you might be able to resynchronize the database by using that
technique. Otherwise, you might have to manually modify or create objects in the
source-controlled version and then redeploy the solution.
Q What are the implications of this situation for managing a Business Intelligence development
project?
A Answers will vary. You should focus on the fact that multiple tools are provided for
convenience but that in order to have a coherent and disciplined development
environment, clear procedures and processes must be in place so that all project staff,
including production staff, know the right ways to make changes. If changes occur
outside the Business Intelligence Development Studio, clear communication and
documentation about such changes must exist. One approach might be to use SQL
Server Business Intelligence Development Studio exclusively in the development
environment, and SQL Server Management Studio (which also supports source control
for projects) exclusively in the production environment.

For more information

For more information about managing source control for Analysis Services projects, see Project
REAL: Analysis Services Technical Drilldown on the Microsoft TechNet Web site.
Lesson 2: Managing Business Intelligence Development

Lesson objective:
After completing this lesson, you will be able to:

Manage a Business Intelligence development project.

Instructor note
This is not intended to be a course on project management. It is intended as an overview of key issues
that a Business Intelligence development manager needs to understand when running a Business
Intelligence project.

Ask students what experience they have, if any, in running Business Intelligence development
projects. If any have experience, encourage them to share their own insights on the topics raised in this
lesson. Try not to let the discussion wander too much or turn into just a storytelling episode. The aim
of such a discussion is for students to learn critical lessons from each other about the real problems of
running projects of this nature and scale.

Introduction
Your role as the manager of a Business Intelligence project or as one of several managers of a
Business Intelligence project involves many unique considerations in relation to the ways in which
such management can be performed, the tools required, and the associated skills.
This lesson provides an overview of the features of project management that are specific to a Business
Intelligence context.
Use the information presented here to determine the scope of the tasks that you will need to complete
in your particular project scenario.
Considerations for Planning the Development Process

Principle: Describe considerations for planning a Business Intelligence development project.

Instructor note
Explain why development projects of this nature need to be thoroughly planned and managed.

Ask students whether they are familiar with configuration issues involved with the relevant
development tools.

Describe the requirements for assembling a development team. Identify personnel and skills required.

Identify the considerations for Business Intelligence project management from the design stage all the
way through to deployment and follow-up support.

Describe the benefits of a naming standard in a Business Intelligence project context. Emphasize the
greater need for such standards in a complex, enterprise-level Business Intelligence environment.

Explain why the development environment needs to be configured and the specific considerations
related to doing so. Be sure to identify configuration requirements for all development tools involved
in the project.

Encourage students to discuss their experiences with naming standards and determine whether their
organizations adhere to such standards. Explain why naming conventions are particularly important in
a complex Business Intelligence project environment.
Introduction
Running any major solution development project is a challenging task. This is particularly true of
Business Intelligence development projects because they encompass a wide range of products,
technologies, infrastructure, servers, applications, and business issues.
To be a good Business Intelligence solution manager requires that you have awareness of these
various topics, even if you are not an expert in each one. You must appoint or train the experts that
you require or hire them through a contracting agency.
This section focuses on four main areas that you must address properly to ensure the success of your
project, namely selection of the development team, establishment of project management, adherence
to naming conventions, and proper configuration of the development environment. Failure in any one
of these core development areas can result in a failed or seriously flawed project.
Development team selection
After you have performed the system analysis, general project planning, and business requirements
analysis, it is then time to focus on project implementation. The first step in this process is to select a
development team. To do this, you need to have a list of the skills required, technologies that the
development staff must be familiar with, infrastructure specifications, and information about any other
project requirements that have been identified.
After you compile these lists, select through a systematic and rigorous process the team members who
have the requisite knowledge and skills. Because Business Intelligence projects are often long term
and almost without exception complex, it is important that development staff have the following:
• The ability to see problems from a business perspective rather than a purely technical
perspective
• Good communication skills, oral and written
• Excellent technical skills (according to what your lists require)
• A willingness to learn properly, not in a piecemeal fashion
• A willingness to be trained if skills are missing or out-of-date
• Appreciation of the complexity of Business Intelligence development
• Experience with Business Intelligence development generally
• Experience with the chosen technologies
• Good coding skills, where appropriate
• Excellent problem solving skills
• A commitment to excellent results through teamwork
• A keen willingness to use Help systems and product technical documentation
• A modest attitude and willingness to listen to others
Important
Business Intelligence development is almost always a team effort. It is extremely important that the
team members be able to interact well with one another and with project management and
stakeholders. Particularly if rapid application development (RAD) techniques are used, the ability to
interact well with stakeholders on a regular basis is critical to success of the project.
Project management
The presence of an experienced project manager and project management team is also critical to the
success of a Business Intelligence project. Because a project involves many components, technologies,
and solution steps, management and management processes must be robust and able to deal with all
specific details.
The project needs one or more experienced senior project managers. A project manager can save
enormous amounts of time and money just by being aware of how to juggle the components of a
complex set of tasks. While it is best if such a person has software (and preferably Business
Intelligence software), along with development experience, with highly complex projects these
elements are less essential than good project management experience.
It is often desirable to have more than one such manager; there is typically more than a single project
manager can handle in a Business Intelligence solution, and he or she is likely to be overloaded and
therefore to need help with critical events, tasks, or resource issues.
In addition to deciding on the people for project management, you need to also decide on the tools.
Project management software (for example, Microsoft Office Project and Microsoft Office Project
Server) used in conjunction with workflow, tasking, e-mail, and calendaring, is necessary for
management of such projects. This is important so that tasks, resources, budget, workload, and task
dependencies can all be adequately tracked. Team members also need to be notified of tasks, project
changes, and adjustments to schedules, preferably automatically (for example, by using Microsoft
Office Outlook® and Microsoft Exchange Server), so that errors are less likely to occur through lack
of team awareness or a failure to notify team members of significant changes to any aspect of the
project of which they need to be aware.
Naming and documentation standards
Projects of any type benefit from using systematic procedures and processes. Software projects and
Business Intelligence projects in particular benefit especially from have agreed-upon naming
conventions.
It is important to include the following areas in the development of your naming guidelines:
• Network
• Server naming
• Web site naming
• Object naming in code
• Object naming in all components of SQL Server 2005
• Business Intelligence elements naming (cubes, dimensions, hierarchies, and attributes)
• Database naming
• Report naming
It is necessary not only to have naming conventions in place but also to have policies in place to
ensure that they are used. Developers, in all phases of the project, need to be aware of the naming
conventions and adhere to them. It is important to monitor adherence to naming conventions and fix
any deviations from them at the earliest possible opportunity. Uncorrected, nonconforming names can
endure for many years in a solution and cause recurring problems; it is extremely important to avoid
such a result. It is essential to document the naming conventions, data dictionaries, and other named
objects as a core part of the project. If exceptions to the naming conventions are discovered, it is
critical that they be documented and highlighted as exceptions so that subsequent development can
take proper account of them.
When defining a naming convention, it is important to consider the audience for the object names. It
might be appropriate for database table and column names to use a name format that will be used by
database developers and administrators that encapsulates an object’s type and purpose; but full names,
including spaces, might be more appropriate for objects that business users interact with, such as cube
dimensions and attributes. For example, the name ut_dimProductCategory might be used to name a
dimension table for product data in a data warehouse, but the name Product Category might be more
suitable for the corresponding dimension in the Analysis Services cube.
Additionally, your naming convention and standards should take namespace constraints into account.
For example, in an Analysis Services cube, you cannot use the same name for an attribute and a user
hierarchy. So for example, if an attribute named Department exists, you cannot also create a user-
defined hierarchy with that name. One possible convention is to prefix hierarchy names with the word
By (for example By Department), which resolves the problem of name uniqueness and also indicates
to the user that the hierarchy can be used to analyze data based on the hierarchy (for example, users
can view sales totals by department).
Development environment configuration
Each of the major technologies to be used in a Business Intelligence project needs to be assessed as to
whether they require particular software, hardware, network, or other configuration work.
Specialists in each technology should determine the configuration requirements in your particular
project context. Configurations need to be thoroughly assessed, documented, and implemented. Your
project needs to allow for this process as well as for any training that might be required to enable staff
to perform configuration and configuration management.
The following areas should be checked for configuration requirements:
• Networks
• Servers
• Storage
• Core applications—for example, SQL Server 2005, Visual Studio, and Microsoft Office
applications
• Core technologies—for example, ASP.NET, ADO.NET, and ODBC
• Supplementary tools
• Web services and other relevant Web sites
• Supporting technologies—for example, IIS, Microsoft Internet Security and Acceleration
(ISA) Server, and Microsoft BizTalk® Server ports and channels
• Legacy systems that must interact with the new system
• XML and associated technologies
Most Microsoft products have a Software Development Kit (SDK). These include detailed
information on configuration requirements for each specific product or technology. You should be
familiar with the appropriate SDKs for the products in your solution.
At the hardware infrastructure level, you must ensure familiarity with all key hardware configuration
requirements, such as Domain Name System (DNS), Dynamic Host Configuration Protocol (DHCP),
Active Directory, TCP/IP addresses, communication protocols, replication, backup/restore hardware
and software, security, and operating system–specific requirements (for example, configuration of
services in Windows Server 2003).
Guidelines for Testing a Business Intelligence Solution

Principle: Identify the processes and infrastructure required to properly test a Business Intelligence
solution.
Instructor note
Ask students whether they are familiar with the differences among the various types of testing and the
requirements for each.

Emphasize the importance of involving stakeholders in the determination of pass/fail criteria where
relevant. Ask students to identify such areas.

Discuss the importance of allowing sufficient test time but balancing it with project completion. Ask
students to consider the inherent contradictions in these requirements and how they would resolve
them.

Introduction
When the project has advanced to the state where it is ready for testing, you should do several things
to ensure that such work is done properly and efficiently.
Testing is a unique skill, requires a particular kind of environment, and needs to be performed in a
context of proper procedures and processes to get the most value from of it.
Consider the following facts and guidelines when you are planning and implementing your project
testing.
Create a testing environment
The first thing that is needed in any testing scenario is a test environment. The precise meaning of this
varies somewhat from project to project. In principle, the test environment should be as close to
identical as possible to a real production environment. To achieve this similarity for a Business
Intelligence test environment entails deep consideration of the following factors:
• Infrastructure
• Network, protocols, DNS, DHCP, firewalls, TCP/IP ports
• Active Directory and associated services
• Servers, which should be as close to identical as possible in all respects
• Server instances and roles
• Software, which should all be the same version and have the same service packs and patches
installed
• Security and malware protection software
• Physical network components
• Storage systems, which should be as identical as possible including type, redundant array of
independent disks (RAID), partitioning, and drives
• Domain structure
• Security model, implementation, roles, and groups
• Legacy connectivity
• Intermediate technologies
• User load (simulated or real)
• Demands on bandwidth and data transfers
• Volumes of backup data, which should be identical or nearly so
• Files, file groups, and file placement, which should be the same as production
• Activities and patterns of work flow
• Patterns of transaction processing, which should be as identical as possible (simulated or real)
• Hardware components, which should be identical, particularly with RAID controllers and
network interfaces
In most organizations, it is not possible for testing environments to achieve exact replicas of the
production environment or environments, but this should not deter you from getting as close to that
goal as possible. It is critical to carefully document all of the infrastructure and configuration of your
test and development environments.
It has become convenient and common to use virtual technologies for development and testing. These
can help enormously in the development environment to provide a good simulation of the real
production environment. However, you should always remember that virtual environments have
differences from physical computers that might be important when fully testing a Business
Intelligence solution.
Identify appropriate testers, test matrices, and methods
To test a Business Intelligence solution, you need to ensure that you have fulfilled the following three
requirements:
• The test team needs to have appropriate knowledge, experience, and awareness of business
issues and processes and be alert to the requirements of the ultimate users of the production
system.
• Appropriate tests and test matrices need to be defined and implemented to ensure coverage of
all aspects of the solution.
• Testers need to know, or be trained in, correct testing procedures because errors in testing can
be costly if problems in the solution are not caught in the testing phase.
There are many books and articles on how to perform software testing in a thorough and
comprehensive manner. If you are not already familiar with such resources, you should ensure that
your Business Intelligence project team uses them and puts in place a comprehensive software testing
and quality assurance program.
Establish bug tracking and resolution procedures
One of the most important features of a software testing regime is the tracking of bugs and bug
resolution. Many tools are available for such purposes, and most developers and testers are familiar
with ways of doing bug finding and bug fixing.
However, bug reporting procedures need to be as stringent and detailed as possible, given the time and
budget constraints of the project. Bugs must be fully described and all relevant information provided
to avoid wasting the time of both the testers and developers who must fix the bugs. Bug resolution
must be equally detailed and precise.
Bugs should be properly numbered, sequenced, and described and the resolution of each bug paired
with the original bug report so that a complete history of bugs and fixes exists, in proper chronological
order.
Set criteria for testing pass/fail with stakeholders
To test effectively, it is also important that you agree with stakeholders and users on the criteria to be
used for passing and failing component testing. Ultimately, the users who work with the system in
daily business activities need to know that they have a reliable, easy-to-use, and well-presented
application. A badly designed interface can destroy the most technically sophisticated project simply
by annoying users with too much or too little information. This can be further aggravated by poor use
of color, fonts, layout, and forms design.
You should ensure that stakeholders and users are involved in designing and testing the testing
regime, bug detection and identification, and bug resolution criteria. Also, no matter how trivial the
input from these parties may seem, the feedback should be listened to very carefully. Many a project
has failed after deployment because users refused to use it, had not been consulted by developers, or
were listened to but misunderstood.
Set time frames for testing and problem resolution
Software testing can be time consuming and expensive—particularly for Business Intelligence
projects because of their complexity and infrastructure requirements. You should establish realistic
test time frames with developers, testers, and users. Rushing through testing is dangerous, but there
are serious negative consequences of taking too long as well.
Your aim should be to establish a testing environment and bug resolution regime that keeps testing to
limits that are reasonable in terms of your overall project design, organizational context, budget, and
staff availability.
To do efficient testing and problem resolving requires excellent racking mechanisms, documentation,
and good communication among users, stakeholders, and developers. There also need to be appeal
processes so that disputes about features, priorities, or bug resolution can be fairly and properly
mediated.
Identify types of testing required
Many different types of testing can be performed. You must ensure that you and your development
team have planned to perform appropriate testing. The following are among the main types of testing
that you should consider, depending on the project component or process being tested:
• Functional testing
• User acceptance testing
• Load testing
• Revision testing
• Stress testing
• Accessibility testing
• Performance testing
• Technology specific testing (for example, replication or message queuing)
• Bug resolution regression testing
Clearly, testing is crucial to the success of your Business Intelligence solution project. Use all the
means at your disposal to ensure that it is done well and in a cost-effective manner.
Guidelines for Deploying a Business Intelligence Solution

Principle: Describe guidelines for deploying a Business Intelligence solution.

Instructor note
Ask how many students have deployed a major project.

Ask how many have deployed a Business Intelligence project, and then get the students who have to
describe the size and complexity of the projects.

Encourage students to discuss the manner in which deployment guidelines as described here can assist
in risk management and reduction of mistakes of omission or commission.

Introduction
After your solution has passed all of the necessary testing, you need to deploy it in the production
environment. By the time you have done deployment in the final pre-production test environment, it
should be clear what, if any, issues need to be addressed or resolved to deploy the solution and its
components.
To successfully deploy the project solution, you need to be certain that the required infrastructure is in
place and properly configured. You also need to plan the deployment in detail, including what will be
done in what order and by whom.
Determine infrastructure requirements
Deployment infrastructure needs to be correct in terms of all the factors discussed previously. If the
determination of infrastructure has been done properly, these issues have all been addressed long
before deployment, during the detailed project planning and estimation processes. If this is not the
case, the project needs to assess or reassess the infrastructure and then put it in place, properly
configured and with appropriate administration tools and procedures. Careful documentation of test
and development environments can prove very useful at this stage.
If RAD development has been used, there should be few surprises when it comes to deployment time.
If traditional development has been performed, with less iterative testing, it is possible that
infrastructure choices have evolved, changed, or become unavailable (as can often happen with
changes in workstation or server hardware). The project needs to be certain that infrastructure is
ordered well ahead of deployment and is installed and fully tested so that when deployment is to be
done there are no major issues with regard to hardware, software, or configuration.
Identify staging requirements
In many enterprise environments, you might need to stage some components of the Business
Intelligence solution before deploying them into production. For example, you might create a staging
server on which you deploy an Analysis Services database and then process the cubes in the database
before synchronizing them with the production servers. A staging environment can be particularly
useful when you need to prepare a component of the BI solution for deployment into a Windows
Network Load Balancing (NLB) cluster because it provides a single location in which final
configuration changes can be made before deploying the solution to multiple servers.
Plan deployment
Careful planning of the production deployment must take place from an early point in the project. This
must occur for many practical reasons related to budgets, purchasing, staffing, training, and other
support.
Such planning needs to take into account all associated systems, data connections, data transfers or
ETL routines, validation issues, security and access permissions, transitional arrangements (if older
systems are being downgraded when the new system comes online), and any other issues that are
pertinent in your particular installation. Examples of such issues requiring further considerations are
air gap security (often required in government agencies), firewalls, biometrics, certificates, or public
key infrastructure (PKI).
Whatever issues the deployment is likely to encounter should have been anticipated, planned for, and
handled in an expeditious fashion. The ultimate test of a well-managed project is the success it has in a
flawless deployment. It is a good idea to arrange to do one or more mock deployments to guarantee
that all processes, procedures, checklists, and personnel are ready for the real deployment.
Plan change control
All projects undergo changes after designs have been finalized. Be sure to put in place a change
control process that has clear rules for communicating change requests, recording changes performed,
and identifying any flow-on consequences that need to then be handled.
Change control is often a weak point in many projects. It is also a critical element of effective project
management, so it is important to put in place policies and procedures that are clear, are understood by
the entire project team, known to stakeholders, and well documented. Failure to properly record
change requests, resolve them, and document them can cause confusion to project staff and frustration
to users and result in the delay of project deadlines and deliverables.
Be sure to use an appropriate tool to manage change control. Microsoft Office Excel, a database in
Microsoft Office Access or SQL Server, or more specialized products can all provide such
functionality.
Lesson 3: Determining Data Management Processes

Lesson objective
After completing this lesson, you will be able to:

Determine effective data management processes.

Instructor note
Ask what background students already have in managing enterprise data. Discuss with them the
various topics that are addressed in this module, and ask them to share their experiences with each
topic.

With ETL routines, ask students what they currently use and determine whether they have automated
the processes. Identify what range of source systems they are familiar with.

This is an overview of the issues from an architecture standpoint. Detailed content on specific
techniques is covered in a number of other Microsoft Learning SQL Server 2005 courses. You should
encourage students to take these courses if they have not already done so.

Introduction
The core material used by a Business Intelligence solution is the corporate data of the business. The
data may reside on a wide range of systems and in many locations and be collected through a broad set
of applications and automated processes.
Regardless of how the data is collected by the organization, it needs to be stored in appropriate
databases or files and processed according to business requirements. The most fundamental
requirements are that data be safely stored and that reliable backup and restore procedures be in place.
However, for the data to be useful for the business, it also, of course, must be valid. Validity checking
can be done at various points in the data handling processes—at data entry, during import and export,
before specific use, or at any other time deemed appropriate by the organization. Without valid data,
most analyses are of little value, so validation processes are crucial. Furthermore, particularly in
Business Intelligence scenarios, invalid data can seriously distort business analyses and forecasting,
which in turn, if acted upon, can place the business in danger financially or operationally.
Your task as a Business Intelligence project manager is to help ensure that data is stored correctly, is
valid, and is available for whatever business requirements have been decided on as goals for the
project.
After completing this lesson, you will know how to manage data issues effectively for Business
Intelligence purposes, including what you should consider for ETL routines, data validation, data
lineage, and reporting.
Guidelines for Managing Data Issues

Principle: Describe how to manage data issues effectively.

Instructor note
Explain the importance of data planning in a Business Intelligence environment. Relate this to the
various views (philosophies) discussed at the beginning of the course in order to emphasize that
different views lead to different scoping of Business Intelligence projects.

Identify tools that can help to implement these guidelines, including Visio, Excel, and SQL Server
Integration Services and SQL Server Management Studio. Ask students what experience they have
with these tools with regard to data handling.

Discuss the ways in which SQL Server job automation can support and enhance data management in a
Business Intelligence project solution.

Introduction
Data handling falls into a number of distinct areas, each of which requires specific management,
policies, and procedures. The core areas are discussed here with a focus on the steps that you should
take to ensure that each area is performed systematically and in a manner that will support a Business
Intelligence project solution. It is important to appreciate that some of these matters are more of an
issue in Business Intelligence projects than in standard online transaction processing (OLTP) data
scenarios. They may differ in the volume of data, the length of time for which it is held in active data
stores, and specific requirements for tracking data adjustments and changes.
From each of the following general guidelines, you should derive the specific guidelines to implant in
your Business Intelligence project and determine how they are to be implemented, when they are to be
used, and who is assigned to manage them.
Plan for ETL routines
Data that ends up in data warehouses and data marts always starts off somewhere else. The set of
processes for exporting data, manipulating it, and importing it into a destination application is
collectively referred to as an ETL routine. Many different tools can be used for such activities. Use the
following list to assist in your review and planning for determination of your project-specific ETL
requirements and processes.
• Identify source systems and their characteristics.
• Document connectivity required between source and destination systems.
• Establish staging data stores if they are required.
• Identify destination systems that receive data from the ETL processing.
• Assess storage requirements for all systems involved in ETL.
• Document characteristics of source systems at the data source level (for example, data store
names, field names, data types, security, and access permissions required for ETL use).
• Document characteristics of destination systems at the data destination level (for example,
data store names, field names, data types, security, and access permissions required for ETL
use).
• Determine network characteristics, bottlenecks, and bandwidth.
• Select tools for ETL planning, deployment, and management.
• Identify what can be automated and what must be done manually, and the team members to
be involved.
• Document your plan, and change control processes in an appropriate manner.
• Design, test, refine, and deploy your ETL solution.
Adherence to these guidelines will help to ensure that your ETL processing is well designed, executed
properly, and well documented. ETL routines often change as business processes and requirements
evolve, so it is particularly important for ETL to be well documented and for revision and change
control policies and procedures to be in place.
Determine transformation requirements
Particularly in data warehousing and Business Intelligence projects, there are requirements that entail
passing incoming data through various transformations to fulfill the structural requirements of data
cubes, hierarchies, and dimensions.
You should determine the following for your particular project:
• Types of transformations required
• Specific data involved—input data and output data formats, types, and characteristics
• Transformation mechanisms—scripts, data conversions, and text manipulation
• Frequency and schedule required
• Supporting infrastructure for staging and processing
• Exception handling
• Data lineage requirements
You should use appropriate tools to design and implement the transformation environment and
sequences. For many projects, the implementation tool of choice will be SQL Server Integration
Services, but this should not be assumed to be the only tool. It may also be necessary to use other tools
such as Microsoft Host Integration Server, BizTalk Server, or third-party proprietary drivers or
products.
Regardless of the technologies chosen, the Business Intelligence project solution should be robust and
automated and should provide feedback mechanisms to indicate completion of processing, errors, and
exceptions that require further action.
Assess data volumes
Every Business Intelligence project has to determine the volume of data to be processed and stored.
Your challenge is to properly assess these volumes, identify where they are sourced and where they
are to be stored, and when they are going to vary. Variation in volumes due to business processes,
seasonal variations in data patterns, business changes, product evolution, and many other factors can
affect data volumes.
To assess data volumes, you should identify the following:
• Data types and numbers of transactions to calculate estimates of volume
• Data patterns by business process, time of year, department, section, and location
• Nature of data storage required – relational, XML, multidimensional, raw, transformed,
calculated, or cube
• Data volume as affected or required by associated systems—supply chain, accounting,
ordering, scientific, regulatory, inventory, and maintenance
• Storage media and technologies
• Storage locations
• Retention requirements
• Archiving requirements
In your specific Business Intelligence environment, there may well be other factors that will affect the
data volumes to be handled. Your project team should be certain that it has identified all such relevant
factors and assessed their impact on processing times, processing methods and applications, storage,
data transfers, and backup routines. Use Excel to calculate data volumes based on your assessments of
the items listed previously as well as from the various properties exposed through database and file
characteristics.
Document the logic of the volume calculations along with the information itself. Identify the means by
which the information can be updated as data patterns change. Specify mechanisms for monitoring
volume and volume changes as well as policies and procedures for managing the variation of data
volumes.
Determine lineage requirements
Most Business Intelligence systems have some type of data lineage requirement, usually for specific
auditing purposes, but it can also be useful for process analysis or error determination. Whatever the
reason, lineage is the ability to track data from source to final use, including any changes that occur
during processing stages or steps. In the course of planning your project, you should find out from
stakeholders what, if any, lineage requirements they have. For those who have such a requirement,
you need to determine the level of tracking detail required and the method of implementation.
You should familiarize yourself with the lineage options available in the tools that your project has
decided to use.
Determine reporting and presentation requirements
In the end, a Business Intelligence project is only as good as its ability to deliver information to
stakeholders and users in ways that meet requirements and business needs. You should assess the
reporting requirements carefully and establish processes for identifying changes in them. It is
routinely the case that as stakeholders and users become more familiar with the capabilities of
Business Intelligence, their expectations and associated requests grow significantly in both number
and complexity.
Reporting patterns and products can vary widely by the reporting technologies in use for the project.
The development team should help to educate users and stakeholders about the reporting options and
alternatives that are available. Among the considerations to assess are whether the reporting solution
supports Web delivery, e-mail delivery, a variety of output file formats, and easy integration with
other products in use in the solution.
Establish review processes for data issues
For all data related topics and issues, the project should also have in place review processes to ensure
that as the project is deployed and evolves, any additional data handling or presentation requirements
are captured and able to be considered by the project managers and development teams.
Because such changes may have significant impact on the Business Intelligence solution, you must
have support from the project owners and stakeholders to ensure that changes are considered. When
considered changes are decided upon, you then must have clear procedures in place to support the
development and deployment of the changes. Along with such changes, you may have other
requirements such as additional infrastructure, software licenses, or training for users.
Considerations for ETL

Principle: Describe considerations for ETL.

Instructor note
Without proper ETL any Business Intelligence project will fail. Describe in as much detail as possible
how critical ETL is to the efficient functioning of a Business Intelligence solution.

Discuss with students their experience with ETL processing, including types of data, source systems,
and destination applications. Ask them to share any problems or issues that they have encountered
with such ETL activities.

Introduction
The specifics of ETL routines and processing need to be detailed precisely and completely. These
routines drive the Business Intelligence solution of any organization, whatever its nature. It is
important to consider the major areas of ETL planning discussed here.
Sources and categories of data
You should identify all of the sources of data that are to be used in the project and consider what the
implications of each source are. The nature of the source may lead you to consider particular
connection technologies, software libraries, application programming interfaces (APIs), ODBC
drivers, or OLE DB libraries.
There are many variations on these in detail and it is important to establish the nature of your
particular sources; sources of data tend to fall into the following main categories:
• Legacy systems—mainframe computers and associated database technologies, including flat
file and hierarchical databases
• Desktop systems—text files, spreadsheets, accounting packages, and databases.
• Server systems – databases, financial systems, and customer relationship management (CRM)
and enterprise resource planning (ERP) systems
• Manufacturing systems—automation, production control, ordering, and fulfillment
• Sales systems—sales tracking, shipping, inventory control, and marketing
• Web services and other Web-based systems and associated XML, relational, and text data
stores
• Miscellaneous tracking systems
Within these many sources will be a wide range of data types and data formats. In each particular
case, you will need to know the nature of the data at this specific level so that the ETL routines can be
properly configured and coded.
You might need to examine sources if they have not been well documented or if the people running
them are not sufficiently technically aware. Microsoft Office Excel and Access have particularly
useful import tools that have evolved over many years and can often save considerable guesswork and
time in sorting out the details of the data. They can also provide a rapid testing environment where
sample data loading and data typing can be readily tested. After the details are established, you can
use these tools or SQL Server Integration Services to do the final processing.
It might also be the case that the data sources and processing requirements suit data-driven technology
solutions by which the ETL routine is responsive to the data being used. This type of approach can
provide a powerful and flexible solution but usually takes considerably more effort to develop.

Tip
Microsoft has done a great deal of research into ETL and data transformation. You should familiarize
yourself not only with what is discussed in SQL Server Books Online but also with the extensive
resources on Microsoft MSDN® and in particular the information about Data Transformation Services
(DTS) and SQL Server Integration Services package design and development. Search on the keywords
DTS packages, SSIS packages, and data-driven queries.
Data types in use
For each ETL routine, you will need to ascertain the data types being used in the source data and
know the type of data that you want to have in the destination system. This will not only inform the
choices for project data transformations and conversions but also will enable you to estimate data
volumes more precisely.
With each major category of data, there are particular considerations, for example:
• Numeric data—precision, scale, and monetary types
• Text data—fixed or variable length and encoding
• Binary Large Object (BLOB) data—size, type, and update requirements
• Logical—representation
• Binary—size and encoding
• Object—type and characteristics
The more precisely the project team understands the data, the better it will be able to develop
appropriate and efficient solutions for data handling.
Data
As previously discussed, the nature of data volumes, update patterns, and transaction patterns must be
established so that the project can cater to the particular needs in each ETL case. The precise contents
of the data can have a significant impact on ETL. For example, text fields might require splitting or
concatenation, numbers might need to be rounded, numbers of one type might need to be converted to
a different type, or images might need to be stored in native format or converted. The better that your
development team understands the precise nature of the business and its use of the data in question,
the less likely it is that it will handle it inappropriately.
Packages, scheduling, and security
ETL packages need to be managed with regard to scheduling of use or execution. They also need to be
managed with regard to security because processing usually has the potential to expose critical
business rules or logic. Such information usually needs to remain confidential to the organization and
may even need to be confidential within the business (for example, intelligence services, taxation
systems, and medical records systems).
For each ETL process, you will need to assess the nature of these requirements and the particular
means of delivering the solutions for them.
Considerations for Data Validation

Principle: Describe considerations for validating data.

Instructor note
Depending on the background of students, they may be able to relate their expertise in other areas
(such as economics, IT, statistics, and survey research) to the importance of data validity. Explain that
there are many types of validity and that each type involves different types of validity tests.

Use the notes to describe the various major types of validity and use the examples provided. If you are
comfortable with extending the discussion, it will help students to appreciate the complexity of
validity; if this is not the case, stick to the course notes fairly closely.

You should be prepared to discuss validity in some detail, so it is important to develop examples from
your own professional experience to use for discussion.

Introduction
Data validation is a significant part of any BI solution. When planning and designing an ETL process
for a BI solution, it is important to allocate time for examining source data, assessing its validity, and
identifying any data inconsistencies or issues that your ETL solution might need to address. To
effectively validate data, you must first establish the validation requirements for the data, and then
determine how best to check that the data meets those requirements, and what to do with invalid data.
Validation requirements
In the first instance, any Business Intelligence project needs to determine the data to be validated. The
validation requirements can be determined by the following:
• Legal requirements
• Numeric accuracy
• Financial regulations
• Data type
• Conversion requirements
• Ultimate purpose
• Security or secrecy requirements
• Timeliness of data
• Timescales
• Trend analysis requirements
• Graphing, charting, or other presentation requirements
• Pattern matching
• Logical tests
• Functions or calculations
• Data format
• National language or other cultural requirements
You should be prepared to use this list as a starting point for assessing your validation requirements.
However, you should also be aware that many other types of validity might be relevant. In social
research, there are additional types, such as conclusion validity, internal validity, external validity, and
cultural validity. Depending on the field of endeavor, other types of data validity may also be relevant.
Not all users appreciate the distinctions between referential integrity, domain integrity, entity integrity,
and user-defined integrity and their relationships to data validity.
Stakeholder and user involvement in assessing validation requirements
When assessing validation requirements, it is particularly important to have relevant organizational
input into the decisions. The stakeholders and users are the most likely to know what data validity
problems need to be addressed for their data and its use. However, not all stakeholders are equally
aware of data validation issues, and project managers and architects may need to ensure that users are
aware of what can invalidate data and work with them to establish the correct checks on data validity.
Approaches to data validation
In addition to determining the type of validity, you should determine which mechanisms should be
considered for use in the validation processes. The specific options will depend on the technology
being used for the validation. For example, in SQL Server 2005, validation can be enforced by data
types, constraints, triggers, indexes, and stored procedures.
Validation can also be done through code, by visual inspection, by checking row counts, and by
comparing input values against output values, either in data calculations or in terms of rows returned.
Whatever the mechanisms chosen, the purpose is the same—to provide to the Business Intelligence
solution data that actually represents what it purports to represent and to do so accurately within the
boundaries determined by the organizational requirements. From a project management perspective,
you need to ensure that data validity and validation are specifically addressed and that the
appropriately skilled people are available to work on the data validation processes, whether they be
coders, statisticians, social researchers, auditors, or people with any other required skills and
knowledge.
In addition to validating data as it is extracted from the data sources in your BI solution, you should
determine a mechanism for confirming that the data has been successfully transferred to the data
warehouse. Approaches that you can use to achieve this include logging rowcounts and lineage
information as part of the ETL process, and designing a regular report that compares the results of a
SELECT * query in the source systems with an All level MDX query in an OLAP solution. By
regularly running this kind of data verification report, business users can be confident that the data
used for analysis is consistent with the data in the production line of business systems.
Guidelines for Managing Lineage

Principle: Determine lineage requirements and considerations for implementing lineage support.
Instructor note
Ensure that students understand what is meant by lineage before describing the guidelines in this
topic.

Introduction
Lineage is essentially a matter of tracking data changes over time. The precise natures of requirements
for lineage are often closely related to auditing requirements. However, they may also be related to
research or quality assurance requirements. Consequently, it is important to consider the reason or
reasons why lineage is being specified as a project requirement.
An example of data lineage use would be the inclusion of an identifier in each import of new data into
the data warehouse that could be used to determine the data source and the specific instance of the
ETL process that loaded the data. This lineage identifier can be useful in a number of ways, for
example, in identifying the rows that were inserted by an ETL process that failed partway through—
therefore making it possible to back the changes out before re-running the entire ETL process.
Determine lineage requirements for stakeholders
For each stakeholder or group of stakeholders, the project planners should determine whether there are
any lineage requirements. Usually this can be determined simply by asking the stakeholders
themselves about this. However, if you or your team has extensive experience in this, you may find
that you can see areas where lineage tracking would be beneficial for particular users or purposes.
In general terms, lineage is about being able to trace data from the data warehouse back to the original
source, as well as to be able to identify any and all transformations or changes that have occurred
along the way from source to warehouse. Lineage is a major practical and theoretical issue for data
warehousing and Business Intelligence; a full discussion is well beyond this course. Transformations
can include aggregations, conversions, calculations, and other changes, including errors or malicious
changes.
Select lineage types and mechanisms
If lineage is required, it is important to ascertain what kind of lineage tracking is required. This will
depend on the product being used as well as on storage or security requirements. These need to be
comprehensively assessed and must be considered early in the project because they may impact the
final technology decisions.
Determine storage requirements to support lineage
After lineage requirements have been determined, the project managers will need then to determine
what amount of storage is required and for how long. Because lineage is essentially an audit trail, it
involves the storage of considerable quantities of data.
It is important to do appropriate pilot studies and testing of data lineage to confirm the infrastructure
that will be required for this storage. In addition, the project planners should determine how lineage is
to be used in terms of both frequency and pattern because these considerations may also affect the
decisions as to how to do lineage tracking or which products to use.
In SQL Server 2005, data lineage is managed through SQL Server Integration Services.

Tip
The SQL Server group has developed a sample package that illustrates data lineage issues and
solutions. You can find this package on the Capture Data Lineage Package Sample page on the MSDN
Web site.
Set policies on degree of lineage support, cube design, and storage
Due to the nature of the storage requirements that result from data lineage tracking, you will need to
consider the policies and procedures for the use of data lineage. Data lineage requirements also have
an impact on development activities and solution design, so the project development team needs also
to discuss the ways in which lineage tracking impacts components, routines, and particularly
performance or query behavior.
Determine security policies for lineage
Because data lineage provides a trail back to source data, it may also expose critical data to the risks
of being seen or used inappropriately. Consequently, it is important to ensure that use of lineage data
and access to it are properly controlled through security and permissions management. There may also
be issues of privacy and other legal constraints in some types of Business Intelligence solutions (for
example, insurance or health care). So the project managers need to be sure to handle those issues as
well.
Guidelines for Reporting

Principle: Determine reporting requirements, volumes, and supporting infrastructure required.

Instructor note
Ask which students have already used SQL Server Reporting Services.

Asks students to discuss how they would go about implementing the guidelines in their own
organizations.

Discuss with students what reporting experiences they have had. Be prepared to discuss reporting in
some detail because not all students have in-depth reporting experience.

Introduction
Reporting is one of the most demanding areas of a Business Intelligence project. Reports are the
primary means of sharing information with the organization, stakeholders, users, managers, and other
interested parties.
There are several key things to remember when working on the reporting parts of your Business
Intelligence solution. Use the following guidelines to assist with the development of reports for your
project.
Determine number, size, type, and frequency of reports
Reports can vary on many dimensions. Most critically, the number and size of reports can determine
the amount of infrastructure that must be available to support reporting requirements. Reports can
range in size from less than a page to thousands of pages of output. There may be requirements for
multiple copies or for security tracking. Reports can be presented in different types of formats, from
Microsoft Word documents to Excel spreadsheets to PDF files. Each type has its own particular
characteristics.
The frequency of reports and their timeliness can impact server infrastructure decisions and storage if
snapshots are required for particular time-bound or other “static” requirements. The reporting
development team needs to take all of these factors into account when designing the reporting parts of
the solution.
Assess history and retention requirements
For some reporting environments, there are also business and legal constraints that require that reports
be held for a long time (for example, taxation records) or stored in a particular manner. When
designing reports and the supporting processes, you should determine precisely how long the reports
need to be kept, in what format, and with what supporting data. These issues will in turn determine the
manner in which the data is stored, whether they are archived or kept in a live data store, and when the
storage can be freed by further archiving or deletion.
Review infrastructure for reporting requirements
Infrastructure required to support reporting may be quite substantial in large organizations and may
involve multiple servers, extra mail systems, or massive storage. Depending on methods of report
presentation, there may also be requirements for Web farms, printers of different types and sizes,
plotters, photographic storage, scanners, and so on. Consultation with stakeholders is essential in
determining these requirements because they should have a reasonable sense of their presentation
requirements from which you can calculate the infrastructure required to support this aspect of the
Business Intelligence solution.
Additionally, you must identify the security requirements for your reporting solution and its impact on
infrastructure and configuration. Specifically, you must determine how users will be authenticated and
implement the appropriate authentication configuration for the reporting system (for example by
enabling the appropriate authentication settings in a Reporting Services IIS Web application). You
must also identify users with similar requirements and group them into roles to help you manage
authorization. Finally, you must identify the requirements for data encryption and ensure you
incorporate the required SSL certificates and configuration into the solution.
Determine how snapshots can be used
There may be opportunities or requirements to use snapshots to capture particular points in time, in the
sense that data is frozen for a specific reporting purpose (such as the end of a financial year for
accounting data). Snapshots require both design and storage. You should assess the snapshot needs in
your project and determine the amount of storage that is required and the frequency of storage.
Retention of snapshots needs to be managed, as does security.
All of these factors make reporting a core activity of a Business Intelligence project, to which
adequate resources, planning, and time need to be committed.
Use SQL Server Reporting Services design and management tools
SQL Server Reporting Services 2005 has an extensive set of development and management tools
within the product. If you are using SQL Server Reporting Services, you should use these tools to
ensure that you get the rapid development benefits, the easy deployment processes, and the flexibility
for both formats and scheduling of report execution.
For more information
SQL Server Reporting Services is covered in more detail in other Microsoft Learning courses. You
should ensure that if you are using SQL Server Reporting Services as one of your solutions that team
members are properly trained as report developers, managers, or users, as required. It is a powerful
product with many features and benefits and usually requires specific staff who are dedicated full time
to those roles.
Lab: Business Intelligence Development

Time estimated: 60 minutes

Instructor note
The aim of this lab is to get students to think about some of the issues raised in the module, and in the
time available to work out a reasonable response to the scenario while addressing the issues raised.

This lab has two exercises. Students should complete this lab in groups of two or three, although they
can also complete it individually. You should review the answers at the end of each exercise with the
class. There are no definitive right or wrong answers. The aim of the exercise is to ensure that students
understand the range of issues involved and the importance of doing the processes properly and
thoroughly; students can make different assumptions based on the input documents and their own
experiences.

In Exercise 1, students will examine resumés for a number of employees. Based on the available
information, they will then select candidates for senior roles in a Business Intelligence project and
identify any additional training requirements. Students should initially select the roles and candidates
individually and then work together in small groups to reach a consensus. Finally, each group should
present its conclusions to the rest of the class. Students should spend approximately 30 minutes on this
exercise.
In Exercise 2, students will work in small groups to complete a naming standards document for the
database, SQL Server Integration Services, SQL Server Analysis Services, and SQL Server Reporting
Services objects required in the Business Intelligence solution. Students should spend approximately
30 minutes on this exercise.

Scenario
You have begun planning a Business Intelligence solution for Adventure Works, and now you must
assemble a development team and establish development standards for the project.
Introduction
In this lab, you will determine the development roles that are required and select appropriate
employees for those roles based on their resumés. You will then identify any additional training that
will be needed. After you have selected a senior development team, you will define naming standards
that you would like the team to adhere to for Transact SQL, Integration Services, Analysis Services,
and Reporting Services objects.
Preparation
Ensure that the virtual machine 2794-MIA-SQL-03 is running. Also, ensure that you have logged on
to the computer. To log on to the computer, use the following credentials:
• User name: Student
• Password: Pa$$w0rd
Tip
For best results, adjust the screen resolution of the virtual machine to at least 1024 x 768 pixels and
switch the virtual machine to full screen mode.
Exercise 1: Identifying Team Requirements
Introduction
In this exercise, you will review employee resumés and select appropriate team members for the
senior developer roles in a Business Intelligence project. You should perform the first three steps
individually and then compare your findings with a small group of other students, and then work as a
group to reach a consensus for the project personnel recommendations.
Scenario
Adventure Works needs to assemble a Business Intelligence development team. You must identify the
senior developer roles required for the project and evaluate potential candidates for those roles. After
you have selected the most appropriate employees, you must identify any additional training
requirements that must be met to ensure the success of the project.
Identifying team requirements
Summary Specifications
1. Working individually, review the • The file VisionAndScope.doc is in the
vision and scope and requirements D:\Labfiles\Starter folder. This file
documents and the solution contains the vision and scope for the
architecture diagram for the Business Business Intelligence solution.
Intelligence project.
• The file Requirements.doc is in the
2. Working individually, identify the D:\Labfiles\Starter folder. This file
main components of the overall contains the requirements for the
solution and the senior developer roles Business Intelligence solution.
required for them. Record the roles that
you have identified in the project • The file BISolutionArchitecture.vsd is
personnel recommendations document. in the D:\Labfiles\Starter folder. This
file contains a solution architecture
3. Working individually, examine the diagram for the Business Intelligence
employee resumés and select the most solution.
appropriate candidate for each role.
Record your selections in the project • The file Project Personnel
personnel recommendations document. Recommendations.doc is in the
D:\Labfiles\Starter folder. This file
4. Working in a small group, compare contains a template for your proposed
your selections and reach a consensus project roles.
about the roles and appropriate
employees. • The file Employee Resume
Summaries.doc is in the
5. Working in a small group, identify D:\Labfiles\Starter folder. This file
additional training requirements for contains resumés for a number of
your proposed developers and record Adventure Works employees.
them in the project personnel
recommendations document.
Discussion questions
Q What senior developer roles did you identify for the project?
A Answers will vary. One approach is to allocate a lead developer to each significant
component of the project. For example:
• Lead Analysis Services Developer
• Lead ETL Developer
• Lead Reporting Services Developer

Q What factors affected your choice of team members?

A Answers will vary. Suggested factors might include:
• Experience with relevant technologies.
• Familiarity with the business context.
• Certifications.
Exercise 2: Determining Naming Standards
Introduction
In this exercise, you will define naming conventions for your Business Intelligence project.
Scenario
You want to ensure consistency throughout all areas of the business intelligence project, and so you
have decided to define naming standards for developers to use when implementing the solution.
Determining naming standards
Summary Specifications
1. Review the list of objects in the • The file Naming Standards.doc is in
naming standards document. the D:\Labfiles\Starter folder. This
2. Working as a group, identify naming document contains a list of object types
standards for each object type. for which you must identify naming
standards.
3. Document your naming standards, and
provide an example for each standard
in the naming standards document.

Discussion questions
Q Why are naming standards important in development projects?
A Answers should include the benefits of predictability, consistency, improved
communication, risk reduction, and easier maintenance.
Q What factors affect your choice of naming standards?
A Answers will vary. Suggested answers include:
• The need to indicate the purpose and type of object.
• Limitations of the object namespace.
• The users who will use the object.
Q Who can you ensure that naming standards are used?
A Answers will vary. Suggested answers include:
• Defining simple, logical, and meaningful naming conventions that developers find
intuitive.
• Communicating naming standards clearly to all relevant project participants.
• Performing code reviews and raining bugs for non-compliance with naming standards.
Turning off the virtual machine
When you have completed the lab, turn off the virtual machine and discard the undo disks.