CIO Executive Board

Information Technology Practice
Cio Executive Board
Overcoming
the Insight Deficit
Big Judgment in an Era of Big Data
© 2011 The Corporate Executive Board Company. All Rights Reserved.

Information Technology Practice
Cio Executive Board
Executive Directors Directors
Content Publishing
Shvetank Shah Aron Kuehnemann Solutions
Warren Thune Tim Macintyre Production Designer
Managing Directors Dorota Pietruszewska Carolyn Lamond
Jaime Capellá Kristin Sherwood
Contributing Designer
www.executiveboard.com Brian Foster Associate Directors Supriya Dhasmana
Andrew Horne Brent Cassell
Editor
David Kingston Christiane Groth
Nidhi Vikram Choudhury
Jay Shankavaram Karolina Laskowska
Senior Directors Consultants
Vimarsh Bakaya Kelly Chambers
Miles Gibson Marina Murray
Bill Lee
Senior Analyst
Chris Mixter
Pallavi Goel
Rumki Saha
Shilpa Pental
Alex Stille
KD Weitzel Analyst
Raf Gelders
Project Manager
Matthew Charlet
Shalini Das
Copies and Copyright Legal Caveat

As always, members are welcome to an unlimited number of The Corporate Executive Board Company has worked to ensure
copies of the materials contained within this handout. Furthermore, the accuracy of the information it provides to its members. This
members may copy any graphic herein for their own internal report relies upon data obtained from many sources, however, and
purpose. The Corporate Executive Board Company requests only The Corporate Executive Board Company cannot guarantee the
that members retain the copyright mark on all pages produced. accuracy of the information or its analysis in all cases. Furthermore,
Please contact your Member Support Center at +1-866-913-8101 for The Corporate Executive Board Company is not engaged in
any help we may provide. rendering legal, accounting, or other professional services. Its
reports should not be construed as professional advice on any
The pages herein are the property of The Corporate Executive
particular set of facts or circumstances. Members requiring such
Board Company. Beyond the membership, no copyrighted materials
services are advised to consult an appropriate professional.
of The Corporate Executive Board Company may be reproduced
Neither The Corporate Executive Board Company nor its programs
without prior approval.
are responsible for any claims or losses that may arise from a)
any errors or omissions in their reports, whether caused by The
Corporate Executive Board Company or its sources, or b) reliance
upon any recommendation made by The Corporate Executive
Board Company.
CIO0361911SYN
TABLE OF CONTENTS
Executive Summary • iv
An Information Management Strategy on a Page • vi
Assess Insight IQ in Your Organization • vii
Occasion for the Research • 1
Enable Information Usability • 25

Capability-Driven Tool Portfolio (Blue Cross Blue Shield of North Carolina) • 33
Flexible and Transparent Analytics (GlaxoSmithKline) • 47
CXO KPI Identification (PepsiCo) • 61
Support Big Judgment, Not Just Big Data • 77

Information Training Portfolio (Tiffany & Co.) • 87
Observe and Question to Identify Opportunities • 101

“Prosumer” Sounding Board (Ford) • 105
Anthropologically Driven Needs Identification (Alpha Company1) • 109
Appendix • 116
1 Pseudonym.
CIO0361911SYN
iv
Executive Summary
Our ability to store and analyze information has grown exponentially across Core information governance activities such as information architecture,
the past decade, but the buzz about “big data” misses the reality that as business stewardship of data, and an analytics center of excellence are
data volumes explode, it becomes harder, not easier, for knowledge workers important drivers of Insight IQ. Most organizations know this and already have
to make sense of it. Less than 40% of knowledge workers have the skills and initiatives in place. What is more surprising is the impact of three capabilities
judgment to exploit information for competitive advantage. Paradoxically, directly related to the knowledge worker: information usability, knowledge
as organizations amass more data and spend more on analytics, they face a worker skills, and the identification of emerging knowledge worker information
growing insight deficit. needs. These three capabilities are often overlooked, yet all directly impact
Insight IQ.
The insight deficit is important to IT, as information management will likely
be IT’s main contribution to enabling growth. In business areas that drive 1. Enable Information Usability
growth—innovation, marketing, sales, and customer service—up to 80% of
IT enablement opportunities relate to business intelligence, collaboration, or Only 50% of knowledge workers find information from corporate sources
the customer interface. Yet, the insight deficit means that efforts to improve to be in a usable format. The problem will only get worse as the number
information management may lead to worse, not better, decision making. of information sources, uses, and users continue to increase. In this diverse
environment, trying to improve usability by offering self-service usually
Insight IQ: A Unique View of How Organizations Exploit Information for disappoints as few self-service capabilities are flexible enough to meet
Decision Making knowledge worker needs. To overcome this challenge, the best companies
offer a choice of analytic tools and deploy information filtering and
To give IT leaders practical guidance on overcoming the insight deficit, we
visualization capabilities. They develop a deeper understanding of how, when,
built an Insight IQ index to measure individuals’ ability to find and analyze
and why information will be used by specific user segments, and they vary
information to make better decisions. Insight IQ has three elements:
quality standards accordingly.
information attainability, information usefulness, and knowledge worker
capability. Offer Greater Tools Choice—Greater choice in analytic tools drives Insight
IQ. The greatest increase comes from moving from one standard tool
Using the model, we surveyed almost 5,000 knowledge workers at hundreds of
for the enterprise to tools aligned to specific business units or functions.
organizations globally. We used the survey to quantify the value of overcoming
Rationalization programs for analytic tools often over-standardize, leaving
the insight deficit and pinpoint the actions needed to realize this value. (See
employees without necessary capabilities and opening the door to unofficial
page vii to learn how your organization can take the Insight IQ diagnostic).
tools. In response, progressive organizations, such as Blue Cross Blue Shield of
The benefits of overcoming the deficit are significant. Functions with the North Carolina offer knowledge workers a portfolio of tools aligned to a list of
highest Insight IQ perform, on average, 24% better than their peers across a analytic capabilities.
wide range of metrics. All three components of Insight IQ are vital to achieve
Visualize and Filter—Many knowledge workers complain about the time
this goal—if any are missing, the chance of failure is 65% or more.
required to aggregate information from supposedly user-ready reports.
Consequently, leading organizations invest in capabilities such as information
visualization, aggregation, and filtering before they invest in more sophisticated
analytics.
CIO Executive Board

IT Practice
www.cio.executiveboard.com
CIO0361911SYN
Executive Summary (Continued)
Make Information Quality Transparent and Targeted—Only one-third of Provide Tools for Collaborative Decision Making—Making biases explicit
knowledge workers trust information from other functions in the organization. and listening to alternative points of view are characteristics of informed
To overcome this distrust, progressive organizations such as GlaxoSmithKline skepticism. To support these behaviors, IT can provide workflow tools that
clarify the information’s source, how it has been manipulated, its current connect stakeholders involved in decision making, capture “wisdom of the
quality level, and caveats about how it can be used. Others, such as PepsiCo, crowds” input, and add contextual data. In addition, business leaders can help
recognize that certain groups, such as C-level executives, have distinct patterns promote informed skepticism by introducing formal decision-making models
of information usage that need to be targeted for different-in-kind support. and by embedding the right behaviors in performance expectations.
2. Support Big Judgment, Not Just Big Data 3. Observe and Question to Identify Opportunities
Exploiting information requires broader and deeper analytic skills across the Most IT organizations are familiar with process automation projects where
organization. Most employees are now knowledge workers and spend on business needs are known and stable. In contrast, information needs are
average 36% of their time collecting and analyzing information. However, only context dependent, dynamic, and may be unarticulated or even unknown.
38% of employees are “informed skeptics” who combine analytic skills with Solving this challenge requires anthropological skills that are in short supply
judgment to make sound decisions. These critical thinkers are comfortable with in IT.
analytics but maintain a healthy skepticism about its limitations. Identify and Learn from Early Adopters—Traditional requirements gathering
Develop an Analytics Training Curriculum—Two-thirds of knowledge workers fails when assessing information needs as the needs are fast changing,
have access to analytics training, but only 25% find the training effective. diverse, and difficult to articulate. Instead, progressive organizations, such as
Training that focuses on the tools but does not address information sources Ford, surface information needs by segmenting knowledge workers by their
or analytical techniques is ineffective. Leading organizations, such as Tiffany & information usage patterns and their stance toward technology adoption.
Co., adopt a multipronged approach that supports knowledge workers across They also observe early adopters as an early-warning mechanism for
the information lifecycle and focuses on conducting analysis. This type of emerging needs.
training has no natural organizational home, so IT and business leaders should Observe and Question—Knowledge workers often struggle to articulate their
collaborate to determine ownership. information needs, and these needs differ across individuals, organizational
Hire Analysts Who Can Coach—Knowledge workers should be supported by cultures, and collaborative practices. To proactively identify pain points in
business intelligence experts or analysts who act as force multipliers. While end-user workflows, leading IT groups adopt an anthropological approach to
coaching skills are what make these analysts most effective at driving Insight identify opportunity, observe knowledge workers in diverse user settings and
IQ, organizations typically look for quantitative and business skills when hiring conduct open-ended interviews.
analysts. Leading organizations have changed their recruiting approaches to
actively test for coaching skills.
CIO Executive Board

IT Practice
CIO0361911SYN
v
vi
An Information Management Strategy on a Page
Statement of Strategy
Shift the focus of IT investment and skills toward information management with the goal of providing knowledge workers with attainable and useful information
and boosting their capability to exploit that information for competitive advantage.
Top Initiatives Current Goal

Key Information Management Metrics
(2011) (2014)
1. Launch an information management and analytics center
of excellence. Percentage of IT budget devoted to 23% 40–50%
information and analytics projects
2. Redesign IT's opportunity identification process to make it
proactive and informed by observation of distinct knowledge Percentage of budget spent on < 5% > 10%
worker segments. knowledge worker capability
3. Identify analytic capabilities used by knowledge workers, Percentage of information subjects targeted > 80% < 40%
and offer a portfolio of tools to meet those needs. for harmonization and integration
Target number of analytic tools 1–3 8–12
4. Develop and hire usability and interface design skills in IT.
Percentage of knowledge workers 38% 59%
5. Coach knowledge workers to boost their analytic capability
who are informed skeptics
and foster informed skepticism.
Average Insight IQ score 51 65
6. Harmonize and integrate a small subset of information subjects
where there is greatest enterprise need.
Underlying Assumptions and Beliefs
1. The number of opportunities to drive growth through information management will equal or outstrip the opportunities for process automation.
2. Many of our knowledge workers lack the skills and judgment to use information effectively for decision making.
3. Not all information needs to be harmonized or integrated at enterprise level. Similarly, some information needs higher levels of quality than others.
4. Our business partners will take the lead in information stewardship.
5. Knowledge worker reliance on external information sources and on unstructured information will continue to rise.
Obtain a customizable template for a strategy on a page at www.cio.executiveboard.com.
CIO Executive Board

IT Practice
CIO0361911SYN
Use the Insight IQ
Diagnostic to benchmark
Assess Insight IQ in Your Organization
the maturity of your
information management Overview of the Insight IQ Diagnostic Diagnostic Benefits
and prioritize areas for
investment.
■■ The diagnostic measures Insight IQ using online 1. Assess Readiness—Identify which business
surveys for knowledge workers and an IT leader. functions and teams in your organization are
ready for additional investment in information
■■ Organizations identify at least 30 knowledge
management and analytics and be forewarned
workers per business function, and at least three
about areas where adoption may be low.
business functions participate in the diagnostic.
2. Set Priorities—Prioritize the technology
■■ Each survey takes approximately 20 to 25 minutes
investments, governance processes, and skills
to complete.
that have the greatest impact on Insight IQ.
■■ The diagnostic is included in your existing CIO
3. Communicate—Demonstrate the business
Executive Board membership.
leadership, communication, and change
management efforts necessary to close your
organization’s insight deficit.
4. Benchmark—Compare your organization’s
Insight IQ against industry peers.
Key Deliverables
Custom Diagnostic Report and Improvement Recommendations
Detailed results benchmarked Guidance on practical Executive Advisor® walk-

against 5,000 knowledge improvement steps, including through of results and key
workers worldwide tactics recommendations
For more information, contact your account manager or e-mail ITPracticeDiagnostics@executiveboard.com.

CIO Executive Board
IT Practice
CIO0361911SYN
vii
viii
CIO Executive Board

IT Practice
CIO0361911SYN
Occasion for the Research
CIO Executive Board

IT Practice
CIO0361911SYN
1
Overcoming the Insight Deficit 2
“Big data” is quickly

becoming a reality as
MORE INFORMATION, MORE INFORMATION WORK
information volumes
grow by 60% annually, Estimated Rise in Global Data Volumes, 2010–2015
and 36% of all work time Indexed to 100
is devoted to information
collection and analysis. 1,050
1050
■■ A 60% annual rise

R
in information volume means CAG 660
that average storage per end 60%
user will increase from 2 GB
410
to 20 GB in five years.
260
Knowledge worker time 160
■■
100
spent collecting and
analyzing information is
2010 2011 2012 2013 2014 2015
similar across all functions,
for example:
–– Finance, 40%
Time Spent Collecting and Analyzing Information Drivers of Democratized Decision Making
–– General Management,
Percentage of Total Knowledge Worker Work Time
35%
–– HR, 33% 1 Decisions are made closer to the market
(e.g., product design, channel mix).
–– Marketing, 36%
–– Production and 36%
Collecting 2 Decisions are more dynamic and varied
Operations, 33%
and (e.g., demand forecasts, discounts).
–– Sales, 27% Analyzing
64% Information
All Other Knowledge workers have access to more
■■ Knowledge workers dedicate Work 3
more time to collecting information and better tools (e.g., customer
and analyzing information segmentation and value analysis).
as decision making is
democratized.
n = 4,941 knowledge workers.
Source: “All Too Much” The Economist, 27 February 2010; Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board
IT Practice
Occasion for The Insight IQ Overview Roadmap for
Existing Research
CIO0361911SYN the research Diagnostic of findings the discussion
To drive growth, IT must
support less familiar
FROM PROCESS TO INFORMATION TO DRIVE GROWTH
areas of the business
where the majority of Breakdown of IT Enablement Opportunities by Business Process
opportunities are in Percentage of Total Opportunities
information management, None
9%
9%
not process automation. 15% Information Management
18%
27% (Collaboration, Business
28% Intelligence, Customer Interface)
■■ IT enablement opportunities Process Automation
are divided between:
–– Information 30%
Management—Includes
collaboration, business 28% 63%
intelligence, and the use
of technology at the 63%
customer interface, and 60%
–– Process Automation—
Processes enabled by
enterprise systems such
as ERP, PLM, HRIS, and 54%
46%
CRM.
28%
18%
13%
Finance Production and Customer Marketing Product/Service

and HR Supply Chain Service and Sales Innovation
“Information …but increased support

management means ■■ Extensive experience ■■ Little experience working with IT for growth moves IT into
we encounter functions working with IT ■■ Primarily drive growth much less familiar areas.
with little prior experience ■■ Primarily drive efficiency
working with IT. But to contribute and quality
to growth, we have to understand
these areas.” Continued focus on process automation, will
mean IT misses opportunities to drive growth…
CIO
Consumer Goods Company Note: Totals may not equal 100% due to rounding.
Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industry-
CIO Executive Board excel-version-51-0.
IT Practice
Existing Research
Occasion for the Research 3

As 62% of knowledge
workers lack the ability
Without Big Judgment, Big Data Magnifies Risk
to apply “big judgment”
to information they use
not Opportunity
for decision making,
increased information Have Processes and Skills to Use Information Effectively for Decision Making
Percentage of Knowledge Workers
availability may do more
harm than good.
Risks from Big Data When Big Judgment

Is Lacking
38%
Yes ■■ Exaggerated or overlooked opportunities
■■ Underestimated risks
■■ Disputed decisions
62%
■■ Misleading performance measures
No ■■ Wasted knowledge worker time
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
“The biggest talent

challenge we face is
finding people with the
skills to create insight from data.”
VP, HR
Global Food Company
CIO Executive Board

IT Practice
Existing Research
Effective information
management reframed
MEASURING THE INSIGHT DEFICIT
around the knowledge
worker drives improved The Three Components of Insight IQ
decision making.
■■ Insight IQ is a quantifiable
Insight IQ entails the ability to find
outcome that can be and analyze relevant information to
measured at the level of the drive actions and decisions effectively.
individual knowledge worker.
■■ Linking information
management initiatives with
higher Insight IQ will refocus
information management
on the important outcome.
The right information is Information Information Information is of a known
available and easy to find. Attainability Usefulness quality and in a usable format.
Insight IQ
Knowledge
Worker
Capability
Knowledge workers have the

analytic ability and predisposition
to generate insight from information.
Source: Corporate Executive Board, IT Practice Insight IQ Diagnostic, 2011.

CIO Executive Board
IT Practice
Existing Research

The CIO Executive Board

would like to extend its
With Sincere Appreciation
sincerest appreciation
to the following Partial List of Participating Organizations
organizations for their
participation in this
research.
CIO Executive Board

IT Practice
Existing Research
The CIO Executive
Board’s database spans
The Only Survey of Its Kind
a variety of organization
sizes and industries and Respondents by Industry Respondents by Function
includes responses from Sample Distribution Sample Distribution
almost 5,000 knowledge 2%
workers globally. Energy and Utilities
3% 2%
Telecommunication Legal
4%
■■ Survey administration 3%
Other
Corporate Strategy
involved in-depth 4%
Aerospace/Defense, 3%
collaboration with 22 HR
Automotive, and
member organizations. Transport 6% 18%
4% 27% Other Finance
■■ Additional surveying of Government High Tech 6%
knowledge workers at more 5% General 18%
Professional Management IT
than 1,000 organizations Services
7%
worldwide was conducted 5% 18% R&D and
to supplement the dataset. Manufacturing Insurance 15%
Engineering
9% Sales, Marketing,
10% and Retail
Chemicals, Pharma,
Customer Service
and Health Care
9% 12%
Banking and Production/
Financial Services Operations/
Procurement
10%
Retail
Respondents by Geography
Sample Distribution
1%
Other
2%
Latin America
14% 65%
APAC North America
18%
EMEA
CIO Executive Board
IT Practice Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
Existing Research

A Model of Insight IQ
Insight IQ entails the ability to find and

analyze relevant information, to drive
actions and decisions effectively.
1 Attainability—Information is easy to find. 2 Usefulness—Information is useful.

■■ The information that I need to do my job ■■ Information is useful and does not require
is available. modification or manipulation.
■■ I know which sources to go to for the ■■ I understand how information is defined.
information I need. Information Information
■■ There is a single version of the truth for
■■ I can find subject matter experts and Attainability Usefulness specific information.
other sources of tacit knowledge. ■■ Information is directly relevant to
business decisions and processes.
■■ Information is timely, up-to-date, precise,
Insight IQ
and accurate.
Knowledge
Worker
Capability
3 Knowledge Worker Capability—Knowledge

workers have the analytic ability and
predisposition to generate insight from
information.
■■ I rely on the results of analysis to support
business decisions, processes, and
deliverables.
■■ I am comfortable with advanced analytics.
CIO Executive Board

IT Practice
Existing Research
ASSESSING THE MOST POWERFUL DRIVERS OF Insight IQ
1 Deploy Surveys 2 Determine Drivers of Insight IQ 3 Translate Findings into Practical Guidance
■■ Distill the practical implications borne out by
Knowledge Worker Survey (n = 4,941.) Potential Drivers of Insight IQ
the data.
Knowledge workers from all major ■■ Provide real-world illustration of key insights
functions and business areas are 1. Knowledge Worker via practitioner tools and tactics.
surveyed on information usage. Attitudes and Incentives
2. Training and Support
3. Staff Knowledge and Competencies
4. Information Quality
5. Information Usage Patterns
IT Leader Survey (n = 22.) 6. Demographics
IT representative provides details on 7. Information Management Structures Illustrative Driver: Availability of Training
information management platforms, and Processes on Conducting Analysis
strategy, governance, and architecture. 8. Investment and Resource Level Average Insight IQ Score, Indexed
9. Information Strategy and Governance
10. Technical Environment ∆ = 29%
(Maximum 129
Impact)
100
Sample Questions
Which of the following best describes the
analytic tools available to you?
Information Information
Attainability Usefulness
No Tools Multiple Tools
Insight
IQ
How effective is the training provided by

your organization on conducting analysis?
Knowledge No Training on Yes, Training
Worker Conducting Available
Very Very Capability Analysis on Analysis
Ineffective Effective Available
CIO Executive Board

IT Practice
Existing Research

Some knowledge workers

have up to three times
Finding 1: Insight IQ Varies Substantially Across
higher Insight IQ than
their peers.
Knowledge Workers
Knowledge Worker Insight IQ1 Range
Database Percentile Values of Insight IQ
Some knowledge workers demonstrate

almost three times higher Insight IQ
than their peers.
77
65
51
38
28
10th Percentile 25th Percentile 50th Percentile 75th Percentile 90th Percentile
1 Insight IQ is measured as an index constructed from the three model components. Index scores ranges from 0 to 100, with higher
scores reflecting higher levels of Insight IQ.

CIO Executive Board
IT Practice
Existing Research
There is wide variation
in Insight IQ across
Variation Within and across
knowledge workers
within the same Insight IQ1 Range by Organization
organization.
90th Percentile
■■ Although not all knowledge Average
workers need to reach even
10th Percentile
the average level of Insight
IQ, small improvements can 100
have a significant cumulative
effect given the ubiquity of 90
knowledge workers within
most organizations.
80
■■ Variation within
70
organizations demonstrates
that company-level factors
Spread of Average Insight

IQ Across Organizations
60
do not explain all differences
in Insight IQ; there are
important factors at the 50
functional or knowledge
worker level as well. 40
30
20
10
0
Representative Sample of Participating Organizations
n = 22 organizations.
1 Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.

CIO Executive Board
IT Practice
Existing Research

Business functions
with a high Insight
Finding 2: OVERCOMING THE INSIGHT DEFICIT Has
IQ demonstrate 24%
better performance than
a Measurable Payoff
functions with a low
Insight IQ. Indexed Functional Performance by Level of Insight IQ
■■ Performance results were

measured at the functional Performance of
level to minimize the impact Functions with the 1.24x
Highest Insight IQ
of exogenous factors and
self-reporting bias.
Twenty-four percent higher
■■ A performance index of 12 functional performance
questions across the four
categories of self-reported
Performance of
data was created to Functions with the x
compare top- and bottom- Lowest Insight IQ
quartile functional teams.
Components of Functional Performance Index
Relative to similar functions at industry peers:

■■ Functional effectiveness
■■ Functional efficiency
■■ Function-specific performance (e.g., market
share growth, employee engagement)
■■ Impact of analytics on effectiveness,
efficiency, and performance

CIO Executive Board
IT Practice
Existing Research
The CIO Executive
Board controlled for
Finding 3: Three Demographic Factors Impact
demographic factors
that impact Insight IQ
Insight IQ
during all analysis.
Potential Demographic Drivers
What Matters
■■ Seniority The CIO Executive Board applied
statistical controls to void the
■■ Openness to new technologies
impact of demographic factors
■■ Importance of analytics to job role that drive Insight IQ.
What Doesn’t Matter

■■ Age
■■ Highest degree earned
Knowledge Worker–Level Factors
■■ College major
■■ Organizational tenure
■■ Company revenue
■■ Level of regulatory burden
■■ Information intensity
Organization-Level Factors
■■ Industry
■■ IT organizational structure
■■ Geographic span of business

CIO Executive Board
IT Practice
Existing Research

Finding 4: All Components of Insight IQ Matter
Poor performance in any of the three components of Insight IQ significantly diminishes the probability of strong business performance.
Type 1: Knowledge Workers Cannot Type 2: Knowledge Workers Type 3: Knowledge Workers Cannot Type 4: High Insight IQ
Easily Access Information Are Wary of Analytics Translate Information into Action
Information Information Information Information Information Information Information Information

Attainability Usefulness Attainability Usefulness Attainability Usefulness Attainability Usefulness
Insight Insight Insight Insight
IQ IQ IQ IQ
Knowledge Knowledge Knowledge Knowledge

Worker Worker Worker Worker
Capability Capability Capability Capability
Probability of strong business Probability of strong business Probability of strong business Probability of strong business
performance1 if information performance if knowledge worker performance if information performance: 80%
attainability is low: 18% capabilities are low: 27% usefulness is low: 35%
1 Values are calculated as the probability of knowledge workers demonstrating top quartile performance in indexed business performance via logistic regressions.
CIO Executive Board

IT Practice
Existing Research
The likelihood of realizing
full business value
NO SHORTCUTS TO DEFICIT REDUCTION
from an information
management initiative Likelihood of Realizing Full Value from Information Management and Analytics
is only 5% to 35% if Percentage of Likelihood Under a Typical Deployment Pattern
the organization does
80%
not invest in all three
components.
■■ Yet, many organizations

User Engagement Staff
focus on system investments
Typically unfunded openness
and governance at the to analytics
expense of knowledge boosts the
worker engagement. likelihood of
realizing full
value from
■■ Each component considered 27% to 80%.
individually is necessary, but
Data Governance
not sufficient, to achieving
Total Annual FTE costs1:
full business value.
$3 M
27%
Without
effective
System Investments information
governance,
Annual project spend1:
the likelihood
$18 M of realizing
5% full value is
only 5%.
0%
A data desert—no Technology without Enough information Full value

information is available. governance— to be dangerous— realization—
information is information is useful there is complete
attainable but quality but staff don’t know absorption of
and relevance are low. how to analyze it. analytics for
decision making.
1 Estimates for a company with revenue of $8 B.

CIO Executive Board
IT Practice
Existing Research

We analyzed the dataset

to isolate the most
FINDING 5: TOP DRIVERS OF INSIGHT IQ
important drivers of
Insight IQ and translate
the findings into practical
guidance.
Potential Drivers of Insight IQ Tested

1. Knowledge Worker
Attitudes and Incentives Information
Information
2. Training and Support Attainability Usefulness
3. Staff Knowledge and Competencies
Insight
4. Information Quality
IQ
5. Information Usage Patterns
6. Demographics
7. Information Management Structures and
Processes Knowledge
8. Investment and Resource Level Worker
9. Information Strategy and Governance Capability
10. Technical Environment
CIO Executive Board

IT Practice
Existing Research
Partial List of tested drivers Shading Denotes Top 15% Most Impactful Drivers
I. Information Management Structures and Processes III. Knowledge Worker Attitudes and Incentives V. Quality of Information (Continued)
1. Analytic Capabilities and Practices Shared 34. User Confidence in Information Quality (Cleanliness and 60. New Technologies Prescreened for Enterprise Data Model
2. Analytics Teams Centralized Companywide Accuracy) Alignment
3. Central Analytics Group in IT, Reporting Directly to CIO 35. What Level of Confidence Do You Have in the Quality 61. Quality Standards Vary by Function
(Cleanliness and Accuracy) of Information from Each of the 62. Quality Standards Vary by Information Type
4. Central Analytics Group in IT, Within Information
Following Functions? 63. Quality Standards Vary by Seniority
Management
36. Knowledge Workers Are Rewarded and Promoted Based on 64. Satisfaction Level for Each Type of Information
5. Delivery Channels for Providing Information/Analytics Outcomes of Their Analysis and Decisions. 65. Timeliness Standards Vary by Information Type
6. Degree of Integration of Management of Analytics and 37. Knowledge Workers’ Perceived Ease of Use of Information 66. Use of Common Definitions and Vocabulary in Information
Information Contained in Central Repository
Presentations
7. Degree of Knowledge Worker Involvement with Other 38. Knowledge Worker Preference for Push Versus Pull for Types
Functions/Groups of Information
8. Existence of Formal Analysis Processes/Frameworks 39. Knowledge Worker Trust in Information Contained in Central VI. Information Usage Patterns
Repository
9. Existence of Master Data Management Tool
40. Knowledge Worker Trust in Information from Other Groups/ 67. Capabilities in Place to Obtain Cross-System/Functional
10. Existence of Metadata Structure (Standards and Models) Functions (and Reason) Information
11. Integration of External Sources of Information into the Data 41. Level of Blame for Relying on Inaccurate Prediction 68. Degree of Integration of Cross-Functional Information
Warehouse 42. Perception That Being Analytic Is Advantageous to Career 69. Frequency of Using Each Type of Information/Analysis
12. Information/Analytics Requirements Incorporated into Other Development. 70. IT Support for Different Types and Sources of Information
Projects Early 43. Perception That Analytics Is Hard Due to Lack of Time. 71. Nature of Information/Analytics Use (e.g., Customer Analysis,
13. IT Organization Structure (e.g., Centralized or Decentralized) 44. Perceived Importance of Information Aggregation BPO)
Capabilities 72. Need of Information from Different Functions
14. Knowledge Workers Have Enough Time to Conduct Analysis
45. Perceived Importance of Information Filtering Capabilities 73. Number of Sources Required for Each Type of Analysis
15. Master Data Management Process Includes Error Detection 46. Perceived Importance of Information Visualization
and Correction 74. Purposes of Information/Analytics Use (e.g., Decisions,
Capabilities Reporting, Pattern Spotting)
16. Mechanisms of Delivery for Each Type of Information— 47. Visibility of Internal Opportunities That Require Analytic
Pushed or Pulled 75. Relative Use of Different Types of Info. (e.g., Structured/
Skills
Unstructured, Internal/External)
17. Prevalence of Analytics in Group Versus Individual Decisions
76. Satisfaction Level for Mobile Access to Key Information
18. Process for Sharing Information
IV. Investment and Resource Level Types
19. Responsibilities for IM Group (Including Who Owns What) 77. Time Spent on Information Visualization, Filtering,
20. Responsibilities of Central Analytics Group 48. Approach Taken to Getting Approval for Investments in IM/ Aggregation, and Analysis
21. Use of Aggregation, Visualization, and Analysis Analytics 78. Types of Analyses Being Done (e.g., Predictive, Prescriptive)
22. Use of Agile Software Development 49. Cost of/Investment in Analytic Capabilities (by Function) 79. Types of Information Being Used
50. Level of IT Investment in Capabilities to Support Group 80. User Feedback on Information Sources and Analytics Tools
23. Use of External Providers for Analytics
Decision Making Collected
51. Number of IT Staff Responsible for Information Management 81. Users Information Needs Assessed via Surveys
II. Training and Support 52. Technology Costs for Managing Information 82. Users Information Needs Assessed by Tracking Information
Requests
24. Analytic Mind-Set of Respondent’s Manager V. Quality of Information
25. BU/Functional Performance Metrics Reported Regularly
26. Formality of Decision-Making Process 53. Existence of Automation of Information Quality Governance
27. Leadership Attitudes/Behaviors Toward Fact-Based Decision and Standards
Making 54. Existence of Process for Updating and Retiring Information
28. Overall Frequency of Using Analysis (Info. Lifecycle Management)
29. Use of KPIs/Metrics to Track Organizational Performance 55. Existence of Standard Definitions for Information
30. Performance Reviews Based on Measurable Goals 56. Data Transformations Are Transparent to Users.
31. Performance Reviews Evaluate Analytic Ability 57. Frequency of Reviewing/Updating Analytic Models
32. Time Spent on Analysis and Assumptions
33. Existence of Processes/Capabilities to Overcome Biases 58. Functional Information with Standard Definitions
59. Information Quality Regularly Reviewed/Audited
CIO Executive Board

IT Practice
Existing Research

Partial List of tested drivers (Continued) Shading Denotes Top 15% Most Impactful Drivers
VII. Staff Knowledge and Competencies VIII. Information Strategy and Governance IX. Technical Environment
83. Analyst Skills Valued 1 07. Approach Taken on Analytic Tool Selection and Distribution 134. Analytic Tools Bought Off the Shelf or Developed In House
84. Analytics Staff Hired and Assessed for Training and Coaching 108. Approach Taken on Analytic Tool Selection and Distribution 135. Analytics Solutions Deployed Using a Test and Learn
Abilities 109. Clear Goals or Strategies for Social Media in Place Approach
85. Assessment of Analyst Speed, Quality, and Responsiveness 110. Components of Analytic Strategic Plan 136. Automated Decisions Implemented
86. Availability of Support in Finding or Analyzing information 111. Coverage of Analytics Strategic Plan 137. Customizability of Analytic Tools
87. Availability of Training on Finding or Analyzing Information 112. Decision Stakeholders 138. Enterprise Search Implemented
88. Consumption of Training on Finding or Analyzing 113. Degree of Standardization of Key Business Processes 139. Existence of and Types of Collaboration Tools/Capabilities
Information 114. Degree of Sharing Information with Customers 140. Existence of Automated Information Capturing Capabilities
89. Existence of Training on Use of IM and Analytic Systems 115. Degree of Sharing Information with Suppliers 141. Existence of Central Repository
90. Effectiveness of Training on Finding or Analyzing Information 116. Documentation of Business Processes 142. Existence of a Data Warehouse
91. High-Quality Training on Finding, Understanding, and 117. Existence of Analytics Strategic Plan 143. Existence of In-Memory Technology Capabilities
Analyzing Info. 118. Existence of Data Steward Role 144. Existence of Non IT-Supported Information Repositories
92. Inclusion of Analytics in Job Descriptions 119. Existence of Dedicated Information Management Team 145. Degree of Integration of Operational Systems
93. Knowledge Worker Analytic Skills and Level 120. Existence of Strict Security Policies Around Information 146. Information Sources (e.g., Smart Networks, Sensors)
94. Knowledge Worker Awareness of Security Policy Related 121. Information Quality Standards and Policies Are Defined and 147. Knowledge Workers Have Access to Analytic Tools Other
to Information Sharing Enforced Than Excel
95. Knowledge Worker Degree of Proficiency in Using Analytic 122. Information Quality Standards and Policies Enforced by Info. 148. Knowledge Worker Can Choose from Multiple Analytic Tools
Tools Management Group Available
96. Knowledge Worker IT Skills 123. Information Quality Standards and Policies Enforced by 149. Legacy Systems
97. Knowledge Worker Knowledge of What Information Exists Shared Services Group 150. Number of Analytic Tools Being Used/Provided
in the Central Repository 124. Information Security and Access Policies Do Not Inhibit 151. Number of Features of Analytic Tools
98. Knowledge Worker Perception of Ease of Use of Sharing Attainability of Information 152. Number of Information Repositories
Analysis/Info. Using Analytic Tools 125. Inclusion of Analytics in Broader Corporate/IT/Other 153. Number of Operational Systems (by Function)
99. Knowledge Worker Relies on Judgment When Disagreeing Function Strategic Plan 154. Tool Interface Design, Testing, and Enhancements
With the Results of Analysis 1 26. Information Architecture in Place or Planned 155. Use of Cloud Applications to Generate, Store, or Analyze
100. Knowledge Worker Understanding of Analysis Behind 127. Keeping Track of New Information Systems and Analytic Information
Strategic Decisions Capabilities
101. Knowledge Worker Understanding of How Information is 128. Organization Characteristic (e.g., Consensus-Based or
Useful in Other Functions/Groups Hierarchical)
X. Demographics
102. Knowledge Worker Understanding of Responsibilities of 1 29. Organization Strategic Business Goals/Objectives
Other Functions/Groups (i.e., What They Do) 130. Overall Ownership of IM Group 156. Function Responsible for Developing Analytics Strategic
103. Presence of Analytics Coach (Formal or Informal) 131. Perceived Clarity of Roles and Responsibilities for IM Group Plan
104. Processes for Understanding Tool Usage and Workflows 132. Policies for Adding Information to a Data Warehouse 157. Knowledge Worker Career Background (Experience,
105. Skills and Knowledge of Staff Responsible for Managing 133. Robustness of Enterprise Data Model Number of Functions Worked In)
Information 158. Knowledge Worker Function
106. Use of Hypotheses to Drive Analysis 159. Knowledge Worker Seniority
160. Organization Industry
CIO Executive Board

IT Practice
Existing Research
1
1. KWs Having Enough Time to
Conduct Analysis
30.3%
IT Practice
n = 4,941.
CIO0361911SYN
9. Having an Information
Architecture
CIO Executive Board
11. Having a Mature Data Model
KW = Knowledge worker.
17.2% 16.4%
13. Having Central Analytics
Group or COE
The Usual Suspects
15. Defining and Enforcing

Quality Standards
16.2% 15.5%
2. KWs Having Confidence in
Information Quality
25.2%

14. Providing Visibility into Data
Transformations
16.0%
17. Providing Information

Visualization and Aggregation
Tools
the research
Occasion for
1. Enable
19. Enabling Enterprise Search

14.9% 14.7%
Information Usability
The Top Drivers of Insight IQ1
20. KWs Choosing from Multiple

Analytic Tools
13.7%
Diagnostic
4. Training KWs on How to Find
The Insight IQ
and Conduct Analysis
5. Formalizing Decision-Making
Process
24.7% 24.6%

6. KWs Are “Informed Skeptics”
20.0%
7. Providing KWs with Access to

Analytic SMEs
Overview
18.6%
of findings
8. Regularly Reporting Business/
Functional Performance Metrics
17.4%
2. Support Big
11. Basing Performance Reviews

on Measurable Goals
16.4%
Judgment, Not Just Big Data
16. Considering KW’s Analytics

Capability When Evaluating
Performance
the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
18. Hiring Analytic Staff for Their

Roadmap for
Training and Coaching Abilities

the discussion
15.0% 14.8%
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and
3. Collecting KW Feedback
on Information Sources and
24.8%
Analytics Tools
10. Surveying KWs to Determine

to Identify
3. Observe
Information Needs
16.8%
and Question
Opportunities
Occasion for the Research

Existing Research
19
To help drive insight

IQ, IT organizations will
FINDING 6: INSIGHT IQ HAS IMPLICATIONS FOR IT ROLES
require new roles or skill
sets in four areas.
AND SKILLS
IT Roles and Skills Required in a High Insight IQ Organization
1. Enable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Information Architects Information Insight Enablers Corporate Anthropologist

Develops and maintains Supports knowledge workers Uses observational and
information architecture that with insights, business experience mapping
enables knowledge worker intelligence, and management techniques to identify unmet
productivity and decision reports for effective decision needs for information and
making making analytic capabilities and other
technology functionality
User Experience Specialists

INFORMATION TECHNOLOGY PRACTICE
CIO EXECUTIVE BOARD ™
Designs and configures
user-centric interfaces and
information visualization
The IT Talent Implications
of the Future of Corporate IT techniques based on a
A Guide to Filling Emerging IT Skills Gaps deep understanding of
end-user requirements and
behaviors
For full details about our analysis

of emerging IT skills and roles,
see The Talent Implications of the
Future of Corporate IT study.
CIO Executive Board

IT Practice
Existing Research
Business leaders outside
IT have a vital role to play
Finding 7: Information Ownership is a Critical
in creating a high Insight
IQ organization.
Business Role
Guidance to Business Leaders for Driving High Insight IQ
■■ To highlight the criticality of
business leader involvement
in information management,
1. Take Ownership of Your Information
Corporate Executive
Business leaders must determine what data is necessary, what
Board sent a summary of
analyses they want to run, how data is shared across organizational
Overcoming the Insight
boundaries, and how they derive insight from data.
Deficit to all members in all
functions globally. 2. Put Information at the Heart of the Business Plan
EXECUTIVE GUIDANCE FOR 2011
Tightly integrating information management plans with business
Overcoming the Insight Deficit strategic plans drives higher Insight IQ. This means deconstructing
Big Judgment in an Era vague strategy into more stable business capabilities.
of Big Data 3. Create Incentives for Cross-Company Data Standardization Efforts
Executives should create a strong governance process that cuts
across the company, including a corporate “tax” to overcome the
first-mover disadvantage.
4. Improve the Quality of Your Information
Business leaders should own the maintenance of authoritative
data sources, approve new and modified definitions, establish
confidentiality levels, and define data-quality processes.
5. Centralize Management, Not Information
Foundational analytic investments, such as a dedicated analytics
team, training design, and information quality norms, benefit
from centralization, create the oversight to sustain decentralized
information sources.
To access copies of this report for business executives, visit

http://www.executiveboard.com/executive-guidance/2011/Q3/index.html.
CIO Executive Board

IT Practice
Existing Research

Overcoming the Insight Deficit
1. E
nable 2. S
bserve and Question
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Capability-Driven Tool Portfolio Information Training Portfolio “Prosumer” Sounding Board

1
Flexible and Transparent Analytics Anthropologically Driven

Needs Identification
CXO KPI Identification
Emerging Information User Information Insight Enablers Corporate Anthropologists

IT Roles Architects Experience

Specialists
Appendix: Ownership Models for Analytics Centers of Excellence

1 Pseudonym.
CIO Executive Board

IT Practice
Existing Research
Information management
is not a new topic; most
Building on a Solid Foundation
organizations have
foundational capabilities Existing Capabilities and Relevant Resources
in place.
Architecture Roadmapping
■■ While necessary, these
Target Architecture Roadmaps Business
Architecture
capabilities are not sufficient (Wells Fargo) (KBC Bank)
to boost an organization’s
Insight IQ or to move IT’s
capabilities from process Information Strategy and Stewardship
automation to information
management. Task-Specialized Data
Stewardship (RBC Financial)
Information Quality
Data Quality Audit Team Information Health Scorecard Process Breakpoint

(Hewlett-Packard) (International Monetary Fund) Identification (Intel)
Report Generation
Business Metrics Selection Tool

(Corporate Executive Board)
Systems Integration and Rationalization
End-to-End Simplification Applications Portfolio

Business Case (Charles Schwab) Rationalization (CNA Financial)
To access the Board’s existing User Support

resources, Visit the Information
Segmented Information User
Management topic center at Support (Hallmark)
www.cio.executiveboard.com.
CIO Executive Board

IT Practice
Existing Research

CIO Executive Board

IT Practice
CIO0361911SYN
1. E
nable 2. S
bserve and Question
Case Examples

1



Specialists

1 Pseudonym.
CIO Executive Board

IT Practice
CIO0361911SYN
Enable Information Usability 25

One-half of all knowledge

workers find that
THE USABILITY DEFICIT
information from
corporate sources is Information from Corporate Sources Is in a Usable Format
not in a usable format. Percentage of Knowledge Workers
■■ Improve information
usability by letting
knowledge workers choose
the tools they find easiest to
use, prioritizing information
visualization and offering
greater transparency into
50%
quality levels.
No
50%
Yes

CIO Executive Board

IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
Knowledge workers
rely on a wide range of
INFORMATION OMNIVORES
information sources, both
internal and external. Mean Number of Corporate Information Sources Used by Knowledge Workers at Least Weekly
■■ The proliferation of
information sources creates
challenges for information Corporate Operational Systems 4.7
aggregation, presentation,
and quality.
Corporate Unstructured 4.2
Use External Sources of Information in Day-to-Day Work

24%
No
76%
Yes

CIO Executive Board
IT Practice
CXO KPI
Identification

The proliferation of
information users, uses,
INFORMATION EVERY WHICH WAY
and sources drives new
usability problems. Usability Challenges
More Information Users More Information Uses More Information Sources
Executives Customer value analysis Internal

Line managers Risk management
Analysts Mobile presence
Frontline staff Executive dashboards
Customers and suppliers Automated decisions External
1 Inadequate Tools—No single 2 Inflexible Reports and Cubes— 4 Information Quality Issues—
analytic tool can support all Predefined reports cannot Quality varies significantly by
needs and skills levels. answer all knowledge worker information source in ways that
questions while cubes are slow are not always transparent to
and offer limited dimensions. the user.
3 Centralized Ad Hoc Reporting

Inefficiencies—Relying on a
central group to run custom
queries creates lags in analysis
time and can be an inefficient
use of resources.
CIO Executive Board

IT Practice
CXO KPI
Identification
Usability can be
improved by providing
USABILITY = Bounded FLEXIBILITY AND
knowledge workers
with greater flexibility in
TRANSPARENCY
tools choice, information
access, and use.
Capability-Driven Goal: High Uptake of

■■ To avoid a free for all, the Tool Portfolio Self-Service Analysis
choice should be shaped High
by a deep understanding
of the business questions
knowledge workers need Flexible and
Value Associated with the Approach

to answer. Transparent Analytics
CXO KPI
Identification
Data Cubes
Ad Hoc EDW Queries

by Specialist Data Free
for All
Dashboards
Standard
Reports
Low
One Size Choice Shaped
Fits All by Business Need
Approach to Usability
1 Pseudonym.
CIO Executive Board
IT Practice
CXO KPI
Identification

If knowledge workers
don’t find the analytic
Users have choice (official or not)
tools provided by IT
useful, they have an array Use Analytic Tools That Are Not Supported Reasons for Using Analytic Tools That Are
of other choices. by IT Not Supported by IT
Percentage of Knowledge Workers Percentage of Knowledge Workers
■■ The most common reason

for using an unofficial tool is
lack of an official alternative,
the second most common IT Does Not Provide 51%
reason is ease of use. an Alternative Tool
–– This is a 1.5 times more

frequent reason than the
need for a more powerful 32%
tool. Yes
The Unofficial Tool

45%
Is Easier to Use
68%
No
The Unofficial
Tool Is More 29%
Sophisticated

CIO Executive Board

IT Practice
CXO KPI
Identification
Greater choice in
selecting analytic tools
Choice has Value
to use is a powerful
driver of Insight IQ. Maximum Impact on Insight IQ1
■■ The greatest increase

comes from moving from
∆ = 6.0% 13.7%
one standard tool for the
enterprise to standard tools
aligned to a specific business Knowledge ∆ = 16.0% 7.7%
unit or function. Only Analytic Workers Must Use a
Tool Available Single Tool That Is
to Knowledge Standardized Across
Workers Is Excel the Organization
Knowledge Workers Must Knowledge Workers Can

Use a Single Tool That Has Choose from Multiple
Been Selected for Their Analytic Tools Available
Business Unit or Function
(8.3%)
∆ = 13.1%
(21.4%)
1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively
“high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of
multivariate regressions with controls.
CIO Executive Board

IT Practice
CXO KPI
Identification

Greater tool choice

doesn’t mean chaos
Not too Few, Not Too Many
but rather providing a
portfolio of tools that Three Approaches to Standards for Analytic Tools
meets the full range of
capabilities needed by
different knowledge “Capability-Driven” Increases knowledge worker
worker communities. productivity
Align the Tool
Portfolio by Capability Reduces redundancy while
acknowledging diverse needs,
■■ Rationalization programs preferences, and skills
often try to over-standardize
Provides principled criteria
the analytic toolset, leaving
for future tool selection
knowledge workers without
needed capabilities and
opening the door to ROI of the
unofficial tool adoption. Tool Portfolio Number of Tools in the Portfolio
■■ Progressive organizations
rightsize their portfolio
of analytic tools by
identifying and aligning
with capabilities.
■■ IT leaders should avoid

applying the same criteria “Dictatorship” “Anarchy”
to analytic tools that they Rationalize Ungoverned
use for other applications to One Tool Tools
rationalization efforts, as
the optimal number of No single tool provides all capabilities. Proliferation of tools leads to uncontrolled
analytic tools is likely to licensing costs.
Rationalization expense erodes savings.
be higher than for other
Different tools are used by different teams
categories of application. Having only one tool may encourage use
for the same purpose.
of unofficial tools.
ROI dips as tools have limited usage
or unexploited functionality.
CIO Executive Board

IT Practice
CXO KPI
Identification
Capability-Driven Tool Portfolio
Overview
A methodology for selecting a set of analytic tools that collectively provide 10 key capabilities to knowledge workers
Executive Teaching
Focus on helping knowledge workers use the right tool for the right purpose rather than aiming to minimize the total
number of tools. Identify the analytic capabilities most needed by knowledge workers, and use the capabilities
to assess current and future analytic tools to identify gaps and redundancy in the tool portfolio.
Company Snapshot
Blue Cross Blue Shield of North Carolina

Industry: Health Insurance BlueCross and BlueShield of North Carolina (BCBSNC) provides
2010 Employees: 4,600 health care insurance products and related services to about 3.7
million members in North Carolina. BCBSNC is a licensee of the Blue
Cross and Blue Shield Association and was founded in 1933. The
company’s health plans include Blue Care (HMO) and Blue Options
(PPO), as well as consumer-directed plans. BCBSNC also provides
dental, life, disability, long-term care, and Medicare supplemental
insurance, as well as prescription drug coverage.
Source: http://www.hoovers.com.
CIO Executive Board

IT Practice
CXO KPI
Identification

In an internal survey
of tools used for
A Plentiful but Ill-Fitting Portfolio
analytic functions,
BCBS NC learned that Blue Cross Blue Shield of North Carolina Analytic Tool Environment
its portfolio of analytic 2009
tools is misaligned and One Tool Fits All
duplicative and used for Teams use one tool (often Excel or Access) for all needs, missing the chance to use tools
suboptimal purposes. that better meet their needs.
Everyone Gets Their Own Tool

Multiple tools are used for the same purpose by different teams, resulting in wasteful
duplication.
A Screwdriver to Hammer in Nails

Tools are used for purposes for which they are not optimized.
Tools with Unfulfilled Potential

Tools have with limited usage or unexploited functionality.
“We found that many

teams were using the
one tool they knew for
as many functions as possible,
while better tools went unused.”
Todd Norris
Director of IM Delivery
BCBS NC
CIO Executive Board

IT Practice
CXO KPI
Identification
Define analytic
capabilities needed
Ten Analytic Capabilities
by knowledge workers
across tools, not
Capability Description
functionality within tools.
1. Data Collection/ Gathering, integrating, consolidating, modifying, and validating data
Conditioning/Validation
■■ BCBS NC defines 10
capabilities supported by 2. Static/Printable Reports are specifically designed to be consumed as a printout. These reports have no
analytic tools. Reporting “interactivity” and are typically scheduled to run on a regular basis or have a few short
prompts for the user to enter.
■■ The company surveys 3. Self-Service Reporting Reports/queries are given to users to run on demand. They can be interactive with
knowledge workers to prompts and drill downs.
determine which tools meet 4. Ad Hoc Report Reports are developed as part of exploration and are typically for a one time use. These
which capabilities. Development may turn into self-service or printable reports.
■■ The capabilities are reviewed 5. Advanced Analytics/ Analysis typically done by a small number of analysts who look for new business
annually and adjusted Data Mining insights by examining relationships of information and information trends, this includes
according to evolving data and text mining.
knowledge worker needs. 6. Data Visualization Not limited to charts, this includes any functionality providing faster consumption
of information through “pictures” or presentations other than data grids.
■■ See page 37 for an overview
7. Dashboards/Scorecards Dashboards—On-screen visualizations that provide a quick status of key performance
of the survey process and
indicators specific to a role
page 38 to 39 for a list of the
survey questions. Scorecards—Use colors, indicators, and numbers to compare actual performance of key
metrics against a target or budget. These are slightly more detailed than dashboards
and less visual.
8. Geospatial Analysis This is an array information across a geography to show relationships between data
and locations.
9. Operational Reporting This is not used to analyze performance, but used for operational reporting. Examples
include sending an invoice to a customer or a letter to a member.
10. Publishing and Delivery Report distribution including e-mail, print and mail, or using a business intelligence
(BI) tool to distribute through a built-in or custom portal for providers, members,
or internal staff.
Nonspatial analysis and mobile BI will be added as

emerging capabilities during the next refresh.
CIO Executive Board
IT Practice
CXO KPI
Identification

Maintain a portfolio
of tools that supports
A Portfolio of ENTERPRISE and
all 10 capabilities. Specialty Tools
■■ Tools are designated in one
of two categories:
–– Enterprise Tools support
Dashboards/Scorecards
Conditioning/Validation
Publishing and Delivery

Self-Service Reporting
Operational Reporting
multiple capabilities used
Advanced Analytics/
Geospatial Analysis
for the same purposes
Data Visualization
Data Collection/
Static/Printable
across the company;
Ad Hoc Report
Development
between three and five
Data Mining
Reporting
of these tools should
support most capabilities.
–– Specialty Tools support
one capability in one
particular function;
Enterprise Tools (3 to 5)
another six to nine
of these tools may
BObj Web Intelligence
be needed.
SAS Core
■■ BCBS NC identifies
capability gaps and helps Crystal Reports
teams use the tools that best
provide the capabilities they
need.
Speciality Tools (6 to 9)
“While we will always
retire tools where the Amisys MACESS
ROI isn’t there, we are
Teradata SQL Assistant
more concerned about helping
people use the right tool for the
MCSource
right purpose and get away from
using one tool for everything.” EDIWatch
Todd Norris
Director, IM Delivery
BCBS NC
Capability gaps
CIO Executive Board
IT Practice
CXO KPI
Identification
The process of capturing
analytic capabilities by
IMPLEMENTATION GUIDE: CAPABILITY
surveying knowledge
workers requires
SURVEY ROADMAP
executive sponsorship
and clear communication. Keys to Successfully Surveying Knowledge Workers on Their Capability Requirements
Only key analytic areas

1. Obtain sponsorship and direction from VP–level information
identified by this group
governance group. are surveyed.
Representatives are
2. Identify user representatives in each division to administer told that each survey
the survey. should take less than
an hour per analytic
area to complete.
3. Hold an orientation session to make sure knowledge workers

understand the intent of each question.
“The process of
surveying was just as
important as the
questions we asked. The survey
was not intended to capture 4. Follow up surveys with an interview with each group to validate
perfect information but to give results and ask questions.
informed guidance for next steps.”
Susan Helm-Murtagh
Three Weeks
VP, Information Management
for Entire
and Analytic Services
Process
BCBS NC
CIO Executive Board

IT Practice
CXO KPI
Identification

IMPLEMENTATION GUIDE: Capabilities Survey

I. Tools and Technology
1. Indicate the tools used in your area, by capability. 2. N

umber of people proficient with each tool and proficiency level
(1 = general understanding of the tool; 5 = expert level)
BI Capabilities Tool 1 2 3 4 5
Tool 1 2 1
Advanced Analytics/
Geospatial Analysis
Tool 2
Ad hoc Reporting
Data Collection/
Static/Printable
Tool 3
Publishing and
Conditioning
Dashboards/
Visualization
Operational
Self Service
Scorecards
(mapping)
Reporting
Reporting
Delivery
Other 2
Other 1
Mining
Feedback on BI
Technical Skills:
Tool
Tool A X X X X X
Tool B 3. C
omment on tools that are particularly critical to your area or feel
Tool C X X X X X free to mention tools you feel, if improved, could save you time or
Other 1 X X provide greater information insights.
Other 1 X X X X
Any other feedback on the
tools used in your area 4. C
omment on activities you do in your area that you feel should
Any other feedback on the not be your responsibility or you feel your team is not adequately
tools used in your area trained to support.
II. Effort
5. Number of people who do analytical or BI development work. 6. Percentage of time spent by high-level activity
Number of full-time equivalent employees in your area Collecting and assembling data for analysis 20%
9
Conditioning, filtering, or validating data for analysis 20%
How many FTE resources would you need to fully
9 Developing new reports, report solutions, or analysis 20%
satisfy the requests of your team?
CIO Executive Board

IT Practice
CXO KPI
Identification
IMPLEMENTATION GUIDE: Capabilities Survey
7. O
f the time your area spends collecting and assembling data, list 8. O
f the time spent conditioning and filtering information, how do you
the percentage spent on each activity. spend that time?
Activity % Activity %
Pulling information from various sources 50% Developing/applying filtering and rules unique to your needs 90%
Manipulating information in spreadsheets/MS Access/SAS 50% Developing/applying filtering rules than are/could be used by others 10%
Other 1
Total 100% Total 100%
9. L
ist the main sources of information you use or plan to use in the near 10. How does your report solution development time breakout?
future.
Source Type of Information Activity %

Source 1 Claims, membership, provider info Time spent developing a specific report for a specific question 20%
Source 2 Call center information Time spent creating a solution to answer many questions 80%
Source 3 Call center forecasting, staffing models Total 100%
III. Applications and Solutions
11. In the grid to the left enter the following information:

Support Hours
Supported (X)
Critical (1–5)
■■ Solution Name: type in a solution name.
■■ Critical: 1 = nice to have; 5 = mission critical in your area

■■ Supported: put an X if the solution is supported by your area
Solution Name ■■ Support Hours: monthly hours spent supporting the solution
Solution 1 5 X 20
Solution 2 4 X 2
Solution 3 5
CIO Executive Board

IT Practice
CXO KPI
Identification

CIO Executive Board

IT Practice
CIO0361911SYN
Usability can be
knowledge workers
greater flexibility in
TRANSPARENCY
access, and use.

knowledge workers need Flexible and

to answer. Transparent Analytics
CXO KPI
Identification
Data Cubes
Ad Hoc EDW Queries

for All
Dashboards
Standard
Reports
Low
CIO Executive Board

IT Practice
CXO KPI
Identification

Less than one-half of all

knowledge workers find
REPORTING THE END OF THE REPORT
standard reports and
dashboards useful. Reports and Dashboards That Are Pushed1 Are Valuable
■■ Standard reports rarely

contain the right information
depicted in the right way
to be to useful knowledge
workers as they make
decisions.
49% 51%
Yes No
1 Information supplied in standard dashboards and reports at a predefined frequency,

often distributed electronically through e-mail.

CIO Executive Board
IT Practice
CXO KPI
Identification
The proliferation of
information sources
Putting it All Together
increases the value of
information visualization, Maximum Impact on Insight IQ1 of Information Presentation and Analysis Capabilities
aggregation, and
filtering.
■■ These capabilities can be 14.9%

defined as follows:
13.6%
–– Visualization: the
graphical depiction
11.6% ∆ = 2.1x
of information
–– Aggregation: the
combination of
information from
multiple sources
–– Filtering: the selective
omission of information 5.5%
to improve relevancy
and legibility
Information Information Information Advanced

Visualization Aggregation Filtering Analytics
Capabilities Capabilities Capabilities Capabilities

CIO Executive Board
IT Practice
CXO KPI
Identification

It is nearly impossible to
uphold all dimensions of
Can All Information Be Perfect All the Time?
quality in all information
sources. Common Dimensions of Information Quality
■■ Organizations with less than

1. Usability—Information is 2. Precision—Information is 3. Currency—Lag time before
perfect information quality
presented in a usable format. sufficiently granular. updated information is accessible.
have two options: hide
information until it is fixed or
4. Accuracy—Information accurately 5. N
on-Duplication—Information 6. T
imeliness—Information
use information selectively in
reflects the real world. is not duplicated. is available when needed.
its current state.
7. Validity—Information conforms 8. Consistency—The information 9. C

ompleteness—Information fields
■■ See pages 74–75 for full
to business rules. conforms to its definition. are complete.
definitions and tips for
measuring each of the nine
dimensions of information
quality.
Information quality has many dimensions;

few information sources comply with all of them.
Option 1: Avoid Using Information Until Option 2: Help Knowledge Workers Use
All Quality Problems Are Fixed Information in Its Current State
“Perfect” quality requires costly Extracts value from information faster

multiyear effort. Helps prioritize most important
External information quality may quality fixes
be unfixable.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board
IT Practice
CXO KPI
Identification
Despite the high impact
on Insight IQ, only about
Trust No One
one-third of knowledge
workers trust information Maximum Impact on Insight IQ1 of Have Confidence in Information
from other functions in Knowledge Worker Confidence in Provided by Other Functions
the organization. Information Quality Percentage of Knowledge Workers
25.2%
■■ This lack of confidence
32%
translates into perceived Have Confidence
low information quality in Information from
and usability. Other Functions
■■ The most trusted function is 16.5%

Finance, followed closely by
Legal, with 47.5% and 43%
knowledge workers willing
to trust information from
these functions, respectively.
■■ Marketing and R&D functions

were the least trusted
functions, with only about
25% knowledge workers
willing to trust information 68%
from these functions. Don’t Have Confidence
Average for All Knowledge Worker in Information from
Top 10% Drivers Confidence in Other Functions
■■ See page 73 for additional Information Quality
(Cleanliness and
details on how each function Accuracy)
assesses the quality of its
own and other functions’
data.
CIO Executive Board

IT Practice
CXO KPI
Identification

Lack of transparency
in how information
Trust From Transparency
is collected, stored,
or analyzed is the Reasons for Lack of Confidence in Information Quality Provided by Other Functions
number-one reason why Percentage of Knowledge Workers
knowledge workers lack
confidence in the quality
of available information. Lack of transparency
is a more significant
Lack of Transparency in How reason for lack of
■■ Knowledge workers need Information Is Collected, 60.0% confidence than
Stored, or Analyzed the underlying
clarity on where information information quality
came from and how it has problems.
been transformed.
Multiple Versions of the Same 42.0%

Information Exist
Lack of Common Vocabulary/ 23.8%

Standard Definitions

CIO Executive Board

IT Practice
CXO KPI
Identification
Flexible and Transparent Analytics
Overview
An approach to information management and architecture that provides the flexibility and transparency knowledge workers
need to conduct the analysis they require within a universe of information related to key business performance questions
Executive Teaching
A lack of understanding of information sources, definitions, quality, and relationships makes it hard for knowledge
workers to assemble the information they need to address key business performance questions. The diversity of
knowledge worker information needs means that it is not possible to support them with predefined reports and cubes
and central support for ad hoc queries is slow and resource-intensive. Instead IT should promote analysis freedom and
confidence by offering a flexible architecture and transparency into underlying information sources.
Component Teachings
1. Business Performance Question Mapping—Identify the universe of information knowledge workers require by
mapping the business performance questions they need to address.
2. Flexible Data Analysis Architecture—Enable flexible, on-the-fly analysis capabilities through in-memory
architecture organized around key business questions.
3. Contextualized Information Access—Provide knowledge workers with access to information organized by their
top business questions.
4. Data Quality Transparency—Provide transparency into data quality and source to help knowledge workers decide
when the data can and can’t be used.
Company Snapshot
GlaxoSmithKline plc
Industry: Pharmaceuticals GlaxoSmithKline (GSK) is one of the world’s leading pharmaceutical firms. GSK
2010 Sales: US$43.9 B produces medicines that treat major disease areas such as asthma, anti-virals,
Employees: 98,485 infections, mental health, diabetes, cardiovascular, and digestive conditions. In addition,
the company is a leader in vaccines and is developing new treatments for cancer.
GSK also markets other consumer products, many of which are among the market
leaders.
CIO Executive Board
IT Practice
CXO KPI
Identification

A knowledge worker
survey revealed high
Assembling Data is Very Time Consuming
dissatisfaction with the
amount of time wasted Knowledge Worker Data Satisfaction Survey Results
trying to assemble data Illustrative Results Shown
provided in disparate
reports and dashboards.
Knowledge Worker Director Director Sales Product Field Sales Field Sales
Pain Points Area Sales Support Director Specialist Manager
■■ GSK’s North American
Sales and Marketing IT 1. Unanswered
department set out to Questions
understand the lifecycle
of information delivery to 2. Missing Data
support business processes.
3. Can’t Access Data
■■ The results revealed a need
for better understanding
4. Tools Aren’t Useful
of key business questions
to improve the speed 5. Slow Response to
and flexibility of analysis Data Needs
capabilities in the sales
6. Data Inconsistency
team.
and Inaccuracy
7. Need More
Historical Data
8. Waste Time
Assembling Data
The most serious pain point No Expressed Problem in This Area

was the amount of time I Experience This Problem, However It Does
knowledge workers spent Not Have a Significant Impact on My Work
assembling data before they
I Experience This Problem, and It Has a
could conduct analysis.
Significant Negative Impact on My Work
CIO Executive Board

IT Practice
CXO KPI
Identification
Reducing wasted
time assembling data
Why Knowledge Workers Waste Time
demands improvements
to data mapping, flexible
Assembling Data
access, and metadata
transparency.
Assembling Data Is
Challenge
Very Time Consuming
Asking knowledge Knowledge workers Knowledge workers

Predefined reports
workers what data don’t know which don’t understand the
and cubes answer
Root Causes they need misses how combination of data data sources’ quality,
a narrow range of
the data should be will answer their definitions, and
business questions.
assembled. business questions. business rules.
“Knowledge workers
wanted to assemble 1 2 3 4
and manipulate data to Metadata
Solution Business Performance Flexible Data Analysis Contextualized
explore cause and effect, Transparency for
Components Question Mapping Architecture Information Access
likelihood of recurrence, and Power Users
predictive analysis, but in our old
model that meant intervention by
IT. We needed to find ways to let
them do this themselves.”
Rob Jester
Director, Enterprise Business
Architecture, Sales and Marketing
GSK
CIO Executive Board

IT Practice
CXO KPI
Identification

To identify the universe

of information knowledge
First, Ask the Right Questions
workers require, start by
isolating the top business
performance questions
they need to answer. Business surveys and focus groups lead to
1. Identify Top Business Performance Questions
identification of top 400 business questions.
Example question: What prescription details help
■■ Asking knowledge workers to predict likelihood to fulfill prescriptions?
what information they want
leads to a siloed approach
that produces information 2. Group Closely Related Questions IT works with business stakeholders to group the
that requires further 400 questions into 20 categories.
aggregation. Example question groups: prescription fulfillment
■■ Instead, the Information

Architecture group examines 3. Map Questions to Data Subject Areas Information architects combine business questions
the business questions and data knowledge to develop a detailed Business
knowledge workers need Qu.1 Qu.2 Qu.3 Qu.4 Qu.5 Qu.6 Subject Area Data Map that documents the
relationships between the questions and the data
answers for; from there, they RX subject areas.
can identify the combination
Example subject areas: prescriber, prescriptions
of information subjects and Prescriber
sources that are necessary
Claims
to answer these questions.
Patient
Product
“To understand how

knowledge workers 4. Identify Related Data Sources Information architects identify related data sources.
needed to assemble Example data sources: patient database, sales
data, we first had to know the database, EDW, prescriber database
questions they were asking.” CRM EDW
Rob Jester
ERP HR
GSK
CIO Executive Board

IT Practice
CXO KPI
Identification
Help knowledge
workers answer business
Flexible Analysis Architecture
performance questions
by equipping them to Data Analysis Tool Architecture
assemble, analyze, and Illustrative
visualize information
from multiple sources as
they need it. Presentation Layer
3
Users can access and segment data through
high-quality visualizations to get fast answers
to their top business questions.
Flexible Analytical Layer

2
Relevant data is moved into the in-memory
In-Memory database of the analysis tool to enable fast
Architecture processing across all possible data dimensions.
Data Sources 1
Data sources are selected based on the business
questions documented by the data map.
Rx Enterprise
Prescriber CRM
Sales Data
DB
DB Warehouse
Note: GSK uses QlikView as its reporting tool.

CIO Executive Board
IT Practice
CXO KPI
Identification

Provide an online data

map to help knowledge
SHOW HOW IT ALL FITS TOGETHER
workers understand
information availability Clickable Data Map Aligned to Business Subject Areas
and interrelationships. Illustrative
■■ The data map provides

an overview of available
information and shows Business Subject Areas: Sales and Marketing
the connections between
information.
Claims
■■ Providing information access
organized by business
subject areas makes it easy
Patient Payer
for knowledge workers
to identify the most relevant
information for their
question or role. Rx Formulary
■■ Detailed data concept

models, fact sheets, and Prescriber Business
information flow charts
lie under each business
subject area, available for Order
heavy analytics users in the
organization. Contracts
Marketing Call Activities
Events (Sales) Non-Retail
Visualizing Data Relationships—A product Clicking on a business subject area

manager can visualize how claims data links provides additional information
back to Rx data through patient data, to help about the data quality, sources,
understand how to conduct analysis. and relationships.
CIO Executive Board

IT Practice
CXO KPI
Identification
Provide full transparency
into information sources,
FULL TRANSPARENCY MAKES DATA
flows, and quality to
increase usefulness
MORE USABLE
and knowledge worker
confidence in the Centralized Metadata Portal
Illustrative
information.
■■ Knowledge workers can GSK Connect

view this information
through the data map. Welcome to Data Point
■■ Metadata and additional Home Data Glossary Subject Areas Standard Reports Data University
information models and Prescription (Rx) Sales Data

training is available for IT
Subject Area Description: Metrics (business facts)
and power users in the
related to both non-retail sales of drug products and
business. retail prescription fills. Retail prescription data are
Quality Transparency—A collected at the national and subnational level.
product manager can review
the Rx Data Fact Sheet to Data Fact Sheet—Provides description of data
learn about the accuracy of A types, sources, granularity, and quality levels
the data and become aware of
any caveats in how it should
Information Flow—Visually depicts flow of
be used for analysis.
information from original source to reporting tools
Data Concept Model—Visually depicts logical

? data model for national prescriptions
Data Owner—Click Here to contact.
Users who need more detail or Users can access training materials
clarification can easily contact and information about registering
data owners. for upcoming training.
CIO Executive Board

IT Practice
CXO KPI
Identification

By providing the
tools and contextual
Faster Assembly, Less Help Required
information needed for
self-service, GSK reduces
the time knowledge
workers spend
assembling information.
Early Benefits
Example: Time to Complete Complex
Analysis on Rx and Claims Data
Patient
Claims
Payer
IT and the rest of the business share an
Rx
enterprise-wide view of customer data needs.
30 Days
Simplified access to information

87%
Reduction in time spent generating reports 4 Days
Traditional New
Approach Approach
The new approach eliminated the

need to create 65 reports when
analyzing patterns in the relationship
between prescriptions and claims.
“Sales and Marketing
have always had strong
demand for
information on a short turnaround.
Now we are equipping them to
get the information they need
faster and with much less effort.”
Rob Jester
GSK
CIO Executive Board

IT Practice
CXO KPI
Identification
Data Fact Sheets
provide power users
Implementation Guide: Data Fact Sheet
with metadata details
to support their
understanding of the
data sources and quality
levels. Data Fields on the Data Fact Sheet
■■ Data Area Name ■■ Integration Requirements

■■ Data Area Type ■■ Dimension Requirements/Gaps
■■ Data Subarea ■■ Data Anomalies
■■ Data Description ■■ Data Security Requirements
■■ Data Objective ■■ Target Delivery Requirements
■■ Data Source/Vendor ■■ Current Usage
■■ Data Granularity ■■ Disaster Recovery
■■ Facts/Metrics ■■ Data Quality Requirements
■■ Source Delivery Mechanics ■■ Date Entered
■■ Source Delivery Frequency ■■ Data Architect
■■ Refresh Strategy ■■ Reference Documentation
CIO Executive Board

IT Practice
CXO KPI
Identification

Analytical power users

can attend a Data
Implementation Guide: Data University
University to deepen
their understanding
Course Schedule
of data sources,
relationships, and
usability. Day One Course
9:00 a.m.–9:10 a.m. Welcome, Introductions, and Course Overview

■■ GSK’s training group 9:10 a.m.–9:30 a.m. Data Use, Disclosure, and Confidentiality
facilitates the training
program, with data 9:30 a.m.–9:50 a.m. Customer Dimensions
architects speaking
9:50 a.m.–10:05 a.m. Break
throughout the sessions
to provide subject matter 10:05 a.m.–10:25 a.m. Alignment Dimensions
expertise.
10:25 a.m.–10:45 a.m. Product Dimensions
■■ Sessions focus on 10:45 a.m.–11:05 a.m. Static Views/Goals Data
understanding the sources
11:05 a.m.–11:15 a.m. Introduction: Retail
of data and relationships
between information, 11:15 a.m.–11:45 a.m. Retail, Dynamic Claims Lifecycle, and Consumer Rx data
enabling knowledge workers
to independently apply 11:45 a.m.–12:30 p.m. Lunch
judgment on data usage. 12:30 p.m.–1:50 p.m. Retail, Dynamic Claims Lifecycle, and Consumer Rx data (Continued)
1:50 p.m.–2:15 p.m. State Data Restrictions
2:15 p.m.–2:40 p.m. Monthly Retail (IRx, PRx)
2:40 p.m.–3:10 p.m. Weekly Rx
3:10 p.m.–3:25 p.m. Break
3:25 p.m.–4:00 p.m. Prescriber Track/Passport
4:00 p.m.–4:20 p.m. CRx (Consumer Rx)
CIO Executive Board

IT Practice
CXO KPI
Identification
Implementation Guide: Data University
Course Schedule (Continued)
Day Two Course
9:00 a.m.–9:05 a.m. Welcome/Introduction
9:05 a.m.–9:35 a.m. Targeting
9:35 a.m.–10:30 a.m. Formulary/Payer Dimensions
10:30 a.m.–11:00 a.m. Break
11:00 a.m.–11:25 a.m. Call Activity
11:25 a.m.–11:30 a.m. Non-Retail Introduction
11:30 a.m.–12:30 p.m. IMS Health: Non-Retail Data
12:30 p.m.–1:15 p.m. Lunch
1:15 p.m.–1:35 p.m. Non-Retail Data
1:35 p.m.–1:55 p.m. Specialty Sales Request System (SSRS)
1:55 p.m.–2:20 p.m. Non-Retail Reports
2:20 p.m.–2:25 p.m. National Data Introduction
2:25 p.m.–3:20 p.m. IMS Health: National Audits Data
3:20 p.m.–3:30 p.m. Break
3:30 p.m.–4:00 p.m. IMS Health: National Audits Data (Continued)
4:00 p.m.–4:20 p.m. National Audits Data from IMS Health
4:20 p.m.–4:50 p.m. Dashboard Reports
CIO Executive Board

IT Practice
CXO KPI
Identification

CIO Executive Board

IT Practice
CIO0361911SYN
Usability can be
knowledge workers
greater flexibility in
TRANSPARENCY
access, and use.

knowledge workers need to Flexible and

answer. Transparent Analytics
CXO KPI
Identification
Data Cubes
Ad Hoc EDW Queries

for All
Dashboards
Standard
Reports
Low
CIO Executive Board

IT Practice
CXO KPI
Identification

The traditional approach

to defining key
Rigid Roll-Ups, Cumbersome Cascades
performance indicators
(KPIs) fails to meet CXO Methods for Defining KPIs
and operational needs
simultaneously.
Traditional Approach 1 Traditional Approach 2 Alternative Approach
■■ Rolling up metrics from Roll Up from Standardized Cascade Down from KPIs Differentiate CXO KPIs and
operations requires Operational Metrics Aligned to Strategy Operational Metrics
extensive systems and
information harmonization CXO KPIs
and may still miss external
or new information that is
CXO:
not used for operations.
Reframe
Strategic
■■ Cascading down from Questions
strategy may be too narrow,
as some operational metrics Operational:
are needed that do not Map Workflow
directly drive strategic Decision Points
outcomes.
■■ Progressive organizations Operational Metrics

recognize that neither
approach is ideal and Misses external information Misses nonstrategic metrics Identifies 18 to 20 CXO KPIs
instead pursue differentiated required for strategy but not required to manage operations (estimated 50% overlap with
approaches to CXO KPIs operations operational metrics)
and operational metrics.
Takes too long and requires Requires ongoing modification Creates flexibility for CXO KPIs and
excessive harmonization of of operational metrics as stability for operational metrics
operations and/or systems strategies change
CIO Executive Board

IT Practice
CXO KPI
Identification
CXO KPi Identification
Overview
A methodology to segment CXO and operational information needs to target the appropriate level of information
depth, quality, and refresh rates for each
EXECUTIVE TEACHING
CXOs’ information needs differ from operational management needs. Executives require a broader perspective, need
speed more than information quality, and have frequently changing needs; operational information needs are narrower,
deeper, and less dynamic. Consequently, a single KPI framework and a single approach to information management will
not meet the needs of both.
Company Snapshot
PepsiCo, Inc.
Industry: Food and PepsiCo offers the world’s largest portfolio of billion-dollar food and beverage
Beverages brands, including 19 different product lines that generate more than $1 billion in
Annual Revenue: US$60 B annual retail sales each. PepsiCo’s main businesses—Quaker, Tropicana, Gatorade,
Employees: 294,000 Frito-Lay, and Pepsi Cola—also make hundreds of other enjoyable and wholesome
foods and beverages that are respected household names throughout the world.
With net revenues of approximately $60 billion, PepsiCo’s people are united
by the company’s unique commitment to sustainable growth by investing in a
healthier future for people and the planet, which the company believes also means
a more successful future for PepsiCo.
Source: http://www.hoovers.com.
CIO Executive Board
IT Practice
CXO KPI
Identification

As PepsiCo migrates from

country-level to Europe-
Consolidated Information for
level management,
its information and
a Consolidated Business
reporting needs change.
PepsiCo Migration from Country-Level Sector-Level Information Needs
to European Sector Organization and Challenges
■■ The current reporting
environment differs
by country, making it Objective
difficult for Europe-level
management to get the
Identify the information needed to manage
information they need.
Spain Italy France Russia the sector.
Sample Sector-Level Management Questions
■■ Where do we get the best returns on trade

promotion spend?
European Sector and Global Functions
■■ Which distributors are most profitable for us?
■■ Procurement ■■ Planning ■■ How can we consolidate transport costs
■■ Finance ■■ R&D1 across Europe?
■■ Commercial ■■ Food
■■ Treasury Technology1 Country-Level Reporting Challenges
■■ HR1
■■ 5,000+ applications and thousands of
country-level reports
■■ Global chart of accounts but local definitions
“Business and markets ■■ Up to eight weeks to aggregate metrics
have changed, and across Europe
following the recession
historical information has lost its
value, so we should forget about
reporting and start thinking about
information.”
Austin Wildmore
Enterprise Architect
PepsiCo 1 Global functions.
CIO Executive Board

IT Practice
CXO KPI
Identification
The information
needs of CXOs and
Diverse and Dynamic versus Focused
operational managers
are fundamentally
and Stable
different.
■■ Executives require high-

level strategic information,
and they tend to focus only CXOs Business Unit and Functional Managers
on one or two objectives, Role ■■ Manage strategy, not operations. ■■ Manage operational performance.
which change frequently. Characteristics ■■ Sequential interest is focused on one ■■ Parallel interest have a portfolio
or two priorities at any given time. of priorities.
■■ Operational managers ■■ Cross-cutting priorities affect ■■ Focused responsibilities lie within
require consistent, reliable, multiple functions or BUs. one function or business unit.
and deep drill downs and
■■ Priorities change with strategy. ■■ Priorities are stable.
their metrics are relatively
stable.
Information Broad Perspective—Information from Diverse Information from Limited Sources

Needs Internal and External Sources
Information Needs Change Frequently Information Needs Are Stable
Where Necessary, Speed to Insight Over Quality and Depth Are Critical
Quality and Depth
CIO Executive Board

IT Practice
CXO KPI
Identification

IT should frontload its

involvement in reporting
Reframe Strategic Questions
by helping to reframe
strategic questions
to Identify CXO KPIs
to make the business
outcomes measurable. Process for Identifying CXO KPIs
■■ PepsiCo’s enterprise Illustrative

architects partner with 1. Frontload Participation in Strategic Discussions
IT business relationship Original Strategic Goal
■■ Enterprise architect supports information
managers and Grow our business in Russia.
collection for annual refresh of three-year
transformation directors
strategic plan.
in Finance to provide What Business Outcome Are We Trying
■■ Proactive outreach anticipates information
guidance during the annual to Achieve?
requests.
strategic planning cycle. We want to become the market leader by
2. Reframe Strategic Questions to Make the exploiting production, distribution, and
■■ Their goal is to define marketing synergies between our soda, snack,
Business Outcomes Measurable
make strategic objectives and juice businesses.
measurable. Two reframing questions
■■ What business outcome are we trying What Can We Measure to Ensure We Are
to achieve? Making Progress on That Business Outcome?
■■ What can we measure to ensure we are ■■ Percentage of total sales consolidated with
making progress on that business outcome? a single distributor in each region
■■ Percentage of total trade promotion spend
3. Identify and Collect Information Needed directed to a single distributor in each region
■■ Percentage of revenue generated through
4. Stop Measuring Objectives That Are Removed a shared sales force
During the Strategy Refresh ■■ Percentage of juice products carried by
“We are not trying to
our soda distribution fleet
define strategy, but
■■ Market share by region
unless we can help
make the strategy measurable we
can’t provide the information the
company’s leadership needs.”
Austin Wildmore
PepsiCo
CIO Executive Board

IT Practice
CXO KPI
Identification
The CXO KPIs require an
approach to information
CXO KPIs Require a Different Approach
management that
prioritizes speed and
to Information Management
flexibility over quality
and depth. Implications of CXO KPIs for Information Management
■■ Use the KPIs as a guide Broad 1. Prioritize Sector-Level Information Standards of Enabling CXO KPIs
to avoid overinvesting in Perspective Most information does not need to be harmonized across countries or
enterprise-level information
functions. The CXO KPIs indicate where harmonization is needed.
standards or reports.
2. Integrate Internal and External Information
CXO KPIs often combine internal and external information.
Frequently 3. Retire Old Reports

Changing Reports fall out of use as strategies evolve so review and retire regularly.
Needs
4. Avoid Overinvesting to Meet Short-Term Needs
CXO priorities change, so avoid overinvestment in gathering information
that may only be needed for a year or so.
Speed and 5. Manual Effort Is Okay

Brevity The need for quick information often requires the data to be pulled
together manually. For short-term needs, automation may not be justified.
6. Drill-Downs Only by Exception

“A lot of important XOs rarely need drill-downs—save time and effort by providing drill-
C
information is now downs on request rather than enabling them proactively.
external. You need to
approach the data with a specific 7. Accept Gaps in System Architecture
question; you can’t try to Incomplete information is sufficient if it avoids delays. Set information
aggregate it all.” management priorities around filling in the gaps.
Austin Wildmore
PepsiCo
CIO Executive Board

IT Practice
CXO KPI
Identification

Highlight information
gaps to drive to
Highlight (Don’t Hide) the Gaps
consensus on
priorities for systems Analysis of Sources of Information Gaps
harmonization or Illustrative
integration.
■■ To establish priorities for

change, PepsiCo enterprise
Russian Distributor Component Business Model—Russia
Consolidation Report
architects use a model of To accurately measure sales through a common distributor and the reuse of distribution
core business components assets, we need to integrate operational systems for soda and snacks, and fill functionality
overlaid with system gaps in our commercial and supply chain systems.
information to demonstrate
to business partners the Snack
Soda Juice Dairy
sources of gaps in CXO Foods
information needs.
Finance Vendor A
Supply
Vendor A Vendor B Vendor B Vendor A
Chain
Vendor A Vendor C Vendor B Gap
Commercial Vendor C
Gap
Operations Unintegrated Unintegrated Vendor D

System System
CIO Executive Board

IT Practice
CXO KPI
Identification
Identify operational
metrics by identifying
MAP OPERATIONAL WORKFLOW TO
the information needed
at key decision points
FIND DECISION POINTS
in business processes.
Decision Points in the Develop and Manage Sales Plans Process
Illustrative
■■ While CXO KPIs are
identified by reframing
strategic questions, PepsiCo
Manage Customers Manage Manage
takes a different approach Process
and Accounts Customer Sales Sales Orders
to defining operational
metrics used by managers
below CXO level.
■■ These operational metrics

are linked to decision points
Perform Perform Record
in business processes. Close
Workflow Pre-Sales Sales Outcome
the Sale
Activities Calls of Sale
Decide Which Decide on

Decision
Prospects to Discount
Points
Call Level
■■ Expected value ■■ Average discount

Information ■■ Gap to quarterly goal offered in quarter
Needs ■■ Product availability ■■ Previous prices paid
by prospect
Source: “Process Classification FrameworkSM,” APQC, September 2009, http://www.apqc.org/knowledge-base/documents/apqc-process-classification-framework-pcf-cross-industry-

excel-version-51-0.
CIO Executive Board
IT Practice
CXO KPI
Identification

By focusing narrowly on
CXO information needs,
Better Decisions, Faster
PepsiCo International
significantly cut the Time to Respond to Requests for New Reports Reduction in Report Volume
time needed to gather Illustrative
new information and
the number of legacy
reports. 8 Weeks
1. Immediate elimination
of 1,000+ legacy reports
2. Ongoing discipline to
< 1 Week refresh or retire reports
2009 2011(E) 2009 2010 2011 2012
“By asking the right questions, we can now track

progress against our goals in Russia despite
imperfect information. In the past we were
swamped with metrics, few of which were useful.
Now we only get those that matter.”
Simon Black
European Commercial Transformation Director
PepsiCo
CIO Executive Board

IT Practice
CXO KPI
Identification
The Information Architect
role is critical to IT’s
Information Architect
ability to tie business
questions to the data
Key Responsibilities New-to-World Skills
knowledge workers
need, to enable flexible ■■ Conducts meetings with senior IT leadership ■■ Information analysis
analytical capabilities. and key business stakeholders across the (structured and unstructured)
organization to understand information needs ■■ Information integration
■■ Analyzes the objectives of information
■■ We estimate that demand Other Important Skills
consumers
for information architects
will increase by 250% by ■■ Visualizes and documents the inter-relations ■■ Information architecture development
2015, the largest increase of between information needs, technology, and ■■ Information policy formation
any role in corporate IT. design
■■ Information modelling
■■ Devises metadata schemas for classifying
content according to business needs ■■ Taxonomy creation
■■ Involved in prototyping and usability testing ■■ Stakeholder management
■■ Works with business stakeholders and IT
project teams to define and promote enterprise
information standards
Possible Backgrounds and Hiring Sources

Most likely to have a background with a consultant or vendor specializing in
information management or information-centric products
Sourcing Difficulty Seniority
Middle Management
CIO Executive Board

IT Practice
CXO KPI
Identification

A user-experience
specialist collaborates
User-Experience Specialist
with service managers
and end users to
understand and improve
user experience and ■■ Analyzes business and functional requirements ■■ Information visualization
workflow for new and ■■ Creates user-centered design ■■ User-behavior analysis
existing applications.
■■ Improves the user experience ■■ Usability design
■■ Visualizes and presents information in a user- Other Important Skills
■■ The role designs and friendly manner to end users
configures user-centric ■■ Agile development
interfaces for in-house
and cloud applications.
■■ Business domain analysis
■■ Functional requirements analysis
■■ This enables end users ■■ Information architecture development
to access, visualize, and
navigate information and
■■ Requirements management
analytics with ease. ■■ Usability design
■■ The skills required in this

role match the primary skill
sets of business analysts,
data analysts, and software
developers. Likely to have a specialist background in behavioral science, graphic design, or
product design; more likely to be found in a consultant or other specialized role
than in a corporation
Sourcing Difficulty Seniority
Individual Contributor
For full details of this role,
see the user-experience guru
job descriptions on the CIO
Executive Board website.
CIO Executive Board

IT Practice
CXO KPI
Identification
Key Takeaways: Enable Information Usability
Make Usability a Core IT Competency: IT requires new skills and processes to improve the usability of information and analytic tools. Drive knowledge worker
productivity by letting them choose the tools they find easiest to use, by offering greater transparency into quality levels, and by increasing their ability to
navigate and analyze information from diverse internal and external sources.
At Organizations with a High Insight IQ... At Other Organizations...
1. Knowledge workers choose from a portfolio of tools that cover

One toolset is provided for all purposes.
key capabilities.
2. Information presentation aggregation and filtering are top priorities. Investment in advanced analytics is prioritized over presentation.
3. Knowledge workers can see all the information available to them

There is no single list of all available information.
and the relationships between that information.
Knowledge workers cannot ascertain the quality of information available

4. Information quality levels are transparent to knowledge workers.
to them.
5. KPIs are defined by reframing strategic questions. KPIs are rolled up from operational metrics or cascaded down from strategy.
6. Speed to insight is prioritized when enabling CXO KPIs. No KPIs are enabled until the underlying information is complete.
7. IT organizations invest in information architects and user experience

IT staff lack information architecture or usability skills.
specialists.
CIO Executive Board

IT Practice
CIO0361911SYN

CIO Executive Board

IT Practice
CIO0361911SYN
Knowledge workers
are universally more
APPENDIX: Keeping Things Close to Home
confident about the
quality of information Average Knowledge Worker Confidence in Information Quality from Their Own Function
from their own function and in Information from Other Functions
than they are about
information from other 68%
High Confidence in Information from Own Function
functions.
62%
High Confidence in Information from Other Functions
56%
54%
48%
45% 44%
38%
34% 35% 34%
27% 27% 26%
Legal Finance R&D and HR Customer Sales Marketing

Engineering Service

CIO Executive Board
IT Practice
CXO KPI
Identification

Appendix: Definitions of information Quality Dimensions
Definition Metrics (Illustrative) Monitors (Illustrative) Ease of Measurement Comments

Measurement
Timeliness The relative ability of data Days between product Manual quarterly audit of lag time Medium Like currency, this depends
to support a given process launch and support between new product launches on data flow information
within the timetable document publish and publication of supporting to understand whether
required to perform the date documentation performance goals are met.
process
Validity The degree of Percentage of Automated cross-check of product Medium It is usually fairly
conformance of data to product codes in use codes in customer database straightforward to create
its domain values and corresponding to valid against master product database automated monitors to
business rules product codes to test code validity measure this characteristic.
Consistency The conformance of the Match rate between Automated monthly audit of Medium Not widely used; this often
meaning of data with its internal and external match rate between all internal and reveals a root cause of
data definition (also called product nomenclature external product names, performed accuracy problems.
Definition Conformance) by macro-comparing Excel fields
for exact match
Completeness The characteristic of Percentage of Automated scan of customer High This is easy to measure
having values for the data complete customer database to measure percentage and is recommended as a
fields entries with complete with existing, valid e-mail address starting point to illustrate why
e-mail information entries monitoring matters.
CIO Executive Board

IT Practice
CXO KPI
Identification
Appendix: Definitions of information Quality Dimensions
(Continued)
Definition Metrics (Illustrative) Monitors (Illustrative) Ease of Measurement Comments

Measurement
Usability The degree to which the Customer satisfaction Survey of business data customers Low Measurement is usually done
information presentation with format and clarity to determine satisfaction with level through some sort of customer
is directly and efficiently of inventory data of usability of data satisfaction survey.
usable for its purpose
Precision The characteristic of Percentage of Automated scan of delivery Medium This characteristic is not widely
having the right degree deliveries scheduled schedules to determine percentage used within Intel, and problems
of granularity in the data as an hourly level of with precision of one or two hours usually surface in usability
values precision tests.
Currency The lag time between Lag time between Automated measurement of Medium Measuring this requires
when information is inventory scan-in and time stamp differences between information about the flow
updated and when it is updates to supply inventory scan-in and data refresh of information—server, ETL
knowable in a redundant chain planning system logs, etc.
database
Accuracy The degree to which the Percentage of active Manual audit conducted (by Low This is the hardest
data accurately reflects the employee records calling or e-mailing a sample characteristic to measure
real-world object or event with accurate home of employees) to assess accuracy automatically; usually requires
being described address information of home address on file manual audits.
Nonduplication The degree to which a Percentage of Automated cross-check of product High This characteristic is very easy
one-to-one correlation products listed more databases to determine duplicate to measure and can uncover
exists between records in than one time in entries basic problems in business
a dataset and real-world product database processes.
object or events
CIO Executive Board

IT Practice
CXO KPI
Identification

CIO Executive Board

IT Practice
CIO0361911SYN
1. E
nable 2. S
bserve and Question
Case Examples

1



Specialists

1 Pseudonym.
CIO Executive Board

IT Practice
CIO0361911SYN
77
Unquestioning faith
in analysis can be as
Why Balancing Analysis With Intuition Matters
damaging as overreliance
on intuition.
Risks from Overreliance on Intuition

■■ Unable to account for complex
environments
High ■■ Highly subject to personal biases
■■ Hard to replicate, scale, or use to
foster buy-in
Reliance on Intuition
Risks from Overreliance on Analysis

■■ May hinge on incorrect assumptions
or inaccurate data
■■ Unable to adapt to discontinuous
change
■■ Fails to account for situational
context by looking at the data
too narrowly
Low
Low High
Reliance on Analysis
CIO Executive Board

IT Practice
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Knowledge workers fall
into three categories
The Value of Big Judgment
based on their relative
reliance on analysis
and intuition.
Visceral Decision Maker
■■ Informed Skeptics like 19% of the sample
High
to make biases and Average Insight IQ1 = 45
assumptions explicit and
often question their own
thinking; they listen to ■■ Seldom trusts analysis
alternative points of view ■■ Has weak analytic skills Informed Skeptic
without defaulting to ■■ Makes decisions 38% of sample
consensus-based decisions. unilaterally Average Insight IQ1 = 60
■■ Applies judgment
■■ The most common profile
Reliance on Intuition
to analysis
is the Unquestioning ■■ Has strong analytic skills
Empiricist, who tends to rely ■■ Listens to others and
excessively on analysis, so
is comfortable with
in many organizations the
dissent
greatest risk comes from too
much analysis, not too little.
Unquestioning Empiricist
43% of the sample
■■ Visceral Decision Makers ■■ Trusts analysis over Average Insight IQ1 = 48
are more rare but represent judgment
almost one-fifth of ■■ Supplements analytics
knowledge workers. with precedent
■■ Values consensus
■■ The profiles are determined
using cluster analysis to Low
identify common groups of Low High
analytic skills and behaviors. Reliance on Analysis
1 Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.

CIO Executive Board
IT Practice
CIO0361911SYN
Support Big Judgment, Not Just Big Data 79

Informed Skeptics are

most common among
Learned Skepticism
the more senior ranks.
Knowledge Worker Profile Distribution
■■ While knowledge worker
profiles vary substantially
by seniority, there is little Unquestioning
Empiricists
difference by organizational
Visceral Decision
tenure, indicating that 22% Makers
institutional knowledge is
not a distinguishing factor. 37% Informed Skeptics
43%
47%
9%
■■ Similarly, education and
extent of cross-functional
experience do not matter.
14%
17%
21%
69%
50%
41%
32%
Individual Midlevel Managers Senior Managers Executive

Contributors Leadership
As knowledge workers gain experience, they increasingly rely

on judgment to supplement data when making decisions.

CIO Executive Board
IT Practice
CIO0361911SYN
Unquestioning
Empiricists represent
No Function Is Immune from Overreliance
at least one-quarter of
knowledge workers in
on Data
every function.
Knowledge Worker Profile Distribution
■■ Informed Skeptics represent
the majority in General
Management and Corporate
Strategy.
25% 31%
30%
36% 39%
■■ Unquestioning Empiricists 41% 43%
47% 50% 48%
most heavily reside in
60% 57%
Marketing, R&D, and HR.
25% 12%
19%
■■ Visceral Decision Makers
are most heavily represented 14%
30% 23% 18%
in Sales, Communications, 16%
18% 14%
and Customer Service.
14%
18%
58%
50% 51%
47%
35% 36% 37% 37% 39%
34%
29%
22%
g
&D
en s/
ce
IT
ns
gy
ga
en
le
ic
tin
H
em er
io
t
te
Sa
rv
R
Le
em
na
ke
ur Op
at
ra
Se
Fi
ic
ar
ag
St
oc /
un
er
Pr ion
M
an
te
m
m
t
ra
M
to
uc
om
po
al
us
od
er
C
or
C
Pr
en
C
G
Informed Skeptics Visceral Decision Makers Unquestioning Empiricists

CIO Executive Board
IT Practice
CIO0361911SYN

Informed skepticism
can be fostered through
WHO DOES WHAT TO BUILD INFORMED SKEPTICS?
a combination of
decision-making models,
analytical skills training
and coaching, and tools
for collaborative decision
making. Activities and Responsibilities for Fostering Informed Skepticism
■■ Some areas have little to do

with IT, while others clearly
require technology. Clear Role for IT Organizational “Gray Area” Business Leadership Role (Unlikely
to Involve IT)
■■ Training and coaching Build Tools for Collaborative Provide Analytical Skills Training Define Decision-Making Models
represent “gray areas” of Decision Making and Coaching and Behaviors
responsibility where CIOs
should work with their
Examples Examples Examples
business counterparts to
determine ownership. ■■ Build workflow tools that ■■ Shift from up-front training ■■ Introduce decision-making
connect stakeholders involved on new tools to ongoing techniques that promote critical
in collaborative decision making. development of analytic skills. thinking and make biases and
■■ Hire and incentivize analytics assumptions explicit.
staff on their effectiveness as ■■ Embed analysis and collaborative
coaches, not just analysts. decision making in competency
models and performance criteria.
CIO Executive Board

IT Practice
CIO0361911SYN
Collaborative tools can
enable more effective
IT Role: Tools for Collaborative
decision making. Decision Making
■■ Collaborative decision Essential Features of Collaborative Decision-Making Tools
making, listening to others,
and accepting dissent are Wisdom of Crowds—Find and connect people Annotations and Note-Taking—Add context
characteristics of informed across the organization to get broad input into to data by annotating reports and integrate all
skepticism. ad hoc decisions. comments in a note-taking platform.
Collaborative Workflow—Improve the quality Integrated Workflow—Integrate information and

of decisions by enabling group comments and collaborative processes into a single platform.
conversations to reach consensus.
Information Freedom—Allow for a free flow of Decision Transparency—Develop a corporate

relevant and contextual information by moving decision repository to record discussions,
data out of siloed e-mail platforms. information, and analyses around past decisions.
Sample Collaborative Decision-Making Tools
EMC uses a The UK Department Macquarie University

collaboration platform for Work and uses Yellowfin to
Do use collaborative decision-
from Jive to enable an Pensions uses improve information
making tools to gather a wide
internal idea-sharing collaborative tools accessibility and
range of opinions from experts
and development from Spigit to create research data
across the organization.
community. a virtual stockmarket visualization.
where employees
DON’T use collaborative tools
“invest” in ideas they
when speed is of paramount
believe have value.
concern and one person’s
expertise far exceeds that of
others. Source: MC Corporation, EMC|ONE: A Journey in Social Media, 2008; Dinham, Peter, “Yellowfin, Macquarie Uni Win Global BI Award” ITWire, 9 December 2011, http://www.itwire.com/
it-industry-news/market/51672-yellowfin-macquarie-uni-win-global-bi-award; Applications Executive Council, “Redefined Governance for Innovation and Improved Project
Prioritization” Business 2.0, vol. 1, Corporate Executive Board, Washington DC, 2011, p. 83; Starner, Tom, “Profiting by Playing Games” Human Resource Executive Online, 27
CIO Executive Board October 2011, http://www.hreonline.com/HRE/story.jsp?storyId=533342586.
IT Practice
CIO0361911SYN

A knowledge worker’s
employment history and
Shared Role: Analytical Skills Training
educational background
have less impact on their
and Coaching
Insight IQ than providing
access to training and Maximum Impact on Insight IQ1 of Knowledge Worker Training, Support, and Background
analysts.
Factors that can be influenced in the
24.7% near term have far greater impact than
long-term factors such as educational
background or employment history.
18.6%
2.4%
0.3%
Training Available Analyst Available Has Held a Science or Business

to Provide Help Previous Role in Degree Program
Another Function
CIO Executive Board
IT Practice
CIO0361911SYN
Effective information
and analytics training
Only 25% Find Information and Analytics
is a strong driver of
Insight IQ, but only 25%
Training Effective
of knowledge workers
experience effective Attendance and Effectiveness of Information and Analytics Training
training.
Total
Knowledge 100%
Workers
No Training 36%
Available
Did Not Attend 15% Sixty-four percent of knowledge

Training
workers have access to training
on information and analytics...
Attended Training but

24%
Didn’t Find It Effective
...but only 25%

Attended Training and 25% attend and find
Found It Effective
it effective.

CIO Executive Board
IT Practice
CIO0361911SYN

Shift IT’s focus from

up-front training on new
TRAIN TO ANALYZE
tools to ongoing support
for conducting analysis. Maximum Impact on Insight IQ1 of Different Types of Information and Analytics Training
Training on how to
28.7%
conduct analysis has more
than twice the impact on
Insight IQ of any other
training subject.
13.2% 13.7%
Tool Functionality Definitions and Conducting Analysis

Terminology
CIO Executive Board
IT Practice
CIO0361911SYN
Information Training Portfolio
Overview
A training portfolio that provides ongoing support to knowledge workers for conducting analysis and using analytics
tools, reports, and dashboards
Executive Teaching
Training on new tools, without ongoing support and reinforcement, is ineffective as knowledge workers struggle to
apply the training to their jobs. Training that focuses on the functionality of the tools and does not cover information
sources or analytical techniques is also ineffective. Adopt a multipronged approach to design training offerings that
support knowledge workers across the tool lifecycle and focus on conducting analysis.
Company Snapshot
Tiffany & Co.

Industry: Jewelry Stores Tiffany & Co., founded in 1837, engages in the design, manufacture,
2010 Revenue: $3.1 B and retail of fine jewelry. The company also offers brand
2010 Employees: 9,200 merchandise, including timepieces and clocks, sterling silver
merchandise, crystal, custom engraved stationery, and fashion
accessories. The company sells its goods exclusively through about
230 Tiffany & Co. stores and boutiques worldwide, its website,
business-to-business accounts, and catalogs.
Source: http://finance.yahoo.com; http://www.hoovers.com.
CIO Executive Board

IT Practice
CIO0361911SYN

IT struggles to meet
rapidly increasing
Tools Training Falls Short
demand from
knowledge workers
for deeper support on
using information and
conducting analysis.
IT Challenges Up-front, one-size-fits-all training doesn’t Knowledge workers fail to capture value
recognize varied knowledge worker needs and because they don’t understand the data or
■■ Most existing training learning styles. know how to conduct analysis.
programs are triggered
by the launch of a new tool.
Cost of Failure ■■ Limited training penetration due to unmet ■■ Under-utilized or misused analytics
■■ To meet this increased varied needs capabilities
demand for analyzing ■■ Lower tool adoption ■■ Unexpected support costs
information, IT ends up
providing ad hoc training
and support.
Solution 1. Make training an ongoing process with a portfolio of training programs mapped across
a tool’s lifecycle.
2. Deliver customized training programs using a variety of channels that map to diverse
knowledge worker needs and preferences.
3. Provide training on understanding information and conducting analysis, not just
using tools.
CIO Executive Board

IT Practice
CIO0361911SYN
Provide a portfolio of
training offerings that
INFORMATION TRAINING PORTFOLIO
supports knowledge
workers across the tool Illustrative
lifecycle. Training Focus Areas
Provide
Build Introduce New Train on Provide
■■ Effective analytics training Support for
Awareness Analysis and New Tool Support
Analysis and
is designed as an ongoing Type of Training of New Tool Information Functionality for Tools
Information
process, not just an event
tied to the launch of a new BI Day ■■ On-site training days on BI/
tool. analytics tools, new subject
matter, reports, dashboards,
query techniques
■■ Offering a range of training
options and channels Video ■■ Online training modules for all
ensures that training maps Training staff globally
■■ Includes training on data
to diverse knowledge worker
subjects as well as a series of
needs and preferences. training on new data marts
Classroom ■■ Weekly program delivered

Training to business unit data stewards,
culminating in a project
■■ Primarily for new analysts
or refreshers
Analytic ■■ For communication to users

Team and troubleshooting
Portal
Ongoing ■■ A BI specialist for two months

Analyst after launch of new tool
Support or information
■■ Data stewards for intra-
departmental training and
support
Ongoing training is provided across the tool lifecycle,

with greater focus on supporting knowledge workers
in using information and analytic techniques.
CIO Executive Board

IT Practice
CIO0361911SYN

Focus training on how to

conduct analysis and use
Train on Analysis, Not Just Tools
information, not just tool
functionality. Business Intelligence Day Agenda List of Online Video Training Topics
Illustrative Illustrative
■■ Tiffany & Co. uses BI Days Agenda

Learning About Our Information—Six Video
to educate groups of
Modules
knowledge workers about 9:00 a.m. BI: The Journey to Here
new information, reports,
■■ Enterprise data, data marts, general
dashboards, tools, and query 10:00 a.m. How to Analyze and Use the New information
techniques available to Customer Information
them. Hands-on training demos and tips
■■ Sales data
■■ General Ledger data
■■ Business subject matter 11:00 a.m. How to Analyze Reports and ■■ Merchandising data
experts and vendors Dashboards
are also invited to lead Standard offerings and ad hoc analysis ………
these sessions; BI Days
are publicized internally Noon Lunch
by e-mail and office
message boards. 1:00 p.m. Overview of Enterprise BI Learning to Use Our Analytic Tools—
and Data Warehouse Seven Video Modules
■■ In addition, the information Includes BI strategy (Specific to Information Types)
management group creates
online videos to offer 2:00 p.m. Using Queries to Help You Make ■■ Analytics tools
knowledge workers easy, Decisions
on-demand access to Tips and tricks
■■ Reporting tools
training on tools and ■■ Dashboard tools
information. 3:00 p.m. BI Technologies and New Tools
…….
■■ Training videos are typically
15 to 20 minutes long Facilitators: Information management group Audience: 800 ad hoc knowledge workers,
and range from general, Attendees: 100 business unit management staff other viewers of analytics, reports and
beginner level to specific, dashboards
Location: Large venue, agenda tailored to
advanced level. Location: Online
regional needs
Training Focused on Conducting Analysis or Using Information

CIO Executive Board
IT Practice
CIO0361911SYN
Hire and incentivize
analytics staff on their
HIRE QUANTS WHO CAN COACH
effectiveness at coaching,
not just analyzing. Maximum Impact on Insight IQ1 Skills That Organizations Rate as Most
of Hiring Criteria for Analyst Skills Important When Hiring Analysts
Percentage of Organizations
■■ Although organizations rate
business knowledge skills as
nine times more important
when hiring analysts than
coaching skills, business Business
14.8% Knowledge 45%
knowledge skills provide no Skills
opportunity for impact on
Insight IQ.
6.8%
Quantitative 36%
■■ See page 98 for interview Skills
questions to use to test Quantitative
for coaching skills when 0.0% Skills
recruiting new employees
Coaching Technical Business
and page 99 for an example Skills Skills Knowledge
Technical
9%
of a training program to Skills
Skills
develop coaching skills.
(11.1%) Coaching 5%
Skills

CIO Executive Board
IT Practice
CIO0361911SYN

Develop specialized
information analysis and
Role Profile: Information Insight Enablers
insight skills to bolster
IT’s ability to support Profile of the Information Insight Enabler Role
knowledge workers.
■■ A subset of the skills ■■ Market and competitor analysis
■■ Understands the decision-making process and
required in this role are the ■■ Information insight generation
the workflows of business unit heads and service
same as those often found in ■■ Information visualization
managers
data analysts; consequently, ■■ Unstructured information analysis
data analyst is the most ■■ Identifies knowledge workers’ information needs
common background for the Other Important Skills
■■ Represents information in a user-friendly manner
information insight enabler
■■ Identifies trends and patterns; generates insight ■■ Business domain analysis
role.
for business units and senior leadership ■■ Coaching and Training Skills
For full details of this role, ■■ Develops framework and process to analyze Data warehousing
■■
■■
see the information insight unstructured information

■■ Functional requirements analysis
enabler job description at ■■ Performs market and customer research and
www.cio.executiveboard. analysis and creates dashboards and scorecards
■■ Global teaming and remote collaboration
com/Members/ ■■ Information taxonomy creation
■■ Teaches knowledge workers about their
ResearchAndTools/Abstract.
information and analytic tools ■■ Information modeling
aspx?cid=100236298.
■■ Usability design

Likely to have a background in market or financial research or in analytics
or statistics
“Right now, we have Sourcing Difficulty Seniority

lots of people who can
build reports. In the
future, I will need people who
help business partners find the
right insight.” Individual Contributor
CIO
US Construction Company
CIO Executive Board

IT Practice
CIO0361911SYN
Informed skepticism can
be fostered through the
Business Leadership Role: Decision-Making
introduction of decision
models and behaviors
Models and Behaviors
that encourage inference
and judgment. Maximum Impact on Insight IQ1 of Decision Models and Performance Criteria
■■ These are areas where IT has

little influence and therefore 24.6%
requires business leadership.
■■ Performance metrics are

transparent and in regular
use and are supplemented 17.4%
16.4%
by formal measures of
15.0%
analytic ability.
Decision Functional Performance Performance

Processes Are Performance Review Is Based Criteria Explicitly
Formalized Metrics Are on Measurable Include Ability
Regularly Goals to Conduct
Reported Analysis

1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when
a knowledge worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores
“low” on a driver. The effect of each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic 2011.

CIO Executive Board
IT Practice
CIO0361911SYN

An insight-driven culture
is best realized when
Sample DECISION-MAKING Models
business and decision-
making processes are Essential Features of Decision-Making Models That Promote Informed Skepticism
formalized.
■■ Not all decision-making Formalized Methodology—Mitigate biases and Nonlinear Methodology—Adapt to change by
models will promote foster scalability by introducing a systematic incrementally following decision steps that can
informed skeptics (e.g., decision-making process. be tested and revised at short intervals.
models such as Six Sigma
and lean that focus on
reducing variation and Critical Thinking—Improve decision making by Incorporate the Unknown—Actively engage
ambiguity are less likely to combining judgment and common sense with with uncertainty by separating the known from
lead to informed skepticism). normative decision-making processes. the unknown in the decision-making process.
■■ See Appendix on page

97 for more details about Explicit Biases and Assumptions—Surface and
applying critical thinking adjust for biases and document assumptions so
skills to decision making. their validity can be tested.
Sample Decision-Making Models
Design Thinking The Mintzberg Model

Integrates intuition, reason, and imagination Incorporates feedback loops to design an
into strategic decision making iterative and dynamic decision-making model
Collaborative Decision Making The Cynefin Framework

Connects disparate teams of experts across Reflects the complexity of strategic decisions
the organization to collectively analyze problems by acknowledging ambiguity
CIO Executive Board

IT Practice
CIO0361911SYN
Informed skepticism can
be fostered by setting
Sample Performance Criteria
the right expectations
through performance Proficiency Criteria Leadership Criteria
criteria.
■■ Competency models and ■■ Uses analytics to identify trends and works ■■ Creates an analytic culture and drives usage
performance criteria should to understand the driving factors by building analytics into day-to-day work
also describe the analytical, ■■ Uses analytics in developing plans ■■ Holds team accountable for using analytic tools
skeptical, and collaborative to achieve target business results and encourages full but skeptical usage
behaviors needed for
■■ Shows good judgment in interpreting ■■ Encourages a collaborative culture and is open
informed skepticism.
the results of analytics to dissent
■■ Collaborates with others in analysis
and decision making
CIO Executive Board

IT Practice
CIO0361911SYN

Key Takeaways: Support Big Judgment, Not JUST Big Data
Foster Informed Skeptics: If knowledge workers cannot apply judgment to analysis, greater information access may do more harm than good. Therefore, IT and
business leaders must both promote a skeptical attitude toward information and analysis through decision models, performance criteria, tools, and training.
Most knowledge workers are Visceral Decision Makers or Unquestioning

1. Most knowledge workers are Informed Skeptics.
Empiricists.
2. Decision making is supported by collaborative tools. There are no tools for collaborative decision making.
3. Analytical skills are developed. Analytical skills are hired.
4. Training is an ongoing process, so a portfolio of training offerings

Training is an event, tied to the launch of a tool.
is available at all times.
5. Training focuses on using information and conducting analysis. Training focuses on using the tool.
Analysts are hired and evaluated solely for their analytical skills and domain
6. Ability to coach is a key job requirement for analysts.
knowledge.
Decision models and performance criteria are lacking or promote

7. Decision models and performance criteria promote informed skepticism.
unquestioning reliance on data.
CIO Executive Board

IT Practice
CIO0361911SYN
Foster critical thinking
skills to increase
Appendix: Critical Thinking Skills
informed skepticism.
Nine Gates for Critical Thinking1

Prepared for First Data Corporation
Clarity
1
Could you illustrate what you mean? Could
you give examples?
Accuracy
2
How could you find out if that is true?
Precision
3
Could you be more specific? Could you give
more details?
4 Relevance
How does the data relate to the problem?
Depth
5
What are some of the difficulties that may arise
that we may need to deal with?
Breadth
6
Do we need to look at this problem from another
perspective?
Logical
7
Does the data make sense together?
Significance
8
What is the most important problem to consider?
Fairness
9 Do you have biases that will affect the outcome?
1 Copyright Complete Intelligence LLC.

Source: Complete Intelligence LLC.
CIO Executive Board
IT Practice
CIO0361911SYN

Identify candidates
with coaching skills
Appendix: Identifying Coaching Skills During
by asking them to
give examples of the
Interviews
following behaviors
Please describe a time when you...
and competencies.
1. Assessed Development Progress: Reviewed 8. Gave Feedback on Personality Strengths: Gave
where an employee stood against their an employee feedback during a performance
development goals review about their personality strengths
2. Helped to Draft an Individual Development Plan: 9. Gave Advice from Your Own Experience: Gave
Helped an employee create a plan to meet their an employee advice based on your experience
development goals
10. Helped an Employee Apply New Skills/
3. Ensured That an Employee Had the Necessary Knowledge: Helped an employee apply new skills
Skills/Knowledge: Ensured that an employee or knowledge in their work
had the required skills and knowledge before they
11. Helped an Employee Find Training: Help an
needed it
employee find the training they need
4. Ensured That a Project Provided Learning:
12. Passed Along a Development Opportunity:
Ensured that each project or assignment was
Passed along relevant development opportunities
a learning experience
to an employee
5. Explained Performance Evaluation Standards:
13. Passed Along Job Openings: Passed along
Helped an employee understand their
information about internal job openings to an
performance review standards
employee
6. Gave Feedback on Performance Strengths:
14. Provided an Employee with Experiences
Gave an employee feedback during a performance
That Develop: Helped an employee obtain the
review about their performance strengths
experiences at work that helped them develop
7. Gave Feedback on Performance Weaknesses: over time
Gave an employee feedback during a performance
15. Taught a New Skill or Procedure: Taught an
review about their performance weaknesses
employee a new skill, concept, or process
Source: CLC Learning and Development research; CIO Executive Board research.
CIO Executive Board

IT Practice
CIO0361911SYN
Use this process to help
employees learn how to
Appendix: A Formula for Coaching
be effective coaches. Success
BCBSMA’s Learning Process for Coaching
Align Coach Factor Team

Reflect on Ideal Learn Coaching Address Common
Profile with Dynamics and
Coach Attributes Fundamentals Coaching Pitfalls
Leadership Style “Coachability”
■■ Which coach ■■ What are my ■■ How is my ■■ How should my ■■ What are

from my own natural strengths coaching and coaching activities the common
experiences do I as a leader behavioral style differ based on challenges of
admire most? based on my similar and performance-, coaching?
■■ What did this DISC1 Leadership different from my development-, ■■ How can I manage
coach do and say Assessment? direct reports? or career-based the emotions
that was effective, ■■ How does my idea ■■ How can I adjust objectives? involved in
and how did he or of an ideal coach or tailor my style ■■ How can I better delivering difficult
she interact with align with my according to the delegate to feedback?
me? leadership style? behavioral styles make projects ■■ What are
■■ What kind of goals ■■ How can I and needs of my a coaching strategies to
did this coach help reevaluate my direct reports? opportunity? address and
me to achieve? definition of an ■■ What are potential ■■ What role does resolve conflict
ideal coach to areas of tension of my coaching style within my team?
better align with my coaching style have in team-
my leadership and the differing based problem
strengths? behavioral styles solving?
and needs of my
direct reports?
■■ What might my
direct reports
need more or less
from me?
Focus on
the “Right”
Coaching Style
+ Focus on the
Most Impactful
Coaching Activities =
Increased
Coaching
Effectiveness
1 DISC refers to dominance, influence, steadiness, and conscientiousness.
CIO Executive Board
Learning and Development Roundtable, “Awareness-to-Action Coaching Toolkit” Profiles of Coaching Programs and Tactics, Corporate Executive Board, Washington DC, 2007, p.29
IT Practice
CIO0361911SYN

CIO Executive Board

IT Practice
CIO0361911SYN
1. E
nable 2. S
bserve and Question
Case Examples

1



Specialists

1 Pseudonym.
CIO Executive Board

IT Practice
CIO0361911SYN
101
Collecting knowledge
worker feedback on
IT PAYS TO LISTEN
information sources and
analytic tools has a 24.8% Maximum Impact on Insight IQ1 of Knowledge Worker Feedback
maximum impact on
Insight IQ.
24.8%
24.8%
16.5%
16.5%
Average for All Knowledge Worker Feedback

Top 10% Drivers on Information Sources and
Analytics Tool Is Collected
CIO Executive Board

IT Practice Approaches to Anthropologically
www.cio.executiveboard.com Observe The IT Skills
Introduction Opportunity Driven Need
Progressive Users Implications
CIO0361911SYN Identification Identification
IT needs to adopt a
proactive collaborative
From LISTENING to OBSERVING
and anthropological
approach to identify Approaches to Identifying Opportunities to Provide Information and Analytic Tools
opportunities to provide
information and analytic
tools. High
■■ Progressive organizations 1
engage knowledge workers

earlier and more frequently
Anthropologically Driven
throughout the project
Need Identification

lifecycle using observation
and experimentation. Use anthropological
observation and
interviewing techniques
“Prosumer” to discover information
Sounding Board needs.
Observe progressive
users to identify
Agile
pain points and
Development
emerging solutions.
Information
Usability Surveys
Requirements
Gathering
Low
Feedback Proactive
Collection Observation
Approach to Observation
1 Pseudonym.
CIO Executive Board

Observe and Question to Identify Opportunities 103

CIO Executive Board

IT Practice
CIO0361911SYN
“Prosumer” Sounding Board
Overview
A methodology to observe knowledge workers (both early adopters and mainstream users) to surface unarticulated
and emerging information needs
EXECUTIVE TEACHING
Segment knowledge workers based on their information usage patterns and their stance toward technology adoption,
then customize engagements to learn from each segment. Observe early adopters as an early-warning mechanism for
emerging needs but balance this with input from the mainstream.
Company Snapshot
Ford Motor Company

Industry: Auto Ford Motor Company, a global automotive industry leader based
Manufacturing in Dearborn, MI, manufactures and distributes automobiles across
2010 Sales: US$129.0 B six continents. The company provides financial services throughout
2010 Employees: 164,000 Ford Motor Credit Company.
CIO Executive Board


Ford’s Digital Worker

Office gathers user
DEVELOP User LISTENING POSTS to track
feedback on a regular
basis to ensure
evolving needs
continued relevance of
collaboration services
and communications.
Target Services Engage

■■ For additional background Across Users Progressive Users
on the Digital Worker Office
see the profile of Ford’s
Digital Productivity Package Key Challenges ■■ User needs vary within roles, making ■■ IT misses the perspectives of the most
in the IT Leader’s Guide to it difficult to scalably engage with tech-savvy users as they often bypass IT
Collaboration at the user base while ensuring services to deploy point solutions.
www.cio.executiveboard. match with needs.
com/Members/
ResearchAndTools/Abstract.
aspx?cid=100159974.
Solutions ■■ Allow users to self-segment based ■■ Ford creates a sounding board of

on preferred engagement and progressive users to understand pain
tech savviness to understand points and pressure-test IT plans.
preferences for collaboration services,
communications, and training.
“We want to make IT

an enabler of user
productivity, rather
than play a ‘control and command’
role.”
Scott Blow
Manager, Digital Worker Office
Ford Motor Company
CIO Executive Board

IT allows users to select
personas based on their
CREATE PERSONAS TO CUSTOMIZE
needs and customizes
engagement accordingly.
ENGAGEMENT
Personas Engagement Strategy
■■ IT creates personas
1 Prosumers
based on user attitude
Willing to assume the risk of using new,
toward technology IT calls on prosumers to pilot new
uncertified, cutting-edge technologies with Early warning of
and workflow needs to a light-touch support model tools and suggest collaboration
emerging needs...
learn about challenges improvements.
Typical Demand Pattern—Employee-owned
and target services and
IT browsers
communications.
2 Senior Leaders
■■ Users can select more than Company officers and others who require high- IT dedicates a communications
one of the five personas end, auditable, discoverable collaboration services liaison and gathers
on the Digital Worker Typical Demand Pattern—Support for off-site recommendations on required
Office site. access to all systems collaboration tools and services.
■■ IT directs users to its site

3 Information Creators
in training sessions and
Contribute information for decision making and IT knowledge management
on the corporate intranet.
need sophisticated knowledge management tools experts provide assistance with
Typical Demand Pattern—Administrative specialized needs such as domain
responsibilities for team websites search.
...is balanced
with insight into
DO provide users with
4 Mobile Workers mainstream
incentives—such as early access
Knowledge workers who work primarily from demand.
to new tools—for providing
outside the office IT connects users with each other
feedback to IT. to share problem-solving tips and
Typical Demand Pattern—High-end access device preferences.
DON’T invest equally in to information from outside the office
gathering input across all user
segments; instead, favor scalable
5 Standard Collaborators
methods such as community Consumers, reviewers, and processors IT provides information about
sites over focus groups for users of information collaboration capability
whose needs are easier improvements.
Typical Demand Pattern—Occasional
to diagnose.
participation in meetings
CIO Executive Board

Create a sounding board

of progressive users to
CO-Opt CRITICS TO Identify Gaps
understand barriers to
end-user productivity Progressive User Sounding Board
and identify innovative
solutions.
Selection Criteria
■■ The Digital Worker Office ■■ Independently solved technology constraints (e.g., obtaining access to enterprise data or e-mail
at Ford engages a subset on personal mobile device)
of the prosumer segment
to surface productivity pain
■■ Comfortable with voicing concerns or dissatisfaction with IT
points and drive innovation. ■■ Understands the economics of niche versus enterprise solutions
■■ Demonstrates the drive to use new tools IT implemented
■■ To prevent technology
evangelism, the facilitator
■■ Has a track record of recommending new technologies to IT
guides users to diagnose the
root cause of the underlying
problem they are trying to
Progressive User Sounding Board
solve with technology.
Description
The sounding board is a global, cross-functional
group of 12 users that meets monthly.
Role
■■ Articulate pain points.
■■ Provide early feedback on IT deployment plans.
■■ Participate in collaboration pilots.
DO engage tech-savvy IT critics Sample Discussion

to identify capability gaps and
push innovation. IT Facilitator Sounding Board Participant
“Where do you see opportunities for improving “We need Device B to access enterprise e-mail
DON’T overengineer productivity?” while traveling.”
conversations with users.
“Will Device A work just as well?” “Device B has better functionality and interface.”
Establish an informal dialogue to
surface concerns about IT and “Device B allows me to carry one device for
understand new technologies “What specific aspects are better?”
corporate and personal use of e-mail and web.”
users are experimenting with.
CIO Executive Board

www.cio.executiveboard.com Observe the IT Skills
ANTHROPOLOGICALLY DRIVEN NEEDs 1
IDENTIFICATION
OVERView
The use of anthropological techniques to discover hidden information needs and opportunities for productivity
improvement, by observing knowledge workers in their own environment and conducting open-ended interviews
Executive Teaching
Traditional approaches to requirements gathering fail when assessing knowledge workers’ information needs because
the needs are fast-changing, diverse, and difficult to articulate. Alpha Company adopts an anthropological approach to
opportunity identification. By observing knowledge workers in diverse settings and conducting open-ended interviews,
Alpha Company gains a deeper understanding of knowledge worker needs and proactively identifies pain points
in end-user workflows.
Component Teachings
■■ Component 1: Direct Observation—Use anthropological observation techniques to surface unarticulated and

context-dependent knowledge worker needs that are missed by more focused requirements-gathering tactics.
■■ Component 2: Interviews and Discourse Analysis—Conduct open-ended interviews with various stakeholders
and analyze end-user stories beyond the immediate language to discover patterns of user experience.
■■ Component 3: Skills Implications for IT—Collectively, these observational and anthropological approaches require IT
to develop a new set of anthropological skills.
Company Snapshot
Alpha Company
2010 Sales: > US$10 B Alpha Company is a leading technology provider.
1 Pseudonym.
CIO Executive Board

While knowledge
workers often lack the
CAN THE KNOWLEDGE WORKER SPEAK? 1
ability to articulate
their information needs,
these needs also differ
across individuals,
organizational cultures,
Needs
and collaborative Identify Opportunities for Productivity Surface Unarticulated Knowledge Workers’
Identification
practices. Improvement Information Needs
Scenarios
Challenges Information needs are context dependent and Knowledge workers lack the ability or inclination
vary across individuals, organizational cultures, to communicate their workflow or information
and collaborative practices. needs.
Illustrative Sales Team Collaboration—Alpha Company Customer Relationship History—By observing

Examples found that the value of a sales team sales teams in two regions, Alpha Company
collaboration tool varied significantly across found that information on relationship history
the year as collaboration declined in Q4. was far more valuable in one region than the
other, due to cultural factors.
Mobile Sales Report Writing—Alpha Company Information Source Diversity—Alpha Company

found that trip reports written on the road were discovered that sales teams with a certain
less detailed, took more time, and had more profile of manager needed access to a broader
errors than those written in the office. variety of information sources when travelling.
“Anthropological
research does not start
from an ideal state to
be achieved. Instead, it provides
input to process redesign based
on actual conditions on the
ground.”
Corporate Anthropologist
Alpha Company
1 Pseudonym.
CIO Executive Board
Traditional process-
centric workflow
FROM WORKFLOW MAPPING TO ANTHROPOLOGY 1
mapping techniques fail

to surface knowledge
workers’ actual needs
and behaviors. Traditional Workflow Mapping Approach Alpha Company’s Anthropological Approach
■■ Assumes there is only one best known ■■ Assumes there may be several context-
■■ Most requirements-gathering
method of working specific and equally valid ways of working
techniques assume that
there is only one right way ■■ Uses standardized process discipline with ■■ Uses observational techniques to identify
to do something, and that pre-set rules for mapping workflows roadblocks in day-to-day workflows and
this is the best solution de-familiarization to mitigate biases
for all knowledge workers ■■ Solutions for knowledge worker needs
in all contexts. aligned to stated needs and a predefined ■■ Solutions based on unarticulated
business case or unstated knowledge worker needs
When to Use an Anthropological Approach
Workflows are intuitively executed

and the result of many ad hoc workarounds.
Knowledge worker needs and workflows are

context specific and difficult to articulate.
“While end-user The goal is to enable greater productivity

surveys show you what and insight by understanding how
knowledge workers say knowledge workers work.
they do, anthropological research
tells you what knowledge workers
actually do.”
Corporate Anthropologist
Alpha Company
1 Pseudonym.
CIO Executive Board

Use anthropological
observation and
OBSERVE AND ASK ASSUMPTION-FREE 1
interviewing techniques
to surface unarticulated
QUESTIONS
and context-dependent
knowledge worker needs. Overview of Common Anthropological Techniques
■■ Gain insights into knowledge Scenario: Help sales reps frame needs in terms of their
worker requirements underlying challenges in accessing information, rather than
through direct observation the technologies they need.
in different user settings,
organizational contexts,
and across points of time.
■■ While end-user surveys

follow a predetermined
Anthropological Observation Techniques Open-Ended Interviewing Techniques
trajectory with little room
for maneuvering, open- 1. Contextualized Observation 1. A
cknowledge Expertise
ended interviews follow Observe sales teams in their natural environments, Neither pretend to know more about the subject
a conversational model such as customer visits or in sales meetings. Look matter than the sales team does, nor buy in to easy
that allows for immediate across internal divisions and at organizational and answers, but always ask follow-up questions.
feedback. cultural contexts.
2. Observation Across Space and Time 2. Reiterate and Qualify
Observe sales teams across multiple spaces and Repeat what is heard to clarify meaning, but also
points of time, looking for differences that may ask whether these are the right questions to ask.
relate to location or time of day.
3. S
elf-Reporting 3. Read the Silence
DO engage knowledge workers Ask sales reps to use written or audio-visual diaries Look for ideas and actions that are missing from
with an open mind, without a to record their workflow and log stage gates that the sales reps’ narrative, and identify variability and
desired business goal, process, cause bottlenecks or stalls. consistency across knowledge worker experiences.
or workflow in mind. 4. Auto-Reporting
4. Ignore Nothing
Record the use of collaboration and analytic tools,
In the context of unarticulated needs, anything
DON’T assume that one process using real-time monitoring to measure response times
may turn out to be significant.
is superior or that knowledge and gain deeper insights into sales reps workflows.
workers use the same workflow
in all situations.
1 Pseudonym.
CIO Executive Board
Screen for attributes such
as intellectual curiosity
CAPTURE THE CURIOUS
and critical thinking to
develop a set of core Overview of Anthropological Skills
anthropological skills
within IT.
Technical Anthropological Skills
■■ While trained Skills Implications of Anthropological

Skill Implications of Open-Ended Interviews
anthropologists are best Observation
qualified to conduct
anthropological research, General Skills ■■ Critical thinking and intellectual curiosity ■■ Critical thinking and intellectual curiosity
IT organizations should first ■■ Social agility and sensitivity ■■ Empathy and open-mindedness
identify employees with
intellectual curiosity and ■■ Contextual understanding ■■ Contextual understanding
critical thinking skills.
Specific Skills ■■ Primary research and fieldwork methodologies ■■ Discourse analysis and intertextuality
■■ Use behavioral interviewing ■■ Identifying patterns of behavior ■■ Interviewing skills with high levels
techniques to locate and of reflexivity
■■ Understanding the evolutionary and material
identify intellectual curiosity
context of end-user needs ■■ Gathering, integrating, synthesizing,
and observational and
and analyzing data
problem solving skills during ■■ Storyboarding
the interview process. ■■ Storyboarding
Where to Find These Skills Where to Find These Skills
■■ Anthropology Graduates ■■ Anthropology Graduates
■■ Human Factors Engineers ■■ Market Researchers
■■ User Interface Designers ■■ Investigative Journalists
■■ Interaction Designers ■■ Information Scientists
CIO Executive Board


IT organizations can
embed anthropological
THE ANTHROPOLOGIST INSIDE US
skills in several existing
or emerging roles.
■■ The ongoing focus on Criteria for Locating Anthropological Skills

and user-behavior analysis
■■ Has domain knowledge from alignment with business units
will see the emergence ■■ Has time outside specific projects
of IT roles that require ■■ Has seniority, credibility, and business engagement skills
capturing knowledge worker
requirements for effective ■■ Links back to project, service management, or architecture teams
decision making.
Alpha Company’s Approach Alternative Approaches
Dedicated Corporate Embed Anthropological Skills Embed Anthropological Skills

Anthropologist Role in Existing Roles in New-to-World Roles
■■ For large organizations Collaboration and Social
Business Analyst
Media Evangelist
■■ For organizations
with people-intensive
Business Liaison Information Insight Enabler
processes (e.g., Sales,
Prof. Services, R&D)
Service Manager User-Experience Designer
■■ When project warrants
For full descriptions of the dedicated resources
responsibilities and skill Solution Architect
requirements for these roles, please
see the CIO Executive Board’s IT
Job Descriptions Database.
CIO Executive Board

KEY TAKEAWAYS: Observe and Question to Identify Opportunities
Use Observations of Knowledge Workers in Addition to Observations from Knowledge Workers: Adopt an anthropological approach to identify opportunities
and deliver analytic capabilities successfully. Identify and learn from early adopters, observe segments of knowledge workers across the organization and in
diverse user settings, and conduct open-ended interviews to identify bottlenecks and pain points.
1. IT is proactive in observing knowledge worker information needs. IT listens to knowledge workers only when gathering requirements.
2. IT segments knowledge workers by their attitude toward technology

IT segments knowledge workers solely by role or seniority.
and information usage patterns.
3. IT learns from early adopters about new analytic functionality. IT restricts early adoption of new functionality.
4. IT observes knowledge workers in diverse user settings, locations, IT gathers requirements for information and analytic tools primarily
and points in time. through business process mapping.
5. IT conducts open-ended interviews to capture the voice of the

IT communicates with knowledge workers through end-user surveys.
knowledge worker.
CIO Executive Board

IT Practice
CIO0361911SYN

1. E
nable 2. S
bserve and Question
Case Examples

1



Specialists

1 Pseudonym.
CIO Executive Board

IT Practice
CIO0361911SYN
CIOs face a range of
options for central or
Information Management—How to Organize,
local management of
information and for
Where to Locate?
management within IT
or outside.
Centralized
Diffused (Integrated
(Dedicated)
with Other Activities)
Managing information
Information should be
centrally avoids
managed close to the
duplication, promotes
people who use it to
integration, and
allow for speed and
provides a single view
flexibility.
of the enterprise.
Key Questions
1. What is the right
Business Ownership
degree of centralization
Only information
for information
producers and users Business/ Business/ management and
understand its meaning Centralized Diffused analytics?
and value and only they
can improve quality. 2. How should
responsibility be
divided between
business areas and IT?
IT Ownership
IT manages the IT/ IT/
information sources Centralized Diffused
and can offer a neutral
perspective.
CIO Executive Board

IT Practice
CIO0361911SYN
Introduction Information Management COEs Skills Implications
Appendix 117
Foundational analytic and

CENTRALIZE Management, NOT INFORMATION
activities benefit from
centralization and create Maximum Impact on Insight IQ1 of Centralized Models for Information Management
sufficiently strong
oversight to sustain 13.0%
decentralized information
sources.
10.5% 10.4%
6.9% 6.7%
0.0% 0.0%
Centralized Centralized Centralized Centralized Centralized Single Single Data
Analytics Knowledge Information User Support Information Unstructured Warehouse
Team Worker Quality for Analytic Architecture Information
Training Tools Repository
Organizations with a high insight IQ …which enables them

centralize information management to keep the information
and support activities… itself decentralized.

1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge
worker scores relatively “high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of
each driver is modeled using a variety of multivariate regressions with controls.
Source: Corporate Executive Board IT Practice, Insight IQ Diagnostic, 2011.

CIO Executive Board
IT Practice
CIO0361911SYN
There is no one ideal
model for an analytics
Where Should An Analytics COE Reside?
center of excellence as
the optimal location Factors Determining Organizational Location of Analytics Center of Excellence (COE)
depends on a number
of organizational factors.
1. Location of IT is the main source Other functions/business

Analytics Expertise of analytics expertise. units have analytics expertise.
2. Organizational IT is the largest central Other central functions are

Diversity function. well resourced or influential.
3. Nature of Information is used to Information is used to price

Analytics Use produce the product. or sell the product.
4. Information Information management Information management

Management Focus is focused on integration is focused on knowledge
and quality. worker capability.
I. Analytics Coe II. Paired IT and III. Analytics COE

Within It Business COEs Outside IT
CIO Executive Board

IT Practice
CIO0361911SYN
Appendix 119
While many activities

migrate between IT
DIVISION OF LABOR BETWEEN IT AND THE REST OF THE
and a COE outside IT,
information stewardship
BUSINESS
remains primarily a
business responsibility
in all models while source
system management Activity I. A
nalytics COE II. P
aired IT and III. Analytics COE
always remains in IT. Within IT Business COEs Outside IT
Source Systems Management
Information Architecture
Data Warehousing
Strategy and Prioritization
External Information Acquisition
Reporting
Knowledge Worker Training
Analytics
Information Stewardship
Examples
IT–Owned Activities
Owned Elsewhere in the Corporate Center
CIO Executive Board

IT Practice
www.cio.executiveboard.com Introduction Information Management COEs Skills Implications
CIO0361911SYN
and analytics activities
I. Analytics COE WITHIN IT
are centralized within IT.
Information Management and Analytics Organization Structure
A. E
ach major business unit
is represented on the Data
Governance Council; the CIO
council was set up as a
business council, and IT’s
participation on the council
A
is purely as one of the Enterprise
represented business units. Data Governance Business Factory
Business
Council Applications Systems
Intelligence (EBI)1
B. EDW and BI user support
team addresses questions
related to both business EBI Asia
intelligence tools and content. EDW Content and Business
Support and
Data Governance Intelligence
Development
C. B
I solutions team develops
reports and tools for users; as C
EDW
there is a strong focus on user Asia Business
Content and BI Solutions
self-service, the team typically Intelligence
Architecture
builds templates for users to
adapt rather than providing D D
ongoing maintenance and Real-Time and Real-Time and
PMO and Data BI Strategy and Asia EDW
support. Application- Application-
Modeling Architecture Content
Specific Specific
D. A
pplication-specific and Reporting Reporting
B
factory-related reporting is
run outside the EBI group to EDW and BI Advanced
cater to special requirements User Support Analytics
of these groups.
Reports to IT
1 Quality and engineering applications support teams are not shown on this chart but they also report into the executive director,
Enterprise Business Intelligence.
CIO Executive Board
IT Practice
CIO0361911SYN
Appendix 121
and analytics
II. PAIRED IT AND BUSINESS COEs
responsibilities are
divided with a business Information Management and Analytics Organization Structure
COE acting as an
interface between IT
and the rest of business. Analytics Resource Center (ARC) Data Analysis COE
Program Executive/ BI Delivery and Data/Information

A. K
ey mandates for the Group Analytics Lead Adoption Team Analysis Lead
Analytics Resource Center
(ARC):
ARC Prioritization
Analytics
–– Represent business and Data Governance
Advisory Board
partners in the Processes
Program
development of BI and
Management
provide testing and B B
Office
training, Business Change
–– Support analysts across SMEs Agents
the organization in non-
actuarial analytics, and
A Technical Lead C
–– Lead data governance
efforts across the
organization. Data
Business
Analytics Governance
Intelligence Data Data
Strategy Data Delivery
B. ARC
is supported by business Integration Analytics
subject matter experts and
Data Policy
change agents from the lines Reporting
Analysis Committee
of business. Data
Analysis
Process
Support
C. D
ata Analysis COE reports Team
Metadata/
to IT and derives data Governance
requirements and data Aligned Aligned by
delivery priorities from ARC’s by BUs Subject Areas
prioritization and governance
processes. Business-Led Function Reports to IT
That Reports to the Actuarial Group
CIO Executive Board
IT Practice
CIO0361911SYN
and analytics activities
III. Analytics COE Outside IT
are centralized under
a corporate function, Information Management and Analytics Organization Structure
shifting most of
traditional information
management activities
Executive VP
out of IT.
and Strategy Officer
Enterprise Information
■■ At point A, touchpoints
Committee
with IT are maintained
through representation in Analytic and Information
committees and through Information Management
Management
regular meetings with the IT and Analytic Services
Advisory Committee
staff and CIO.
Analytics
■■ At point B, a few technical Center of Analytic
information management– Excellence Analytic Continuous
Services and A
related roles such as data Delivery Improvement
Solutions
integration remain outside
B
of the COE; information Information
architecture responsibilities Informatics Analytic Strategy and
are represented on both Solutions Services Architecture
sides.
B
Solutions Informatics Portfolio and Data Integration
■■ BCBSNC’s COE is being Informatics
Portfolio Solutions Process and Production
built iteratively, first by Analysts
Analyst Analysts Analyst(s) Support
including the transactional
activities (e.g., data Informatics
Support
sourcing, reporting, Solutions
training) and some critical Developers
pure analytics functions Technology
(e.g., group analytics, care Architect
management).
Corporate Function That Reports to the Strategy Group
CIO Executive Board

IT Practice
CIO0361911SYN
Appendix 123
An analytics COE within

IT requires new skills
Skill IMPLICATIONS OF Each COE MODEL
such as information
visualization, analytics
service management,
Skills I. A
nalytics COE II. P
aired IT and III. Analytics COE
and knowledge worker
Within IT Business COEs Outside IT
training and coaching.
Database Design and Mgmt.
Info. Systems Design
Info. Architecture
Info. Integration
Business Analysis
Info. Visualization
Analytic Service Mgmt.
Strategy and Portfolio Mgmt.
Info. Policy Formulation
Analysis/Insight Generation
Training and Coaching
IT Skills That Rise in Importance

Business Skills That Rise in Importance
For a description of the key information management and analytics skills, please see page 125.
CIO Executive Board

IT Practice
CIO0361911SYN
Implementation Guide: SKILLS Descriptions
Analytics Service The skills required manage analytics as an end-to-end service from service strategy to service delivery
Management and improvement
Database Design The skills required to produce a detailed data model for a database
Database Management The skills required to create, access, and maintain a database
Information System Design The skills required to analyze and design information systems, concentrating on entities and their attributes and
interrelationships; also includes data modeling for individual databases and the corporate data model, as well as coordinating
the definition of data across multiple distinct databases
Information Architecture
The skills required to define how information and information systems support the objectives of the business
and Development
Information Integration The ability to integrate information and content from multiple sources and channels
Business Analysis The skills required to identify business needs and determine solutions to business problems
Information Visualization The ability to depict information in intuitive and insightful ways
Information Policy
The skills required to establish guidelines for information dissemination and usage
Formulation
Information Analysis and

The ability to understand the meaning of structured and unstructured business information and generate insight from it
Insight Generation
Note: For detailed analysis on the emerging skill gaps and role changes for IT, please ask the CIO Executive Board staff or your account manager for a copy of our recent study on The IT Talent
Implications of the Future of Corporate IT. You can also download this study from www.cio.executiveboard.com.
CIO Executive Board

IT Practice
CIO0361911SYN
Appendix 125
List of Questions Asked in
the Insight IQ Diagnostic
CIO Executive Board

IT Practice
CIO0361911SYN
126
I. Information Management Structures and Processes
1. Has your organization introduced new or enhanced separately. –– Information ownership

information management or analytics capabilities in the past –– Information and analytics are managed separately, but the –– Information architecture
18 months? owners of each are aware of the other’s processes and –– Information quality
accommodate them as appropriate.
2. How are individuals/teams dedicated to analytics structured 13. How are individuals/teams dedicated to analytics structured
–– Processes for managing information and analytics are
within the organization? within the organization?
partially integrated.
–– Centralized companywide
–– Processes for managing information and analytics are fully 14. If your organization has a central analytics group or Center
–– Partially centralized by business unit
integrated throughout the lifecycle. of Excellence, what are its responsibilities?
–– Partially centralized by function
–– Coordinate analytics initiatives.
–– Decentralized—most teams/subfunctions have their own 9. Which best describes the structure of your organization’s
–– Share analytics best practices and provide training.
analytic experts information management group?
–– Provide analytical expertise/capabilities.
–– Formal—within IT (with dedicated staff)
3. If your organization has a central analytics group or Center –– Deploy analysts where needed.
–– Informal—within IT (i.e., a couple of individuals are
of Excellence, where does it reside? –– Develop analytics strategic plan for the organization/
responsible but is not their full-time job)
–– IT functions.
–– Formal—within Shared Services
–– Business—Shared Services –– Provide guidance/rollout of analytical tools/technologies.
–– Informal—within Shared Services
–– Business—office of CEO or President –– N/A
–– Formal—part of broader Analytics Group
–– Marketing
–– Informal—part of broader Analytics Group 15. If your organization has a central analytics group or Center
–– Finance
–– Formal—within individual BUs/functions of Excellence, where does it reside?
–– Other
–– Informal—within individual BUs/functions –– IT—Reports directly to CIO
4. Which of the following capabilities are in place to improve –– Exists, but is integrated into day-to-day tasks of broader –– IT—Applications
decision making within your organization? IT –– IT—Infrastructure/Operations
–– Automated decisions –– Doesn’t exist –– IT—Information Management
–– Collective intelligence tools –– IT—Other, please indicate
10. Who owns each of the following information management
–– Decision making team structure based around diversity –– Business—Shared Services
responsibilities?
and independence –– Business—Office of CEO or President
–– Information architecture
–– Formal decision frameworks or guidelines –– Marketing
–– Information lifecycle management
–– Other –– Finance
–– Information quality and standards
–– Other, please indicate
5. Is information primarily pushed to you or pulled by you? –– Knowledge and content management
–– N/A
6. Do rules exist around which information can be shared –– Analytics tools
–– Enterprise data warehouse 16. Does your organization use an external service provider for
across the organization?
analytics (excluding cloud providers)?
7. Has your organization undertaken the following in the past 11. If you have an Information Management group, what is the
perceived clarity of its roles and responsibilities? 17. If you do use an external service provider for analytics,
12 months.
which of the following is the external service provider
–– Adding new information sources to the data warehouse 12. Indicate the degree to which the following are centralized.
responsible for?
–– Enhancing existing data source feeds –– Data warehouse
–– Information architecture
–– Updating or modifying business rules used in standard –– Analytical tools
–– Information lifecycle management
reports –– Unstructured information repositories
–– Information quality and standards
–– None –– Knowledge worker training in information use and
–– Knowledge and content management
8. Which best describes your organization’s approach to analytics
–– Analytic tools and capabilities
managing information and analytics? –– User support for analytical tools
–– Enterprise data warehouse
–– Information and analytics are managed completely
CIO Executive Board

IT Practice
CIO0361911SYN
List of Questions Asked in the Insight IQ Diagnostic 127

I. Information Management Structures and Processes (Continued) II. Training and Support
18. Do you use cloud applications for generating information? maintained in pockets of solutions or documents, typically 1. Is there an analyst available to you to help you find
at a project level. information or conduct analysis?
19. Do you use cloud applications for storing information?
–– Consolidation of some business definitions or technical –– There is an analyst embedded directly in my team.
20. Do you use cloud applications for analyzing information? information occurs, but governance processes are –– There is a team of analysts in my business unit or function.
21. What capabilities are in place at your organization to ease immature. –– There is a companywide central team of analysts.
the capture of unstructured information in shared systems? . –– Business and IT accountabilities and processes are –– There are informal analytical experts on my team.
–– Automated processes (e.g., automated tagging, document starting to be formed to manage repositories of metadata. –– There is no one available to me for help in finding
uploads) –– Business and IT are utilizing common metadata across information or conducting analysis.
–– Rules and guidelines on what information to capture and departments and projects to improve understanding.
2. Does your organization provide training on finding
tag –– Formal processes and accountabilities are in place for
information or conducting analysis?
–– Rules and guidelines on how to capture information and managing official repositories of metadata utilized for
guiding information use. 3. If you attended training provided by your organization on
tag
finding information or conducting analysis, how effective or
–– Dedicated team or individual responsible for capturing, 25. Do rules exist around types of information that could
ineffective was that training in the following areas?
tagging and transferring information into shared systems be shared between levels and/or different parts of the
–– Finding information I need
–– Other organization?
–– Information definitions and terminology
–– None
26. How would you assess your organization’s current level –– Conducting analysis
22. For which of the following does your organization have of proficiency in the following Enterprise Architecture –– Communicating and displaying information
defined processes in place? disciplines? –– Using the functionality of the tools provided
–– Create/Acquire –– Business Architecture –– Where/how to get help
–– Store/Share –– Information Architecture
4. How formalized is the process for making decisions in your
–– Find/Use –– Applications Architecture
organization?
–– Archive/Retain –– Technical Architecture
–– Non-existent: No decision making processes exist.
–– Dispose
–– Ad hoc: Individuals and/or teams have processes that are
–– Manage/Control
followed irregularly.
–– None
–– Undocumented: Individuals and/or teams follow processes
23. How do you define metadata in your data warehouse? that are not standardized nor documented.
–– For all information types –– Defined: Processes used across business units and/or
–– For information that is shared between functions functions are documented but not enforced.
–– For information that is shared between business units –– Managed: Processes are documented and enforced.
–– For information that is deemed critical to the business –– Optimized: Processes are documented, enforced, and
–– For information that is used most frequently measured to drive improvement.
–– For information that is stored in multiple systems or
5. Which of the following are in place to improve decision
repositories
making within your organization?
–– For information where there is frequent confusion
–– Automated decisions
regarding the definition
–– Collective intelligence tools
–– For unstructured information
–– Decision making team structure based around diversity
–– For no types of information
and independence
24. Which best summarizes your current level of metadata –– Formal decision frameworks or guidelines
management maturity? –– Other
–– Incomplete business and technical metadata are –– None
CIO Executive Board

IT Practice
CIO0361911SYN
III. Knowledge Worker Attitudes and Incentives
1. Are you evaluated in your performance review on your ability 11. How significant a contributor to your personal analysis 15. How important is information and/or analytics in enabling
to conduct analysis effectively? and decision making are each of the following types of you to do the following?
information? –– Make Better Decisions
2. To what extent do you agree with the following: I enjoy
–– Centrally provided information available to and used –– Make Faster Decisions
conducting analysis?
across the organization –– Improve Your Job Performance
3. To what extent do you agree with the following: If I conduct –– Centrally provided information available to and used
16. How frequently do you regret decisions you’ve made where
analysis and disagree with the findings I will rely on my own across my business unit/function
information/analytics contributed to the decision?
judgment instead? –– Information generated by and managed by my
subfunction or team 17. To what extent do you agree with the following: All of the
4. To what extent do you agree with the following: I have
–– Information I personally obtain through external sources information I need to do my job is available to me?
enough time to conduct analysis that is valuable to me in
making decisions in my daily work? 12. Which characteristics do you value most when using 18. To what extent do you agree with the following: In my
information and analytics in your daily work? daily work, I rely heavily on the results of analysis (charts,
5. To what extent do you agree with the following: Expertise
–– Information Accuracy graphics, reports, dashboards, etc.) to support my business
in analytics increases my ability to advance in my desired
–– Information Timeliness decisions, processes and deliverables?
career path?
–– Information Consistency
19. To what extent do you agree with the following: The reports
6. To what extent do you agree with the following: I feel very –– Information Relevance
and dashboards that are pushed to me are very valuable and
comfortable conducting analysis and am able to meet all my
13. How important are the following capabilities when you I read, analyze and use them extensively?
information and analytical needs without external help?
make decisions?
20. To what extent do you agree with the following: The reports
7. When making important decisions in your organization, how –– Visualization—The graphical or visual presentation of
and dashboards that I pulled are very valuable and I read,
critical are each of the following considerations to you, your information (e.g., through static or interactive graphs,
analyze and use them extensively?
direct manager, and your organization’s leadership? charts, maps, images) to make it easier for the users to
–– Information and Analytics understand the information contents and attributes 21. If information you would like to have as an input to making
–– Intuition –– Analysis—The process of examining information with the a decision is not available in a standard report or dashboard,
–– Precedent purpose of drawing conclusions about that information how willing are you to go look for the information?
–– External Advice –– Filtering—The process by which redundant or unwanted
8. How effective is your direct manager in using analytics in information from an information stream is removed and
carrying out his/her job responsibilities? only information that meets specific criteria (e.g., time,
value, user groups) is displayed or presented
9. To what extent do you agree with the following: My –– Aggregation—The process by which information is
performance review is based largely on metrics and gathered and expressed in a summary form, typically
achievement of measurable goals and objectives? combining two or more information attributes or
10. To what extent do you agree with the following: Performance occurrences
metrics and progress against my business unit/function’s 14. How effective a contributor is information and/or analytics
goals and objectives are reported to me on a regular basis? to your organization’s ability to do the following activities?
–– Product or service development
–– Improving product or service profitability
–– Evaluating and prioritizing investment proposals
–– Developing a corporate or business unit strategy
–– New market identification and market strategy
development
–– Identifying and targeting customer segments
–– Forecasting demand for products, services, or resources
CIO Executive Board

IT Practice
CIO0361911SYN

IV. Investment and Resource Level V. Quality of Information
1. What is your organization’s total budget for the following 1. To what extent do you agree with the following: –– There are no means to assess or verify information quality
projects/initiatives/capabilities in 2011? Include all spend “The information from corporate sources is in a usable –– Other
both inside and outside of IT. format and does not require modification or manipulation?” 9. For which of the following has your organization established
–– Business Intelligence standard definitions?
2. To what extent do you agree with the following:
–– Information Management –– Information that is shared between functions
“I have a good understanding of the definitions of the
–– Collaboration –– Information that is shared between business units
information from corporate sources?”
2. At which groups are your analytics investments and –– Information that is deemed critical to the business
3. To what extent do you agree with the following statement: –– Information that is used most frequently
initiatives targeted?
“There is a single version of the truth across information –– Information that is stored in multiple systems or
3. How many staff are part of the information management from all corporate sources?” repositories
group? –– Information where there is frequent confusion regarding
4. To what extent do you agree with the following:
4. Rank the following four capabilities in terms of effort and “The information from corporate sources is directly relevant the definition
investment. to business decisions, processes, and deliverables in my daily 10. Is each of the following information types defined in a data
–– Visualization work?” dictionary?
–– Analysis
5. To what extent do you agree with the following: 11. What steps does IT take to remedy information quality
–– Filtering
“The information from corporate sources is always timely issues (such as duplicates, errors, outliers, contradictions,
–– Aggregation
and up-to-date enough to be useful in my daily work?” inconsistencies, missing values) in relation to the information
5. How are analytics projects and initiatives approved? 6. To what extent do you agree with the following: that is entered or maintained in the data warehouse?
–– Very formal business case review “The information from corporate sources is always precise –– There is an information quality framework that establishes
–– Somewhat formal business case review and accurate enough to be useful in my daily work?” quality standards and guidelines for information to be
–– No formal business case review added or kept in the data warehouse or data marts.
–– Piggyback on other IT/business initiatives 7. What level of confidence do you have in the quality
–– The information is fixed in the source systems before
–– Other (cleanliness and accuracy) of information from each of the
entering it into the data warehouse or data marts.
following?
6. Which of the following investment approval processes do –– The information is fixed both in the source systems and
–– Human Resources
you use for analytics investments and initiatives? during information movement.
–– Finance
–– A business-enabling capital investment –– The information is fixed in the data warehouse or data
–– Legal and Compliance
–– An IT infrastructure investment mart with the fixes fed back to the source systems.
–– Marketing
–– An R&D investment –– The quality of information in our data warehouse is
–– Sales
reviewed or audited on an ongoing basis.
7. How are information management projects and initiatives –– Customer Service
–– The quality of information in our data warehouse or data
funded? –– Supply Chain and Operations
mart is reviewed or improved as and when business users
–– As a business-enabling capital investment –– Research and Development
raise quality concerns.
–– As an IT infrastructure investment 8. For functions where you have low or no confidence in the –– The information quality process is integrated into the ETL
–– As an R&D investment quality of their information, what are the reasons behind process to maintain information quality thresholds.
8. How are information management projects and initiatives your lack of confidence?
12. To what degree are information quality governance and
approved? –– Lack of transparency in how information was collected
standards automated through technology?
–– Very formal business case review –– Lack of transparency in how information was stored
–– Somewhat formal business case review –– Lack of transparency in how information was analyzed 13. Which of the following describes the information quality
–– No formal business case review –– Multiple versions of the same information exist standards within your data warehouse or data marts?
–– Piggyback on other IT/business initiatives –– Lack of trust in systems –– All information has the same standards for information
–– Other, please indicate –– Lack of common vocabulary/standard definitions quality.
CIO Executive Board

IT Practice
CIO0361911SYN
V. Quality of Information (Continued) VI. Information Usage Patterns
–– Information for certain functions (e.g., Finance, HR, supply 16. Who in your organization is primarily responsible 1. How frequently do you use each of the following?
chain) has higher standards for information quality. for enforcing information quality standards and policies? –– Workforce Information
–– Information that is used more frequently has higher –– Business leaders –– Financial Information
standards for information quality. –– Subject matter experts in the business –– Risk Information
–– Information that has been deemed to be more valuable to –– IT –– Customer Information
the business has higher standards for information quality. –– Information Management group –– Product, Materials, and Suppliers Information
–– Information that will be used for certain types of analysis –– Analytics group –– Research, Product Development and Testing Information
has higher standards for information quality. –– Shared services group –– Supply Chain Information
–– Information that will be used by more senior or executive –– No one –– Demand Information
staff has higher standards for information quality. –– Web Information
17. On which of the following characteristics do you tag
–– Information from certain corporate systems has higher –– Consumer and Marketing Information
information in your data warehouse or data marts?
standards for information quality. –– Customer Service Information
–– Individual author of the information
–– Sales Operations Information
14. Which of the following describes the information timeliness –– Function that created the information
–– Economic and Industry Information
standards within your data warehouse or data marts? . –– Date and time the information was created
–– Financial Market and Trading Information
–– All information has the same standards for information –– Cleanliness and timeliness standards for the information
–– Assets and Maintenance Information
quality. –– Relevant governance processes
–– Business Process Efficiency, Service Levels, and Quality
–– Information for certain functions (e.g., Finance, HR, supply –– Frequency information is used
Information
chain) has higher standards for information quality. –– Purpose of the information
–– Information that is used more frequently has higher –– Types of analysis information is used for 2. Which of information types are most important to you in
standards for information quality. –– Types of decisions information is used to make your daily work?
–– Information that has been deemed to be more valuable to –– Standards reports or dashboards information is used in
3. Is your team or function responsible for owning and/
the business has higher standards for information quality. –– Other
or generating a significant proportion of the following
–– Information that will be used for certain types of analysis –– None
information types?
has higher standards for information quality.
–– Information that will be used by more senior or executive 4. How frequently do you consult each of the following in your
staff has higher standards for information quality. day-to-day work?
–– Information from certain corporate systems has higher –– Operational Structured Information
standards for information quality. –– Analytical Information
–– Unstructured Information
15. Who in your organization is primarily responsible for
–– Undocumented information from colleagues or experts
defining information quality standards and policies?
–– External company-endorsed sources
–– Business leaders
–– External non-company endorsed sources
–– Subject matter experts in the business
–– IT 5. How many different sources of information/analysis do you
–– Information Management group use on a weekly or more frequent basis?
–– Analytics group –– Corporate operational systems
–– Shared services group –– Corporate unstructured information systems and
–– No one document repositories, excluding e-mail
–– Reports, dashboards, data/information warehouses, or
data marts On average, how much of your time at work
do you spend finding information or conducting analysis?
CIO Executive Board

IT Practice
CIO0361911SYN

VI. Information Usage Patterns (Continued) VII. Staff Knowledge and Competencies
6. On average, how much of your time at work do you spend –– Significantly higher value due to speed/convenience 1. Is conducting analysis explicitly mentioned in your job
finding information or conducting analysis? of mobile device accessibility description?
–– Somewhat higher value
7. Of the time you spend finding information and conducting 2. Which of the following skills do you most value in staff
–– Same value
analysis, what percentage is spent finding information? responsible for conducting and providing analysis to you and
–– Lower value, as some information is missing
your team?
8. How frequently do you undertake each of the following –– Significantly lower value, as key information is missing
–– Business knowledge and understanding
types? –– N/A—I do not have mobile access.
–– Quantitative skills
–– Descriptive Analytics—The process of using analytics to
15. Do you analyze information obtained from any of the –– Technical skills
describe what happened in the business (using reports
following sources? –– Relationship and advisory skills
and dashboards)
–– Smart Networks and Sensors –– Training and coaching skills
–– Predictive Analytics—The process of using analytics to
–– Radio-Frequency Identification
predict what will happen in the business (using data 3. How frequently do you interact with staff from each of the
–– Global Positioning System
mining, pattern recognition and alerts, Monte-Carlo following functions for work-related purposes?
–– Collaboration Tools
simulation, forecasting, predictive modeling) –– Human Resources
–– External Social Media
–– Finance
9. How much time do you spend obtaining information or
16. Do you provide information to corporate systems for others’ –– Legal and Compliance
conducting analysis that does not ultimately lead to a
consumption (e.g., inputting information into a CRM system –– Marketing
decision or productively inform your or your colleagues’
or sharing information/knowledge within a collaboration –– Sales
work?
system or document repository)? –– Customer Service
10. To what extent do you agree with the following: When I need –– Supply Chain and Operations
17. Do you have the opportunity to provide feedback on
specific information, I know where to find it? –– Research and Development
information sources and business intelligence/analytics tools
11. To what extent do you agree with the following: In instances for improvement or functionality enhancements? 4. To what degree are your decisions made independently?
where someone in my organization has conducted similar –– I make the majority of decisions completely independently.
18. How does your organization determine end-users’ needs for
analysis before, I am able to find documentation, lessons –– I take my peers’ perspectives into account but ultimately
information with regard to tools, capabilities, quality, etc.?
learned, or notes on their process and experiences? make decisions independently.
–– Link information needs to strategic decisions.
–– I make decisions in conjunction with peers to incorporate
12. To what extent do you agree with the following: For –– Derive information needs from strategic goals or targets/
the group’s perspectives.
projects/tasks where someone in my organization has KPIs.
–– Decisions are largely dictated by my manager or other
worked on something similar before, I am able to find –– Define information needs based on workflow processes.
leaders.
documentation, lessons learned, or notes on their process –– Keep up to date of market/industry/functional trends.
and experiences? –– Directly ask knowledge workers via survey. 5. Which of the following best describes the methods you use
–– Directly ask knowledge workers via focus groups. to conduct analysis?
13. Which of the following formats/channels do you use to
–– Track information requests from knowledge workers to –– I develop specific theories that I then test using analysis.
access each information type?
understand trends in information needs. –– I have some theories in mind that I test using analysis, but
–– PC-based web browser
–– Take in feedback from knowledge workers in terms of their also manipulate the information in other ways to see if
–– Mobile devices
satisfaction with information (to inform future needs). interesting trends or insights emerge.
–– E-mail
–– I manipulate the information without a clear idea of what
–– Online system
I expect to find, and see if interesting trends or insights
–– Paper
emerge.
14. If you access information from mobile devices, how do you
rate their value and functionality relative to when accessing
them on a standard desktop or laptop?
CIO Executive Board

IT Practice
CIO0361911SYN
VIII. Information Strategy and Governance
1. Does your organization have a data steward role? 7. Which components and initiatives are included in your 11. Rank order the business motivations for making analytics
strategic plan for analytics and information management? investments .
2. If you responded “Yes, formal data steward role,” how many
–– Formalizing the structure, roles, and responsibilities for –– Product or service development
data stewards does your organization have?
analytics –– Improving product or service profitability
3. Indicate if the following are available to knowledge workers –– Implementing information architecture requirements –– Evaluating and prioritizing investment proposals
through self service. –– Implementing information governance requirements –– Developing a corporate or business unit strategy
–– Data definition –– Implementing information quality standards –– New market identification and market strategy
–– Name of the data steward –– Implementing new analytical tools or capabilities development
–– Source of the data –– Conducting training on using analytical tools –– Identifying and targeting customer segments
–– Details of transformations performed on the data –– Developing knowledge worker analytical skills –– Forecasting demand for products, services, or resources
–– None –– Developing an analytical culture –– Other
4. To what degree do information security and access policies –– Measuring analytical performance
12. Does your Enterprise Architecture group have information
inhibit your ability to obtain information that is valuable/ 8. Are your analytics and information management initiatives architects?
important to your analysis or decision making? aligned to business strategic goals?
13. How many information architects does your EA group have?
5. How mature are business processes within your function/ 9. Which groups are involved in developing the strategic plan
14. How would you assess your organization’s current level of
business unit? for analytics and information management?
proficiency in the following disciplines?
–– Non-existent: No business processes exist. –– IT
–– Data Strategy and Governance
–– Ad hoc: Individuals and/or teams have processes that are –– Finance
–– Data Stewardship
followed irregularly. –– Marketing
–– Data Standardization
–– Undocumented: Individuals and/or teams follow processes –– Corporate Planning
–– Master Data Management
that are not standardized nor documented. –– Business Shared Services
–– Data Quality Management
–– Defined: Processes used across business units and/or –– Business Unit
–– Reporting and Analytics
functions are documented but not enforced. –– Standalone Information Management Group
–– Information Lifecycle Management
–– Managed: Processes are documented and enforced. –– Sales
–– Optimized: Processes are documented, enforced, and –– Supply Chain 15. When IT provides a new report or analytics tool, does
measured to drive improvement. –– Other, please indicate the project plan include formal steps to change business
–– N/A—No Strategic Plan processes or workflow to make better use of it?
6. Does your organization have a strategic plan for analytics
and information management? 10. If IT is involved in developing the strategic plan for analytics 16. Are the implications of information and analytics
–– Yes, we have a standalone strategic plan for analytics and and information management, what role do the following requirements incorporated into other IT projects?
information management. play? –– Yes, before the project has been approved
–– Yes, analytics and information management are –– CIO –– Yes, when building out requirements for the project
incorporated into a broader IT strategic plan. –– Project Management Office –– Yes, after the project has been deployed
–– Yes, analytics and information management are –– Enterprise Architecture
17. Which of the following best describes your organization’s
incorporated into a broader business strategic plan. –– Infrastructure
approach to master data management?
–– No, we do not have a strategic plan for analytics and –– Applications
–– No formal initiative is currently in place or planned around
information management. –– Security
master data management.
–– Dedicated Information Management Team
–– Master data is defined and managed at the individual
–– Other, please indicate:
system level.
–– N/A—IT not involved in strategic plan
–– Siloed, department or group-level master data
management strategy has been defined and established.
CIO Executive Board

IT Practice
CIO0361911SYN

VIII. Information Strategy and Governance (Continued) IX. Technical Environment
–– A unified, enterprise-level master data management –– Yes, after they are developed or purchased 1. Do you use any document repositories that are not provided
strategy has been defined and established. –– No and supported by corporate IT (e.g., Google documents,
–– N/A—no data model Dropbox, iDisk) for business purposes?
18. Which of the following do you use as part of your master
data management? 24. Does your organization have an information architecture in 2. Do you use any business intelligence/analytics tools that are
–– Source identification place? not provided and supported by corporate IT?
–– Data collection –– Yes—We have documented both the current and future
3. If you answered “Yes,” why do you use tools not provided
–– Data transformation state architectures for the enterprise and have a roadmap
and supported by corporate IT?
–– Normalization in place against which we are moving toward the future
–– IT does not provide an alternative tool.
–– Rule administration state.
–– The non IT provided tool is easier to use.
–– Error detection and correction –– Yes—We have architected the current and future states for
–– The non IT provided tool is more sophisticated.
–– Data consolidation different business units or areas of the enterprise but have
–– The rest of my team uses the non IT provided tool.
–– Data storage yet to aggregate into a unified whole.
–– Other
–– Data distribution –– Yes—We have an initiative in place but are still in the early
–– Data governance stages. 4. Which of the following best describes the analytical tools
–– None –– No—We have a vision for what we want to do but no available to you?
formal initiative. –– There is no tool available to me.
19. Who owns or stewards master data subjects (e.g., customer,
–– No—We don’t have an information architecture and don’t –– I only have access to Excel.
product)?
plan to create one. –– There is a single tool I have access to that is consistent
–– IT
across the entire company.
–– Business unit 25. Which of the following best describes your data governance
–– There is a single tool I have access to that is consistent
–– A central or cross-functional master data management or processes?
across my business unit/function.
governance group –– We have formal governance structures in place that are
–– There are multiple tools with similar functionality I have
–– Other understood and adhered to across the enterprise.
access to and I am free to use the one I prefer.
–– None –– We have formal governance structures in place that
are understood and adhered to, but only in parts of the 5. Which of the following collective intelligence tools are
20. Does your organization have an enterprise data model for
organization. currently being used at your organization?
organizing information?
–– We have some formal and some informal governance –– Discussion forms
21. Which information types are included in the enterprise data processes in place; adherence is mixed. –– Blogs
model? –– Governance processes are generally ad hoc and –– Wikis
adherence is generally low across the organization. –– Prediction markets
22. If your organization does use a data model, which of the
–– There are no formal and few informal governance –– Ratings and online surveys
following components are included in your data model?
processes in place. –– Other, please indicate
–– Data architecture
–– Data modeling 26. Rate the following end user requirements that the 6. Which of the following best describes your organization’s
–– Data properties Information Management group is working to solve in order approach to selecting analytical tools?
–– Data organization of urgency). –– Tools are centrally selected and deployed across the
–– Data structure –– Information quality or accuracy organization.
–– Data model theory –– Information timeliness –– Tools are centrally selected and deployed across the
–– N/A—no data model –– Information consistency organization, but we customize capabilities to meet the
–– Information relevance needs of different functions/business units.
23. If your organization does use a data model, is there a
–– Information accessibility or ability to search –– Tools are centrally selected, but different tools are
process for reviewing new applications, tools, and databases
–– Usability of reporting and analytical tools deployed to different business units/functions based on
for alignment with the enterprise data model?
–– Flexibility to use personally preferred tools, e.g. via cloud their unique needs.
–– Yes, before they are developed or purchased
or mash-ups
CIO Executive Board

IT Practice
CIO0361911SYN
IX. Technical Environment (Continued)
–– Business units/functions choose their own tools with 13. Which of the emerging technologies and capabilities are in contributing to its removal? Please check all that apply.
guidance from IT and the information management and/ use (either broadly or narrowly)? –– No longer required to retain the information for regulatory
or analytics group. and compliance purposes
14. To what extent are analytics and information
–– Business units/functions choose their own tools and IT/ –– Information no longer has business value.
management solutions deployed using a test and learn
Information Management are not/rarely involved in the –– Too costly to continue storing the information
approach?
decision process.
20. Does IT integrate external sources of information into the
–– Individual knowledge workers choose their own tools. 15. Does your organization have an enterprise data warehouse?
data warehouse?
–– Yes—data from both structured corporate systems and
7. How many different analytical tools are in use? –– Yes—we proactively search for external sources of
unstructured information repositories are all stored in a
–– Across the company information that could be valuable to the business and
single data warehouse.
–– Maximum within a business unit integrate it into the data warehouse.
–– Yes—structured information from all corporate systems
–– Maximum within a function –– Yes—when the business recurrently uses an external
are stored in a single data warehouse, but Unstructured
–– Per information type source of information, we integrate it into the data
information is stored elsewhere.
warehouse.
8. How many different operational systems are in use? –– Yes—data from some corporate systems are stored in a
–– Partially—we make some external sources of information
–– Across the company single data warehouse, but there is a significant amount of
available internally, but it is kept separate from the data
–– Maximum within a business unit data outside the data warehouse.
warehouse.
–– Maximum within a function –– No—data is stored in several data marts.
–– No—we do not integrate external sources of information
–– Per information type
16. What is the total number of data warehouses within your into our environment.
9. How many different corporate information repositories are organization?
21. Do you have a master data management tool?
provided?
17. What is the total size of data warehouses within your
–– Across the company
organization in TB?
–– Maximum within a business unit
–– Maximum within a function 18. Which of the following best describes the information that is
–– Per information type included in your organization’s data warehouse?
–– All information generated by our organization is included
10. What is the total volume of storage in your organization
in the data warehouse indefinitely.
(include primary, backup, and archive) in TB?
–– All information generated by our organization is included
11. What is the annual percentage growth rate of total storage in the data warehouse, but it is retired and removed after
volume in your organization (include primary, backup, and a defined period of time.
archive)? –– Only information deemed valuable to the business is
included in the data warehouse.
12. Which of the following emerging technologies and
–– Only certain information is included based on technical
capabilities have you implemented?
feasibility.
–– In-memory analytics
–– Only certain information is included based on business
–– Data mash-ups
requests.
–– Cloud/SaaS BI or analytics
–– N/A—not in data warehouse
–– Columnar databases
–– Mobile BI 19. If your organization removes information from the data
–– Interactive visualization warehouse after a period of time, what are the factors
–– Enterprise search
–– Semantic technologies
–– Complex event processing
–– Advanced analytics
CIO Executive Board

IT Practice
CIO0361911SYN

X. Demographics
1. What is your age? 15. How high is the regulatory burden faced by your
organization?
2. What is the highest level of education you obtained?
16. What is the level of business integration at your
3. In which subject or discipline did you achieve an
organization?
undergraduate- or graduate-level university degree?
17. What is the level of information intensity of each of the
4. What is your tenure at your current organization?
following at your organization?
5. Which of the following most closely describes your current
18. What is the overall annual revenue (in millions of your
level in the organization?
currency) for your organization?
6. Which of the following most closely describes your current
19. To what degree does your organization handle sensitive or
function in the organization?
confidential information?
–– Communications
–– Corporate Strategy 20. How many employees does your organization employ?
–– Customer Service/Contact Center
21. In how many countries does your organization have
–– Finance and Accounting
operations?
–– Financial Services
–– General Management Functions 22. Which of the following best describes your IT
–– Human Resources organization’s structure?
–– Information Technology
23. How many IT users does your organization support?
–– Legal and Compliance
–– Marketing/Market Research 24. How significant a part of your job responsibilities is
–– Production/Operations/Procurement conducting analysis?
–– Professional Services
–– Real Estate and Facilities
–– Research, Development, and Engineering
–– Retail
–– Sales
7. Indicate any additional functions you have worked in

previously.
8. In what country do you primarily work?
9. In which country is your organization headquartered?
10. Where do you spend the majority of your time working?
11. Which of the following best describes your attitude toward

new technologies?
12. Which of the following best describes your organization’s

industry?
13. How geographically dispersed is your organization?
14. What is the ownership structure of your organization?
CIO Executive Board

IT Practice
CIO0361911SYN

CIO Executive Board - Overcoming The Insight Deficit PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CIO Executive Board - Overcoming The Insight Deficit PDF

Uploaded by

Copyright:

Available Formats

Information Technology Practice

© 2011 The Corporate Executive Board Company. All Rights Reserved.

Copies and Copyright Legal Caveat

An Information Management Strategy on a Page • vi

Assess Insight IQ in Your Organization • vii

Occasion for the Research • 1

Enable Information Usability • 25

Support Big Judgment, Not Just Big Data • 77

Observe and Question to Identify Opportunities • 101