Professional Documents
Culture Documents
Overcoming
the Insight Deficit
Big Judgment in an Era of Big Data
1 Pseudonym.
CIO0361911SYN
iv
Executive Summary
Our ability to store and analyze information has grown exponentially across Core information governance activities such as information architecture,
the past decade, but the buzz about “big data” misses the reality that as business stewardship of data, and an analytics center of excellence are
data volumes explode, it becomes harder, not easier, for knowledge workers important drivers of Insight IQ. Most organizations know this and already have
to make sense of it. Less than 40% of knowledge workers have the skills and initiatives in place. What is more surprising is the impact of three capabilities
judgment to exploit information for competitive advantage. Paradoxically, directly related to the knowledge worker: information usability, knowledge
as organizations amass more data and spend more on analytics, they face a worker skills, and the identification of emerging knowledge worker information
growing insight deficit. needs. These three capabilities are often overlooked, yet all directly impact
Insight IQ.
The insight deficit is important to IT, as information management will likely
be IT’s main contribution to enabling growth. In business areas that drive 1. Enable Information Usability
growth—innovation, marketing, sales, and customer service—up to 80% of
IT enablement opportunities relate to business intelligence, collaboration, or Only 50% of knowledge workers find information from corporate sources
the customer interface. Yet, the insight deficit means that efforts to improve to be in a usable format. The problem will only get worse as the number
information management may lead to worse, not better, decision making. of information sources, uses, and users continue to increase. In this diverse
environment, trying to improve usability by offering self-service usually
Insight IQ: A Unique View of How Organizations Exploit Information for disappoints as few self-service capabilities are flexible enough to meet
Decision Making knowledge worker needs. To overcome this challenge, the best companies
offer a choice of analytic tools and deploy information filtering and
To give IT leaders practical guidance on overcoming the insight deficit, we
visualization capabilities. They develop a deeper understanding of how, when,
built an Insight IQ index to measure individuals’ ability to find and analyze
and why information will be used by specific user segments, and they vary
information to make better decisions. Insight IQ has three elements:
quality standards accordingly.
information attainability, information usefulness, and knowledge worker
capability. Offer Greater Tools Choice—Greater choice in analytic tools drives Insight
IQ. The greatest increase comes from moving from one standard tool
Using the model, we surveyed almost 5,000 knowledge workers at hundreds of
for the enterprise to tools aligned to specific business units or functions.
organizations globally. We used the survey to quantify the value of overcoming
Rationalization programs for analytic tools often over-standardize, leaving
the insight deficit and pinpoint the actions needed to realize this value. (See
employees without necessary capabilities and opening the door to unofficial
page vii to learn how your organization can take the Insight IQ diagnostic).
tools. In response, progressive organizations, such as Blue Cross Blue Shield of
The benefits of overcoming the deficit are significant. Functions with the North Carolina offer knowledge workers a portfolio of tools aligned to a list of
highest Insight IQ perform, on average, 24% better than their peers across a analytic capabilities.
wide range of metrics. All three components of Insight IQ are vital to achieve
Visualize and Filter—Many knowledge workers complain about the time
this goal—if any are missing, the chance of failure is 65% or more.
required to aggregate information from supposedly user-ready reports.
Consequently, leading organizations invest in capabilities such as information
visualization, aggregation, and filtering before they invest in more sophisticated
analytics.
CIO0361911SYN
Executive Summary (Continued)
Make Information Quality Transparent and Targeted—Only one-third of Provide Tools for Collaborative Decision Making—Making biases explicit
knowledge workers trust information from other functions in the organization. and listening to alternative points of view are characteristics of informed
To overcome this distrust, progressive organizations such as GlaxoSmithKline skepticism. To support these behaviors, IT can provide workflow tools that
clarify the information’s source, how it has been manipulated, its current connect stakeholders involved in decision making, capture “wisdom of the
quality level, and caveats about how it can be used. Others, such as PepsiCo, crowds” input, and add contextual data. In addition, business leaders can help
recognize that certain groups, such as C-level executives, have distinct patterns promote informed skepticism by introducing formal decision-making models
of information usage that need to be targeted for different-in-kind support. and by embedding the right behaviors in performance expectations.
2. Support Big Judgment, Not Just Big Data 3. Observe and Question to Identify Opportunities
Exploiting information requires broader and deeper analytic skills across the Most IT organizations are familiar with process automation projects where
organization. Most employees are now knowledge workers and spend on business needs are known and stable. In contrast, information needs are
average 36% of their time collecting and analyzing information. However, only context dependent, dynamic, and may be unarticulated or even unknown.
38% of employees are “informed skeptics” who combine analytic skills with Solving this challenge requires anthropological skills that are in short supply
judgment to make sound decisions. These critical thinkers are comfortable with in IT.
analytics but maintain a healthy skepticism about its limitations. Identify and Learn from Early Adopters—Traditional requirements gathering
Develop an Analytics Training Curriculum—Two-thirds of knowledge workers fails when assessing information needs as the needs are fast changing,
have access to analytics training, but only 25% find the training effective. diverse, and difficult to articulate. Instead, progressive organizations, such as
Training that focuses on the tools but does not address information sources Ford, surface information needs by segmenting knowledge workers by their
or analytical techniques is ineffective. Leading organizations, such as Tiffany & information usage patterns and their stance toward technology adoption.
Co., adopt a multipronged approach that supports knowledge workers across They also observe early adopters as an early-warning mechanism for
the information lifecycle and focuses on conducting analysis. This type of emerging needs.
training has no natural organizational home, so IT and business leaders should Observe and Question—Knowledge workers often struggle to articulate their
collaborate to determine ownership. information needs, and these needs differ across individuals, organizational
Hire Analysts Who Can Coach—Knowledge workers should be supported by cultures, and collaborative practices. To proactively identify pain points in
business intelligence experts or analysts who act as force multipliers. While end-user workflows, leading IT groups adopt an anthropological approach to
coaching skills are what make these analysts most effective at driving Insight identify opportunity, observe knowledge workers in diverse user settings and
IQ, organizations typically look for quantitative and business skills when hiring conduct open-ended interviews.
analysts. Leading organizations have changed their recruiting approaches to
actively test for coaching skills.
CIO0361911SYN
v
vi
Statement of Strategy
Shift the focus of IT investment and skills toward information management with the goal of providing knowledge workers with attainable and useful information
and boosting their capability to exploit that information for competitive advantage.
3. Identify analytic capabilities used by knowledge workers, Percentage of information subjects targeted > 80% < 40%
and offer a portfolio of tools to meet those needs. for harmonization and integration
Target number of analytic tools 1–3 8–12
4. Develop and hire usability and interface design skills in IT.
Percentage of knowledge workers 38% 59%
5. Coach knowledge workers to boost their analytic capability
who are informed skeptics
and foster informed skepticism.
Average Insight IQ score 51 65
6. Harmonize and integrate a small subset of information subjects
where there is greatest enterprise need.
1. The number of opportunities to drive growth through information management will equal or outstrip the opportunities for process automation.
2. Many of our knowledge workers lack the skills and judgment to use information effectively for decision making.
3. Not all information needs to be harmonized or integrated at enterprise level. Similarly, some information needs higher levels of quality than others.
4. Our business partners will take the lead in information stewardship.
5. Knowledge worker reliance on external information sources and on unstructured information will continue to rise.
CIO0361911SYN
Use the Insight IQ
Diagnostic to benchmark
Assess Insight IQ in Your Organization
the maturity of your
information management Overview of the Insight IQ Diagnostic Diagnostic Benefits
and prioritize areas for
investment.
■■ The diagnostic measures Insight IQ using online 1. Assess Readiness—Identify which business
surveys for knowledge workers and an IT leader. functions and teams in your organization are
ready for additional investment in information
■■ Organizations identify at least 30 knowledge
management and analytics and be forewarned
workers per business function, and at least three
about areas where adoption may be low.
business functions participate in the diagnostic.
2. Set Priorities—Prioritize the technology
■■ Each survey takes approximately 20 to 25 minutes
investments, governance processes, and skills
to complete.
that have the greatest impact on Insight IQ.
■■ The diagnostic is included in your existing CIO
3. Communicate—Demonstrate the business
Executive Board membership.
leadership, communication, and change
management efforts necessary to close your
organization’s insight deficit.
4. Benchmark—Compare your organization’s
Insight IQ against industry peers.
Key Deliverables
Custom Diagnostic Report and Improvement Recommendations
CIO0361911SYN
vii
viii
CIO0361911SYN
Occasion for the Research
CIO0361911SYN
1
Overcoming the Insight Deficit 2
Source: “All Too Much” The Economist, 27 February 2010; Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board
IT Practice
www.cio.executiveboard.com
Occasion for The Insight IQ Overview Roadmap for
Existing Research
CIO0361911SYN the research Diagnostic of findings the discussion
To drive growth, IT must
support less familiar
FROM PROCESS TO INFORMATION TO DRIVE GROWTH
areas of the business
where the majority of Breakdown of IT Enablement Opportunities by Business Process
opportunities are in Percentage of Total Opportunities
information management, None
9%
9%
not process automation. 15% Information Management
18%
27% (Collaboration, Business
28% Intelligence, Customer Interface)
■■ IT enablement opportunities Process Automation
are divided between:
–– Information 30%
Management—Includes
collaboration, business 28% 63%
intelligence, and the use
of technology at the 63%
customer interface, and 60%
–– Process Automation—
Processes enabled by
enterprise systems such
as ERP, PLM, HRIS, and 54%
46%
CRM.
28%
18%
13%
As 62% of knowledge
workers lack the ability
Without Big Judgment, Big Data Magnifies Risk
to apply “big judgment”
to information they use
not Opportunity
for decision making,
increased information Have Processes and Skills to Use Information Effectively for Decision Making
Percentage of Knowledge Workers
availability may do more
harm than good.
62%
■■ Misleading performance measures
No ■■ Wasted knowledge worker time
■■ Insight IQ is a quantifiable
Insight IQ entails the ability to find
outcome that can be and analyze relevant information to
measured at the level of the drive actions and decisions effectively.
individual knowledge worker.
■■ Linking information
management initiatives with
higher Insight IQ will refocus
information management
on the important outcome.
The right information is Information Information Information is of a known
available and easy to find. Attainability Usefulness quality and in a usable format.
Insight IQ
Knowledge
Worker
Capability
Respondents by Geography
Sample Distribution
1%
Other
2%
Latin America
14% 65%
APAC North America
18%
EMEA
n = 4,941 knowledge workers.
CIO Executive Board
IT Practice Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
www.cio.executiveboard.com
Occasion for The Insight IQ Overview Roadmap for
Existing Research
CIO0361911SYN the research Diagnostic of findings the discussion
A Model of Insight IQ
Knowledge
Worker
Capability
1 Deploy Surveys 2 Determine Drivers of Insight IQ 3 Translate Findings into Practical Guidance
■■ Distill the practical implications borne out by
Knowledge Worker Survey (n = 4,941.) Potential Drivers of Insight IQ
the data.
Knowledge workers from all major ■■ Provide real-world illustration of key insights
functions and business areas are 1. Knowledge Worker via practitioner tools and tactics.
surveyed on information usage. Attitudes and Incentives
2. Training and Support
3. Staff Knowledge and Competencies
4. Information Quality
5. Information Usage Patterns
IT Leader Survey (n = 22.) 6. Demographics
IT representative provides details on 7. Information Management Structures Illustrative Driver: Availability of Training
information management platforms, and Processes on Conducting Analysis
strategy, governance, and architecture. 8. Investment and Resource Level Average Insight IQ Score, Indexed
9. Information Strategy and Governance
10. Technical Environment ∆ = 29%
(Maximum 129
Impact)
100
Sample Questions
Which of the following best describes the
analytic tools available to you?
Information Information
Attainability Usefulness
No Tools Multiple Tools
Insight
IQ
65
51
38
28
10th Percentile 25th Percentile 50th Percentile 75th Percentile 90th Percentile
1 Insight IQ is measured as an index constructed from the three model components. Index scores ranges from 0 to 100, with higher
scores reflecting higher levels of Insight IQ.
■■ Variation within
70
organizations demonstrates
that company-level factors
30
20
10
0
Representative Sample of Participating Organizations
n = 22 organizations.
1 Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
Business functions
with a high Insight
Finding 2: OVERCOMING THE INSIGHT DEFICIT Has
IQ demonstrate 24%
better performance than
a Measurable Payoff
functions with a low
Insight IQ. Indexed Functional Performance by Level of Insight IQ
What Matters
■■ Seniority The CIO Executive Board applied
statistical controls to void the
■■ Openness to new technologies
impact of demographic factors
■■ Importance of analytics to job role that drive Insight IQ.
■■ Company revenue
■■ Level of regulatory burden
■■ Information intensity
Organization-Level Factors
■■ Industry
■■ IT organizational structure
■■ Geographic span of business
Poor performance in any of the three components of Insight IQ significantly diminishes the probability of strong business performance.
Type 1: Knowledge Workers Cannot Type 2: Knowledge Workers Type 3: Knowledge Workers Cannot Type 4: High Insight IQ
Easily Access Information Are Wary of Analytics Translate Information into Action
Probability of strong business Probability of strong business Probability of strong business Probability of strong business
performance1 if information performance if knowledge worker performance if information performance: 80%
attainability is low: 18% capabilities are low: 27% usefulness is low: 35%
1 Values are calculated as the probability of knowledge workers demonstrating top quartile performance in indexed business performance via logistic regressions.
I. Information Management Structures and Processes III. Knowledge Worker Attitudes and Incentives V. Quality of Information (Continued)
1. Analytic Capabilities and Practices Shared 34. User Confidence in Information Quality (Cleanliness and 60. New Technologies Prescreened for Enterprise Data Model
2. Analytics Teams Centralized Companywide Accuracy) Alignment
3. Central Analytics Group in IT, Reporting Directly to CIO 35. What Level of Confidence Do You Have in the Quality 61. Quality Standards Vary by Function
(Cleanliness and Accuracy) of Information from Each of the 62. Quality Standards Vary by Information Type
4. Central Analytics Group in IT, Within Information
Following Functions? 63. Quality Standards Vary by Seniority
Management
36. Knowledge Workers Are Rewarded and Promoted Based on 64. Satisfaction Level for Each Type of Information
5. Delivery Channels for Providing Information/Analytics Outcomes of Their Analysis and Decisions. 65. Timeliness Standards Vary by Information Type
6. Degree of Integration of Management of Analytics and 37. Knowledge Workers’ Perceived Ease of Use of Information 66. Use of Common Definitions and Vocabulary in Information
Information Contained in Central Repository
Presentations
7. Degree of Knowledge Worker Involvement with Other 38. Knowledge Worker Preference for Push Versus Pull for Types
Functions/Groups of Information
8. Existence of Formal Analysis Processes/Frameworks 39. Knowledge Worker Trust in Information Contained in Central VI. Information Usage Patterns
Repository
9. Existence of Master Data Management Tool
40. Knowledge Worker Trust in Information from Other Groups/ 67. Capabilities in Place to Obtain Cross-System/Functional
10. Existence of Metadata Structure (Standards and Models) Functions (and Reason) Information
11. Integration of External Sources of Information into the Data 41. Level of Blame for Relying on Inaccurate Prediction 68. Degree of Integration of Cross-Functional Information
Warehouse 42. Perception That Being Analytic Is Advantageous to Career 69. Frequency of Using Each Type of Information/Analysis
12. Information/Analytics Requirements Incorporated into Other Development. 70. IT Support for Different Types and Sources of Information
Projects Early 43. Perception That Analytics Is Hard Due to Lack of Time. 71. Nature of Information/Analytics Use (e.g., Customer Analysis,
13. IT Organization Structure (e.g., Centralized or Decentralized) 44. Perceived Importance of Information Aggregation BPO)
Capabilities 72. Need of Information from Different Functions
14. Knowledge Workers Have Enough Time to Conduct Analysis
45. Perceived Importance of Information Filtering Capabilities 73. Number of Sources Required for Each Type of Analysis
15. Master Data Management Process Includes Error Detection 46. Perceived Importance of Information Visualization
and Correction 74. Purposes of Information/Analytics Use (e.g., Decisions,
Capabilities Reporting, Pattern Spotting)
16. Mechanisms of Delivery for Each Type of Information— 47. Visibility of Internal Opportunities That Require Analytic
Pushed or Pulled 75. Relative Use of Different Types of Info. (e.g., Structured/
Skills
Unstructured, Internal/External)
17. Prevalence of Analytics in Group Versus Individual Decisions
76. Satisfaction Level for Mobile Access to Key Information
18. Process for Sharing Information
IV. Investment and Resource Level Types
19. Responsibilities for IM Group (Including Who Owns What) 77. Time Spent on Information Visualization, Filtering,
20. Responsibilities of Central Analytics Group 48. Approach Taken to Getting Approval for Investments in IM/ Aggregation, and Analysis
21. Use of Aggregation, Visualization, and Analysis Analytics 78. Types of Analyses Being Done (e.g., Predictive, Prescriptive)
22. Use of Agile Software Development 49. Cost of/Investment in Analytic Capabilities (by Function) 79. Types of Information Being Used
50. Level of IT Investment in Capabilities to Support Group 80. User Feedback on Information Sources and Analytics Tools
23. Use of External Providers for Analytics
Decision Making Collected
51. Number of IT Staff Responsible for Information Management 81. Users Information Needs Assessed via Surveys
II. Training and Support 52. Technology Costs for Managing Information 82. Users Information Needs Assessed by Tracking Information
Requests
24. Analytic Mind-Set of Respondent’s Manager V. Quality of Information
25. BU/Functional Performance Metrics Reported Regularly
26. Formality of Decision-Making Process 53. Existence of Automation of Information Quality Governance
27. Leadership Attitudes/Behaviors Toward Fact-Based Decision and Standards
Making 54. Existence of Process for Updating and Retiring Information
28. Overall Frequency of Using Analysis (Info. Lifecycle Management)
29. Use of KPIs/Metrics to Track Organizational Performance 55. Existence of Standard Definitions for Information
30. Performance Reviews Based on Measurable Goals 56. Data Transformations Are Transparent to Users.
31. Performance Reviews Evaluate Analytic Ability 57. Frequency of Reviewing/Updating Analytic Models
32. Time Spent on Analysis and Assumptions
33. Existence of Processes/Capabilities to Overcome Biases 58. Functional Information with Standard Definitions
59. Information Quality Regularly Reviewed/Audited
Partial List of tested drivers (Continued) Shading Denotes Top 15% Most Impactful Drivers
VII. Staff Knowledge and Competencies VIII. Information Strategy and Governance IX. Technical Environment
83. Analyst Skills Valued 1 07. Approach Taken on Analytic Tool Selection and Distribution 134. Analytic Tools Bought Off the Shelf or Developed In House
84. Analytics Staff Hired and Assessed for Training and Coaching 108. Approach Taken on Analytic Tool Selection and Distribution 135. Analytics Solutions Deployed Using a Test and Learn
Abilities 109. Clear Goals or Strategies for Social Media in Place Approach
85. Assessment of Analyst Speed, Quality, and Responsiveness 110. Components of Analytic Strategic Plan 136. Automated Decisions Implemented
86. Availability of Support in Finding or Analyzing information 111. Coverage of Analytics Strategic Plan 137. Customizability of Analytic Tools
87. Availability of Training on Finding or Analyzing Information 112. Decision Stakeholders 138. Enterprise Search Implemented
88. Consumption of Training on Finding or Analyzing 113. Degree of Standardization of Key Business Processes 139. Existence of and Types of Collaboration Tools/Capabilities
Information 114. Degree of Sharing Information with Customers 140. Existence of Automated Information Capturing Capabilities
89. Existence of Training on Use of IM and Analytic Systems 115. Degree of Sharing Information with Suppliers 141. Existence of Central Repository
90. Effectiveness of Training on Finding or Analyzing Information 116. Documentation of Business Processes 142. Existence of a Data Warehouse
91. High-Quality Training on Finding, Understanding, and 117. Existence of Analytics Strategic Plan 143. Existence of In-Memory Technology Capabilities
Analyzing Info. 118. Existence of Data Steward Role 144. Existence of Non IT-Supported Information Repositories
92. Inclusion of Analytics in Job Descriptions 119. Existence of Dedicated Information Management Team 145. Degree of Integration of Operational Systems
93. Knowledge Worker Analytic Skills and Level 120. Existence of Strict Security Policies Around Information 146. Information Sources (e.g., Smart Networks, Sensors)
94. Knowledge Worker Awareness of Security Policy Related 121. Information Quality Standards and Policies Are Defined and 147. Knowledge Workers Have Access to Analytic Tools Other
to Information Sharing Enforced Than Excel
95. Knowledge Worker Degree of Proficiency in Using Analytic 122. Information Quality Standards and Policies Enforced by Info. 148. Knowledge Worker Can Choose from Multiple Analytic Tools
Tools Management Group Available
96. Knowledge Worker IT Skills 123. Information Quality Standards and Policies Enforced by 149. Legacy Systems
97. Knowledge Worker Knowledge of What Information Exists Shared Services Group 150. Number of Analytic Tools Being Used/Provided
in the Central Repository 124. Information Security and Access Policies Do Not Inhibit 151. Number of Features of Analytic Tools
98. Knowledge Worker Perception of Ease of Use of Sharing Attainability of Information 152. Number of Information Repositories
Analysis/Info. Using Analytic Tools 125. Inclusion of Analytics in Broader Corporate/IT/Other 153. Number of Operational Systems (by Function)
99. Knowledge Worker Relies on Judgment When Disagreeing Function Strategic Plan 154. Tool Interface Design, Testing, and Enhancements
With the Results of Analysis 1 26. Information Architecture in Place or Planned 155. Use of Cloud Applications to Generate, Store, or Analyze
100. Knowledge Worker Understanding of Analysis Behind 127. Keeping Track of New Information Systems and Analytic Information
Strategic Decisions Capabilities
101. Knowledge Worker Understanding of How Information is 128. Organization Characteristic (e.g., Consensus-Based or
Useful in Other Functions/Groups Hierarchical)
X. Demographics
102. Knowledge Worker Understanding of Responsibilities of 1 29. Organization Strategic Business Goals/Objectives
Other Functions/Groups (i.e., What They Do) 130. Overall Ownership of IM Group 156. Function Responsible for Developing Analytics Strategic
103. Presence of Analytics Coach (Formal or Informal) 131. Perceived Clarity of Roles and Responsibilities for IM Group Plan
104. Processes for Understanding Tool Usage and Workflows 132. Policies for Adding Information to a Data Warehouse 157. Knowledge Worker Career Background (Experience,
105. Skills and Knowledge of Staff Responsible for Managing 133. Robustness of Enterprise Data Model Number of Functions Worked In)
Information 158. Knowledge Worker Function
106. Use of Hypotheses to Drive Analysis 159. Knowledge Worker Seniority
160. Organization Industry
30.3%
IT Practice
n = 4,941.
CIO0361911SYN
9. Having an Information
Architecture
www.cio.executiveboard.com
11. Having a Mature Data Model
KW = Knowledge worker.
17.2% 16.4%
13. Having Central Analytics
Group or COE
The Usual Suspects
16.2% 15.5%
2. KWs Having Confidence in
Information Quality
25.2%
the research
Occasion for
1. Enable
Diagnostic
4. Training KWs on How to Find
The Insight IQ
and Conduct Analysis
5. Formalizing Decision-Making
Process
24.7% 24.6%
6. KWs Are “Informed Skeptics”
20.0%
Overview
18.6%
of findings
8. Regularly Reporting Business/
Functional Performance Metrics
17.4%
2. Support Big
The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively “high” on a driver and
3. Collecting KW Feedback
on Information Sources and
24.8%
Analytics Tools
Information Needs
16.8%
and Question
Opportunities
19
Overcoming the Insight Deficit 20
1. Enable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
1. E
nable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Specialists
Architecture Roadmapping
■■ While necessary, these
Target Architecture Roadmaps Business
Architecture
capabilities are not sufficient (Wells Fargo) (KBC Bank)
to boost an organization’s
Insight IQ or to move IT’s
capabilities from process Information Strategy and Stewardship
automation to information
management. Task-Specialized Data
Stewardship (RBC Financial)
Information Quality
Report Generation
CIO0361911SYN
Overcoming the Insight Deficit
1. E
nable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Specialists
CIO0361911SYN
■■ Improve information
usability by letting
knowledge workers choose
the tools they find easiest to
use, prioritizing information
visualization and offering
greater transparency into
50%
quality levels.
No
50%
Yes
■■ The proliferation of
information sources creates
challenges for information Corporate Operational Systems 4.7
aggregation, presentation,
and quality.
24%
No
76%
Yes
The proliferation of
information users, uses,
INFORMATION EVERY WHICH WAY
and sources drives new
usability problems. Usability Challenges
1 Inadequate Tools—No single 2 Inflexible Reports and Cubes— 4 Information Quality Issues—
analytic tool can support all Predefined reports cannot Quality varies significantly by
needs and skills levels. answer all knowledge worker information source in ways that
questions while cubes are slow are not always transparent to
and offer limited dimensions. the user.
CXO KPI
Identification
Data Cubes
Dashboards
Standard
Reports
Low
One Size Choice Shaped
Fits All by Business Need
Approach to Usability
1 Pseudonym.
CIO Executive Board
IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
If knowledge workers
don’t find the analytic
Users have choice (official or not)
tools provided by IT
useful, they have an array Use Analytic Tools That Are Not Supported Reasons for Using Analytic Tools That Are
of other choices. by IT Not Supported by IT
Percentage of Knowledge Workers Percentage of Knowledge Workers
68%
No
The Unofficial
Tool Is More 29%
Sophisticated
∆ = 13.1%
(21.4%)
1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively
“high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of
multivariate regressions with controls.
■■ Progressive organizations
rightsize their portfolio
of analytic tools by
identifying and aligning
with capabilities.
Overview
A methodology for selecting a set of analytic tools that collectively provide 10 key capabilities to knowledge workers
Executive Teaching
Focus on helping knowledge workers use the right tool for the right purpose rather than aiming to minimize the total
number of tools. Identify the analytic capabilities most needed by knowledge workers, and use the capabilities
to assess current and future analytic tools to identify gaps and redundancy in the tool portfolio.
Company Snapshot
Source: http://www.hoovers.com.
In an internal survey
of tools used for
A Plentiful but Ill-Fitting Portfolio
analytic functions,
BCBS NC learned that Blue Cross Blue Shield of North Carolina Analytic Tool Environment
its portfolio of analytic 2009
tools is misaligned and One Tool Fits All
duplicative and used for Teams use one tool (often Excel or Access) for all needs, missing the chance to use tools
suboptimal purposes. that better meet their needs.
■■ The capabilities are reviewed 5. Advanced Analytics/ Analysis typically done by a small number of analysts who look for new business
annually and adjusted Data Mining insights by examining relationships of information and information trends, this includes
according to evolving data and text mining.
knowledge worker needs. 6. Data Visualization Not limited to charts, this includes any functionality providing faster consumption
of information through “pictures” or presentations other than data grids.
■■ See page 37 for an overview
7. Dashboards/Scorecards Dashboards—On-screen visualizations that provide a quick status of key performance
of the survey process and
indicators specific to a role
page 38 to 39 for a list of the
survey questions. Scorecards—Use colors, indicators, and numbers to compare actual performance of key
metrics against a target or budget. These are slightly more detailed than dashboards
and less visual.
8. Geospatial Analysis This is an array information across a geography to show relationships between data
and locations.
9. Operational Reporting This is not used to analyze performance, but used for operational reporting. Examples
include sending an invoice to a customer or a letter to a member.
10. Publishing and Delivery Report distribution including e-mail, print and mail, or using a business intelligence
(BI) tool to distribute through a built-in or custom portal for providers, members,
or internal staff.
Maintain a portfolio
of tools that supports
A Portfolio of ENTERPRISE and
all 10 capabilities. Specialty Tools
■■ Tools are designated in one
of two categories:
–– Enterprise Tools support
Dashboards/Scorecards
Conditioning/Validation
Operational Reporting
multiple capabilities used
Advanced Analytics/
Geospatial Analysis
for the same purposes
Data Visualization
Data Collection/
Static/Printable
across the company;
Ad Hoc Report
Development
between three and five
Data Mining
Reporting
of these tools should
support most capabilities.
–– Specialty Tools support
one capability in one
particular function;
Enterprise Tools (3 to 5)
another six to nine
of these tools may
BObj Web Intelligence
be needed.
SAS Core
■■ BCBS NC identifies
capability gaps and helps Crystal Reports
teams use the tools that best
provide the capabilities they
need.
Speciality Tools (6 to 9)
“While we will always
retire tools where the Amisys MACESS
ROI isn’t there, we are
Teradata SQL Assistant
more concerned about helping
people use the right tool for the
MCSource
right purpose and get away from
using one tool for everything.” EDIWatch
Todd Norris
Director, IM Delivery
BCBS NC
Capability gaps
CIO Executive Board
IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
The process of capturing
analytic capabilities by
IMPLEMENTATION GUIDE: CAPABILITY
surveying knowledge
workers requires
SURVEY ROADMAP
executive sponsorship
and clear communication. Keys to Successfully Surveying Knowledge Workers on Their Capability Requirements
Representatives are
2. Identify user representatives in each division to administer told that each survey
the survey. should take less than
an hour per analytic
area to complete.
“The process of
surveying was just as
important as the
questions we asked. The survey
was not intended to capture 4. Follow up surveys with an interview with each group to validate
perfect information but to give results and ask questions.
informed guidance for next steps.”
Susan Helm-Murtagh
Three Weeks
VP, Information Management
for Entire
and Analytic Services
Process
BCBS NC
BI Capabilities Tool 1 2 3 4 5
Tool 1 2 1
Advanced Analytics/
Geospatial Analysis
Tool 2
Ad hoc Reporting
Data Collection/
Static/Printable
Tool 3
Publishing and
Conditioning
Dashboards/
Visualization
Operational
Self Service
Scorecards
(mapping)
Reporting
Reporting
Delivery
Other 2
Other 1
Mining
Feedback on BI
Technical Skills:
Tool
Tool A X X X X X
Tool B 3. C
omment on tools that are particularly critical to your area or feel
Tool C X X X X X free to mention tools you feel, if improved, could save you time or
Other 1 X X provide greater information insights.
Other 1 X X X X
Any other feedback on the
tools used in your area 4. C
omment on activities you do in your area that you feel should
Any other feedback on the not be your responsibility or you feel your team is not adequately
tools used in your area trained to support.
II. Effort
5. Number of people who do analytical or BI development work. 6. Percentage of time spent by high-level activity
Number of full-time equivalent employees in your area Collecting and assembling data for analysis 20%
9
Conditioning, filtering, or validating data for analysis 20%
How many FTE resources would you need to fully
9 Developing new reports, report solutions, or analysis 20%
satisfy the requests of your team?
7. O
f the time your area spends collecting and assembling data, list 8. O
f the time spent conditioning and filtering information, how do you
the percentage spent on each activity. spend that time?
Activity % Activity %
Pulling information from various sources 50% Developing/applying filtering and rules unique to your needs 90%
Manipulating information in spreadsheets/MS Access/SAS 50% Developing/applying filtering rules than are/could be used by others 10%
Other 1
Total 100% Total 100%
9. L
ist the main sources of information you use or plan to use in the near 10. How does your report solution development time breakout?
future.
CIO0361911SYN
Usability can be
improved by providing
USABILITY = Bounded FLEXIBILITY AND
knowledge workers
greater flexibility in
TRANSPARENCY
tools choice, information
access, and use.
CXO KPI
Identification
Data Cubes
Dashboards
Standard
Reports
Low
One Size Choice Shaped
Fits All by Business Need
Approach to Usability
49% 51%
Yes No
1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively
“high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of
multivariate regressions with controls.
It is nearly impossible to
uphold all dimensions of
Can All Information Be Perfect All the Time?
quality in all information
sources. Common Dimensions of Information Quality
Option 1: Avoid Using Information Until Option 2: Help Knowledge Workers Use
All Quality Problems Are Fixed Information in Its Current State
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO Executive Board
IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
Despite the high impact
on Insight IQ, only about
Trust No One
one-third of knowledge
workers trust information Maximum Impact on Insight IQ1 of Have Confidence in Information
from other functions in Knowledge Worker Confidence in Provided by Other Functions
the organization. Information Quality Percentage of Knowledge Workers
25.2%
■■ This lack of confidence
32%
translates into perceived Have Confidence
low information quality in Information from
and usability. Other Functions
Lack of transparency
in how information
Trust From Transparency
is collected, stored,
or analyzed is the Reasons for Lack of Confidence in Information Quality Provided by Other Functions
number-one reason why Percentage of Knowledge Workers
knowledge workers lack
confidence in the quality
of available information. Lack of transparency
is a more significant
Lack of Transparency in How reason for lack of
■■ Knowledge workers need Information Is Collected, 60.0% confidence than
Stored, or Analyzed the underlying
clarity on where information information quality
came from and how it has problems.
been transformed.
An approach to information management and architecture that provides the flexibility and transparency knowledge workers
need to conduct the analysis they require within a universe of information related to key business performance questions
Executive Teaching
A lack of understanding of information sources, definitions, quality, and relationships makes it hard for knowledge
workers to assemble the information they need to address key business performance questions. The diversity of
knowledge worker information needs means that it is not possible to support them with predefined reports and cubes
and central support for ad hoc queries is slow and resource-intensive. Instead IT should promote analysis freedom and
confidence by offering a flexible architecture and transparency into underlying information sources.
Component Teachings
1. Business Performance Question Mapping—Identify the universe of information knowledge workers require by
mapping the business performance questions they need to address.
2. Flexible Data Analysis Architecture—Enable flexible, on-the-fly analysis capabilities through in-memory
architecture organized around key business questions.
3. Contextualized Information Access—Provide knowledge workers with access to information organized by their
top business questions.
4. Data Quality Transparency—Provide transparency into data quality and source to help knowledge workers decide
when the data can and can’t be used.
Company Snapshot
GlaxoSmithKline plc
Industry: Pharmaceuticals GlaxoSmithKline (GSK) is one of the world’s leading pharmaceutical firms. GSK
2010 Sales: US$43.9 B produces medicines that treat major disease areas such as asthma, anti-virals,
Employees: 98,485 infections, mental health, diabetes, cardiovascular, and digestive conditions. In addition,
the company is a leader in vaccines and is developing new treatments for cancer.
GSK also markets other consumer products, many of which are among the market
leaders.
CIO Executive Board
IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
A knowledge worker
survey revealed high
Assembling Data is Very Time Consuming
dissatisfaction with the
amount of time wasted Knowledge Worker Data Satisfaction Survey Results
trying to assemble data Illustrative Results Shown
provided in disparate
reports and dashboards.
Knowledge Worker Director Director Sales Product Field Sales Field Sales
Pain Points Area Sales Support Director Specialist Manager
■■ GSK’s North American
Sales and Marketing IT 1. Unanswered
department set out to Questions
understand the lifecycle
of information delivery to 2. Missing Data
support business processes.
3. Can’t Access Data
■■ The results revealed a need
for better understanding
4. Tools Aren’t Useful
of key business questions
to improve the speed 5. Slow Response to
and flexibility of analysis Data Needs
capabilities in the sales
6. Data Inconsistency
team.
and Inaccuracy
7. Need More
Historical Data
8. Waste Time
Assembling Data
Assembling Data Is
Challenge
Very Time Consuming
“Knowledge workers
wanted to assemble 1 2 3 4
and manipulate data to Metadata
Solution Business Performance Flexible Data Analysis Contextualized
explore cause and effect, Transparency for
Components Question Mapping Architecture Information Access
likelihood of recurrence, and Power Users
predictive analysis, but in our old
model that meant intervention by
IT. We needed to find ways to let
them do this themselves.”
Rob Jester
Director, Enterprise Business
Architecture, Sales and Marketing
GSK
Product
3
Users can access and segment data through
high-quality visualizations to get fast answers
to their top business questions.
Data Sources 1
Data sources are selected based on the business
questions documented by the data map.
Rx Enterprise
Prescriber CRM
Sales Data
DB
DB Warehouse
■■ Metadata and additional Home Data Glossary Subject Areas Standard Reports Data University
Users who need more detail or Users can access training materials
clarification can easily contact and information about registering
data owners. for upcoming training.
By providing the
tools and contextual
Faster Assembly, Less Help Required
information needed for
self-service, GSK reduces
the time knowledge
workers spend
assembling information.
Early Benefits
Example: Time to Complete Complex
Analysis on Rx and Claims Data
Patient
Claims
Payer
IT and the rest of the business share an
Rx
enterprise-wide view of customer data needs.
30 Days
Traditional New
Approach Approach
CIO0361911SYN
Usability can be
improved by providing
USABILITY = Bounded FLEXIBILITY AND
knowledge workers
greater flexibility in
TRANSPARENCY
tools choice, information
access, and use.
CXO KPI
Identification
Data Cubes
Dashboards
Standard
Reports
Low
One Size Choice Shaped
Fits All by Business Need
Approach to Usability
Overview
A methodology to segment CXO and operational information needs to target the appropriate level of information
depth, quality, and refresh rates for each
EXECUTIVE TEACHING
CXOs’ information needs differ from operational management needs. Executives require a broader perspective, need
speed more than information quality, and have frequently changing needs; operational information needs are narrower,
deeper, and less dynamic. Consequently, a single KPI framework and a single approach to information management will
not meet the needs of both.
Company Snapshot
PepsiCo, Inc.
Industry: Food and PepsiCo offers the world’s largest portfolio of billion-dollar food and beverage
Beverages brands, including 19 different product lines that generate more than $1 billion in
Annual Revenue: US$60 B annual retail sales each. PepsiCo’s main businesses—Quaker, Tropicana, Gatorade,
Employees: 294,000 Frito-Lay, and Pepsi Cola—also make hundreds of other enjoyable and wholesome
foods and beverages that are respected household names throughout the world.
With net revenues of approximately $60 billion, PepsiCo’s people are united
by the company’s unique commitment to sustainable growth by investing in a
healthier future for people and the planet, which the company believes also means
a more successful future for PepsiCo.
Source: http://www.hoovers.com.
CIO Executive Board
IT Practice
www.cio.executiveboard.com Flexible,
CXO KPI
Introduction Tools Portfolio Transparent The IT Skills Appendix
Identification
CIO0361911SYN Information Access
Where Necessary, Speed to Insight Over Quality and Depth Are Critical
Quality and Depth
■■ Use the KPIs as a guide Broad 1. Prioritize Sector-Level Information Standards of Enabling CXO KPIs
to avoid overinvesting in Perspective Most information does not need to be harmonized across countries or
enterprise-level information
functions. The CXO KPIs indicate where harmonization is needed.
standards or reports.
2. Integrate Internal and External Information
CXO KPIs often combine internal and external information.
Highlight information
gaps to drive to
Highlight (Don’t Hide) the Gaps
consensus on
priorities for systems Analysis of Sources of Information Gaps
harmonization or Illustrative
integration.
Supply
Vendor A Vendor B Vendor B Vendor A
Chain
Commercial Vendor C
Gap
By focusing narrowly on
CXO information needs,
Better Decisions, Faster
PepsiCo International
significantly cut the Time to Respond to Requests for New Reports Reduction in Report Volume
time needed to gather Illustrative
new information and
the number of legacy
reports. 8 Weeks
1. Immediate elimination
of 1,000+ legacy reports
2. Ongoing discipline to
< 1 Week refresh or retire reports
Simon Black
European Commercial Transformation Director
PepsiCo
Middle Management
A user-experience
specialist collaborates
User-Experience Specialist
with service managers
and end users to
Key Responsibilities New-to-World Skills
understand and improve
user experience and ■■ Analyzes business and functional requirements ■■ Information visualization
workflow for new and ■■ Creates user-centered design ■■ User-behavior analysis
existing applications.
■■ Improves the user experience ■■ Usability design
■■ Visualizes and presents information in a user- Other Important Skills
■■ The role designs and friendly manner to end users
configures user-centric ■■ Agile development
interfaces for in-house
and cloud applications.
■■ Business domain analysis
■■ Functional requirements analysis
■■ This enables end users ■■ Information architecture development
to access, visualize, and
navigate information and
■■ Requirements management
analytics with ease. ■■ Usability design
Individual Contributor
For full details of this role,
see the user-experience guru
job descriptions on the CIO
Executive Board website.
Make Usability a Core IT Competency: IT requires new skills and processes to improve the usability of information and analytic tools. Drive knowledge worker
productivity by letting them choose the tools they find easiest to use, by offering greater transparency into quality levels, and by increasing their ability to
navigate and analyze information from diverse internal and external sources.
2. Information presentation aggregation and filtering are top priorities. Investment in advanced analytics is prioritized over presentation.
5. KPIs are defined by reframing strategic questions. KPIs are rolled up from operational metrics or cascaded down from strategy.
6. Speed to insight is prioritized when enabling CXO KPIs. No KPIs are enabled until the underlying information is complete.
CIO0361911SYN
CIO0361911SYN
Knowledge workers
are universally more
APPENDIX: Keeping Things Close to Home
confident about the
quality of information Average Knowledge Worker Confidence in Information Quality from Their Own Function
from their own function and in Information from Other Functions
than they are about
information from other 68%
High Confidence in Information from Own Function
functions.
62%
High Confidence in Information from Other Functions
56%
54%
48%
45% 44%
38%
34% 35% 34%
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
Source: CIO Executive Board, “Process Breakpoint Identification” Exploiting IT’s Business Process Vantage, Washington DC, The Corporate Executive Board, 2006 p. 50.
CIO0361911SYN
Overcoming the Insight Deficit
1. E
nable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Specialists
CIO0361911SYN
77
Overcoming the Insight Deficit 78
Unquestioning faith
in analysis can be as
Why Balancing Analysis With Intuition Matters
damaging as overreliance
on intuition.
Reliance on Intuition
Low
Low High
Reliance on Analysis
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Knowledge workers fall
into three categories
The Value of Big Judgment
based on their relative
reliance on analysis
and intuition.
Visceral Decision Maker
■■ Informed Skeptics like 19% of the sample
High
to make biases and Average Insight IQ1 = 45
assumptions explicit and
often question their own
thinking; they listen to ■■ Seldom trusts analysis
alternative points of view ■■ Has weak analytic skills Informed Skeptic
without defaulting to ■■ Makes decisions 38% of sample
consensus-based decisions. unilaterally Average Insight IQ1 = 60
■■ Applies judgment
■■ The most common profile
Reliance on Intuition
to analysis
is the Unquestioning ■■ Has strong analytic skills
Empiricist, who tends to rely ■■ Listens to others and
excessively on analysis, so
is comfortable with
in many organizations the
dissent
greatest risk comes from too
much analysis, not too little.
Unquestioning Empiricist
43% of the sample
■■ Visceral Decision Makers ■■ Trusts analysis over Average Insight IQ1 = 48
are more rare but represent judgment
almost one-fifth of ■■ Supplements analytics
knowledge workers. with precedent
■■ Values consensus
■■ The profiles are determined
using cluster analysis to Low
identify common groups of Low High
analytic skills and behaviors. Reliance on Analysis
1 Insight IQ is measured on a 0 to 100 scale, with higher values indicating higher Insight IQ.
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
17%
21%
69%
50%
41%
32%
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Unquestioning
Empiricists represent
No Function Is Immune from Overreliance
at least one-quarter of
knowledge workers in
on Data
every function.
Knowledge Worker Profile Distribution
Percentage of Knowledge Workers
■■ Informed Skeptics represent
the majority in General
Management and Corporate
Strategy.
25% 31%
30%
36% 39%
■■ Unquestioning Empiricists 41% 43%
47% 50% 48%
most heavily reside in
60% 57%
Marketing, R&D, and HR.
25% 12%
19%
■■ Visceral Decision Makers
are most heavily represented 14%
30% 23% 18%
in Sales, Communications, 16%
18% 14%
and Customer Service.
14%
18%
58%
50% 51%
47%
35% 36% 37% 37% 39%
34%
29%
22%
g
&D
en s/
ce
IT
ns
gy
ga
en
le
ic
tin
H
em er
io
t
te
Sa
rv
R
Le
em
na
ke
ur Op
at
ra
Se
Fi
ic
ar
ag
St
oc /
un
er
Pr ion
M
an
te
m
m
t
ra
M
to
uc
om
po
al
us
od
er
C
or
C
Pr
en
C
G
Informed Skeptics Visceral Decision Makers Unquestioning Empiricists
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Informed skepticism
can be fostered through
WHO DOES WHAT TO BUILD INFORMED SKEPTICS?
a combination of
decision-making models,
analytical skills training
and coaching, and tools
for collaborative decision
making. Activities and Responsibilities for Fostering Informed Skepticism
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Collaborative tools can
enable more effective
IT Role: Tools for Collaborative
decision making. Decision Making
■■ Collaborative decision Essential Features of Collaborative Decision-Making Tools
making, listening to others,
and accepting dissent are Wisdom of Crowds—Find and connect people Annotations and Note-Taking—Add context
characteristics of informed across the organization to get broad input into to data by annotating reports and integrate all
skepticism. ad hoc decisions. comments in a note-taking platform.
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
A knowledge worker’s
employment history and
Shared Role: Analytical Skills Training
educational background
have less impact on their
and Coaching
Insight IQ than providing
access to training and Maximum Impact on Insight IQ1 of Knowledge Worker Training, Support, and Background
analysts.
Factors that can be influenced in the
24.7% near term have far greater impact than
long-term factors such as educational
background or employment history.
18.6%
2.4%
0.3%
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Effective information
and analytics training
Only 25% Find Information and Analytics
is a strong driver of
Insight IQ, but only 25%
Training Effective
of knowledge workers
experience effective Attendance and Effectiveness of Information and Analytics Training
Percentage of Knowledge Workers
training.
Total
Knowledge 100%
Workers
No Training 36%
Available
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Training on how to
28.7%
conduct analysis has more
than twice the impact on
Insight IQ of any other
training subject.
13.2% 13.7%
1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively
“high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of
multivariate regressions with controls.
Source: Corporate Executive Board, IT Practice, Insight IQ Diagnostic, 2011.
CIO Executive Board
IT Practice
www.cio.executiveboard.com
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Information Training Portfolio
Overview
A training portfolio that provides ongoing support to knowledge workers for conducting analysis and using analytics
tools, reports, and dashboards
Executive Teaching
Training on new tools, without ongoing support and reinforcement, is ineffective as knowledge workers struggle to
apply the training to their jobs. Training that focuses on the functionality of the tools and does not cover information
sources or analytical techniques is also ineffective. Adopt a multipronged approach to design training offerings that
support knowledge workers across the tool lifecycle and focus on conducting analysis.
Company Snapshot
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
IT struggles to meet
rapidly increasing
Tools Training Falls Short
demand from
knowledge workers
for deeper support on
using information and
conducting analysis.
IT Challenges Up-front, one-size-fits-all training doesn’t Knowledge workers fail to capture value
recognize varied knowledge worker needs and because they don’t understand the data or
■■ Most existing training learning styles. know how to conduct analysis.
programs are triggered
by the launch of a new tool.
Cost of Failure ■■ Limited training penetration due to unmet ■■ Under-utilized or misused analytics
■■ To meet this increased varied needs capabilities
demand for analyzing ■■ Lower tool adoption ■■ Unexpected support costs
information, IT ends up
providing ad hoc training
and support.
Solution 1. Make training an ongoing process with a portfolio of training programs mapped across
a tool’s lifecycle.
2. Deliver customized training programs using a variety of channels that map to diverse
knowledge worker needs and preferences.
3. Provide training on understanding information and conducting analysis, not just
using tools.
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Provide a portfolio of
training offerings that
INFORMATION TRAINING PORTFOLIO
supports knowledge
workers across the tool Illustrative
lifecycle. Training Focus Areas
Provide
Build Introduce New Train on Provide
■■ Effective analytics training Support for
Awareness Analysis and New Tool Support
Analysis and
is designed as an ongoing Type of Training of New Tool Information Functionality for Tools
Information
process, not just an event
tied to the launch of a new BI Day ■■ On-site training days on BI/
tool. analytics tools, new subject
matter, reports, dashboards,
query techniques
■■ Offering a range of training
options and channels Video ■■ Online training modules for all
ensures that training maps Training staff globally
■■ Includes training on data
to diverse knowledge worker
subjects as well as a series of
needs and preferences. training on new data marts
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Hire and incentivize
analytics staff on their
HIRE QUANTS WHO CAN COACH
effectiveness at coaching,
not just analyzing. Maximum Impact on Insight IQ1 Skills That Organizations Rate as Most
of Hiring Criteria for Analyst Skills Important When Hiring Analysts
Percentage of Organizations
■■ Although organizations rate
business knowledge skills as
nine times more important
when hiring analysts than
coaching skills, business Business
14.8% Knowledge 45%
knowledge skills provide no Skills
opportunity for impact on
Insight IQ.
6.8%
Quantitative 36%
■■ See page 98 for interview Skills
questions to use to test Quantitative
for coaching skills when 0.0% Skills
recruiting new employees
Coaching Technical Business
and page 99 for an example Skills Skills Knowledge
Technical
9%
of a training program to Skills
Skills
develop coaching skills.
(11.1%) Coaching 5%
Skills
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Develop specialized
information analysis and
Role Profile: Information Insight Enablers
insight skills to bolster
IT’s ability to support Profile of the Information Insight Enabler Role
knowledge workers.
Key Responsibilities New-to-World Skills
■■ A subset of the skills ■■ Market and competitor analysis
■■ Understands the decision-making process and
required in this role are the ■■ Information insight generation
the workflows of business unit heads and service
same as those often found in ■■ Information visualization
managers
data analysts; consequently, ■■ Unstructured information analysis
data analyst is the most ■■ Identifies knowledge workers’ information needs
common background for the Other Important Skills
■■ Represents information in a user-friendly manner
information insight enabler
■■ Identifies trends and patterns; generates insight ■■ Business domain analysis
role.
for business units and senior leadership ■■ Coaching and Training Skills
For full details of this role, ■■ Develops framework and process to analyze Data warehousing
■■
■■
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Informed skepticism can
be fostered through the
Business Leadership Role: Decision-Making
introduction of decision
models and behaviors
Models and Behaviors
that encourage inference
and judgment. Maximum Impact on Insight IQ1 of Decision Models and Performance Criteria
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
An insight-driven culture
is best realized when
Sample DECISION-MAKING Models
business and decision-
making processes are Essential Features of Decision-Making Models That Promote Informed Skepticism
formalized.
■■ Not all decision-making Formalized Methodology—Mitigate biases and Nonlinear Methodology—Adapt to change by
models will promote foster scalability by introducing a systematic incrementally following decision steps that can
informed skeptics (e.g., decision-making process. be tested and revised at short intervals.
models such as Six Sigma
and lean that focus on
reducing variation and Critical Thinking—Improve decision making by Incorporate the Unknown—Actively engage
ambiguity are less likely to combining judgment and common sense with with uncertainty by separating the known from
lead to informed skepticism). normative decision-making processes. the unknown in the decision-making process.
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Informed skepticism can
be fostered by setting
Sample Performance Criteria
the right expectations
through performance Proficiency Criteria Leadership Criteria
criteria.
■■ Competency models and ■■ Uses analytics to identify trends and works ■■ Creates an analytic culture and drives usage
performance criteria should to understand the driving factors by building analytics into day-to-day work
also describe the analytical, ■■ Uses analytics in developing plans ■■ Holds team accountable for using analytic tools
skeptical, and collaborative to achieve target business results and encourages full but skeptical usage
behaviors needed for
■■ Shows good judgment in interpreting ■■ Encourages a collaborative culture and is open
informed skepticism.
the results of analytics to dissent
■■ Collaborates with others in analysis
and decision making
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Foster Informed Skeptics: If knowledge workers cannot apply judgment to analysis, greater information access may do more harm than good. Therefore, IT and
business leaders must both promote a skeptical attitude toward information and analysis through decision models, performance criteria, tools, and training.
2. Decision making is supported by collaborative tools. There are no tools for collaborative decision making.
5. Training focuses on using information and conducting analysis. Training focuses on using the tool.
Analysts are hired and evaluated solely for their analytical skills and domain
6. Ability to coach is a key job requirement for analysts.
knowledge.
CIO0361911SYN
Foster critical thinking
skills to increase
Appendix: Critical Thinking Skills
informed skepticism.
4 Relevance
How does the data relate to the problem?
Depth
5
What are some of the difficulties that may arise
that we may need to deal with?
Breadth
6
Do we need to look at this problem from another
perspective?
Logical
7
Does the data make sense together?
Significance
8
What is the most important problem to consider?
Fairness
9 Do you have biases that will affect the outcome?
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Identify candidates
with coaching skills
Appendix: Identifying Coaching Skills During
by asking them to
give examples of the
Interviews
following behaviors
Please describe a time when you...
and competencies.
1. Assessed Development Progress: Reviewed 8. Gave Feedback on Personality Strengths: Gave
where an employee stood against their an employee feedback during a performance
development goals review about their personality strengths
2. Helped to Draft an Individual Development Plan: 9. Gave Advice from Your Own Experience: Gave
Helped an employee create a plan to meet their an employee advice based on your experience
development goals
10. Helped an Employee Apply New Skills/
3. Ensured That an Employee Had the Necessary Knowledge: Helped an employee apply new skills
Skills/Knowledge: Ensured that an employee or knowledge in their work
had the required skills and knowledge before they
11. Helped an Employee Find Training: Help an
needed it
employee find the training they need
4. Ensured That a Project Provided Learning:
12. Passed Along a Development Opportunity:
Ensured that each project or assignment was
Passed along relevant development opportunities
a learning experience
to an employee
5. Explained Performance Evaluation Standards:
13. Passed Along Job Openings: Passed along
Helped an employee understand their
information about internal job openings to an
performance review standards
employee
6. Gave Feedback on Performance Strengths:
14. Provided an Employee with Experiences
Gave an employee feedback during a performance
That Develop: Helped an employee obtain the
review about their performance strengths
experiences at work that helped them develop
7. Gave Feedback on Performance Weaknesses: over time
Gave an employee feedback during a performance
15. Taught a New Skill or Procedure: Taught an
review about their performance weaknesses
employee a new skill, concept, or process
Source: CLC Learning and Development research; CIO Executive Board research.
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
Use this process to help
employees learn how to
Appendix: A Formula for Coaching
be effective coaches. Success
BCBSMA’s Learning Process for Coaching
Focus on
the “Right”
Coaching Style
+ Focus on the
Most Impactful
Coaching Activities =
Increased
Coaching
Effectiveness
1 DISC refers to dominance, influence, steadiness, and conscientiousness.
CIO Executive Board
Learning and Development Roundtable, “Awareness-to-Action Coaching Toolkit” Profiles of Coaching Programs and Tactics, Corporate Executive Board, Washington DC, 2007, p.29
IT Practice
www.cio.executiveboard.com
CIO0361911SYN
Introduction Informed Skepticism Who Does What Appendix
CIO0361911SYN
Overcoming the Insight Deficit
1. E
nable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Specialists
CIO0361911SYN
101
Overcoming the Insight Deficit 102
Collecting knowledge
worker feedback on
IT PAYS TO LISTEN
information sources and
analytic tools has a 24.8% Maximum Impact on Insight IQ1 of Knowledge Worker Feedback
maximum impact on
Insight IQ.
24.8%
24.8%
16.5%
16.5%
1 The maximum impact on Insight IQ is calculated by comparing two statistical estimates: the predicted impact when a knowledge worker scores relatively
“high” on a driver and the predicted impact when a knowledge worker scores “low” on a driver. The effect of each driver is modeled using a variety of
multivariate regressions with controls.
■■ Progressive organizations 1
Information
Usability Surveys
Requirements
Gathering
Low
Feedback Proactive
Collection Observation
Approach to Observation
1 Pseudonym.
CIO0361911SYN
“Prosumer” Sounding Board
Overview
A methodology to observe knowledge workers (both early adopters and mainstream users) to surface unarticulated
and emerging information needs
EXECUTIVE TEACHING
Segment knowledge workers based on their information usage patterns and their stance toward technology adoption,
then customize engagements to learn from each segment. Observe early adopters as an early-warning mechanism for
emerging needs but balance this with input from the mainstream.
Company Snapshot
Selection Criteria
■■ The Digital Worker Office ■■ Independently solved technology constraints (e.g., obtaining access to enterprise data or e-mail
at Ford engages a subset on personal mobile device)
of the prosumer segment
to surface productivity pain
■■ Comfortable with voicing concerns or dissatisfaction with IT
points and drive innovation. ■■ Understands the economics of niche versus enterprise solutions
■■ Demonstrates the drive to use new tools IT implemented
■■ To prevent technology
evangelism, the facilitator
■■ Has a track record of recommending new technologies to IT
guides users to diagnose the
root cause of the underlying
problem they are trying to
Progressive User Sounding Board
solve with technology.
Description
The sounding board is a global, cross-functional
group of 12 users that meets monthly.
Role
■■ Articulate pain points.
IDENTIFICATION
OVERView
The use of anthropological techniques to discover hidden information needs and opportunities for productivity
improvement, by observing knowledge workers in their own environment and conducting open-ended interviews
Executive Teaching
Traditional approaches to requirements gathering fail when assessing knowledge workers’ information needs because
the needs are fast-changing, diverse, and difficult to articulate. Alpha Company adopts an anthropological approach to
opportunity identification. By observing knowledge workers in diverse settings and conducting open-ended interviews,
Alpha Company gains a deeper understanding of knowledge worker needs and proactively identifies pain points
in end-user workflows.
Component Teachings
Company Snapshot
Alpha Company
2010 Sales: > US$10 B Alpha Company is a leading technology provider.
1 Pseudonym.
CIO Executive Board
IT Practice Approaches to Anthropologically
www.cio.executiveboard.com Observe the IT Skills
Introduction Opportunity Driven Need
Progressive Users Implications
CIO0361911SYN Identification Identification
While knowledge
workers often lack the
CAN THE KNOWLEDGE WORKER SPEAK? 1
ability to articulate
their information needs,
these needs also differ
across individuals,
organizational cultures,
Needs
and collaborative Identify Opportunities for Productivity Surface Unarticulated Knowledge Workers’
Identification
practices. Improvement Information Needs
Scenarios
Challenges Information needs are context dependent and Knowledge workers lack the ability or inclination
vary across individuals, organizational cultures, to communicate their workflow or information
and collaborative practices. needs.
“Anthropological
research does not start
from an ideal state to
be achieved. Instead, it provides
input to process redesign based
on actual conditions on the
ground.”
Corporate Anthropologist
Alpha Company
1 Pseudonym.
CIO Executive Board
IT Practice Approaches to Anthropologically
www.cio.executiveboard.com Observe the IT Skills
Introduction Opportunity Driven Need
Progressive Users Implications
CIO0361911SYN Identification Identification
Traditional process-
centric workflow
FROM WORKFLOW MAPPING TO ANTHROPOLOGY 1
■■ Assumes there is only one best known ■■ Assumes there may be several context-
■■ Most requirements-gathering
method of working specific and equally valid ways of working
techniques assume that
there is only one right way ■■ Uses standardized process discipline with ■■ Uses observational techniques to identify
to do something, and that pre-set rules for mapping workflows roadblocks in day-to-day workflows and
this is the best solution de-familiarization to mitigate biases
for all knowledge workers ■■ Solutions for knowledge worker needs
in all contexts. aligned to stated needs and a predefined ■■ Solutions based on unarticulated
business case or unstated knowledge worker needs
Use anthropological
observation and
OBSERVE AND ASK ASSUMPTION-FREE 1
interviewing techniques
to surface unarticulated
QUESTIONS
and context-dependent
knowledge worker needs. Overview of Common Anthropological Techniques
■■ Gain insights into knowledge Scenario: Help sales reps frame needs in terms of their
worker requirements underlying challenges in accessing information, rather than
through direct observation the technologies they need.
in different user settings,
organizational contexts,
and across points of time.
IT organizations can
embed anthropological
THE ANTHROPOLOGIST INSIDE US
skills in several existing
or emerging roles.
Use Observations of Knowledge Workers in Addition to Observations from Knowledge Workers: Adopt an anthropological approach to identify opportunities
and deliver analytic capabilities successfully. Identify and learn from early adopters, observe segments of knowledge workers across the organization and in
diverse user settings, and conduct open-ended interviews to identify bottlenecks and pain points.
1. IT is proactive in observing knowledge worker information needs. IT listens to knowledge workers only when gathering requirements.
3. IT learns from early adopters about new analytic functionality. IT restricts early adoption of new functionality.
4. IT observes knowledge workers in diverse user settings, locations, IT gathers requirements for information and analytic tools primarily
and points in time. through business process mapping.
CIO0361911SYN
1. E
nable 2. S
upport Big Judgment, 3. O
bserve and Question
Information Usability Not Just Big Data to Identify Opportunities
Key Insights from ■■ Provide a portfolio of analytic tools, as a ■■ If knowledge workers cannot apply ■■ Create opportunities to improve
the Diagnostic single tool cannot meet the needs judgment to analysis, greater access knowledge worker productivity via
of all knowledge worker segments. to information may do more harm than observation, not by merely asking
■■ Visualize how information from multiple good. for requirements.
sources can be combined to answer key ■■ Foster informed skepticism through ■■ Identify and learn from early adopters.
business performance questions. tools for collaborative decision making,
■■ Rather than seek “perfect information,” training, and new decision models.
in all cases, make information quality
transparent and targeted.
Case Examples
Specialists
CIO0361911SYN
CIOs face a range of
options for central or
Information Management—How to Organize,
local management of
information and for
Where to Locate?
management within IT
or outside.
Centralized
Diffused (Integrated
(Dedicated)
with Other Activities)
Managing information
Information should be
centrally avoids
managed close to the
duplication, promotes
people who use it to
integration, and
allow for speed and
provides a single view
flexibility.
of the enterprise.
Key Questions
1. What is the right
Business Ownership
degree of centralization
Only information
for information
producers and users Business/ Business/ management and
understand its meaning Centralized Diffused analytics?
and value and only they
can improve quality. 2. How should
responsibility be
divided between
business areas and IT?
IT Ownership
IT manages the IT/ IT/
information sources Centralized Diffused
and can offer a neutral
perspective.
CIO0361911SYN
Introduction Information Management COEs Skills Implications
Appendix 117
Overcoming the Insight Deficit 118
6.9% 6.7%
0.0% 0.0%
Centralized Centralized Centralized Centralized Centralized Single Single Data
Analytics Knowledge Information User Support Information Unstructured Warehouse
Team Worker Quality for Analytic Architecture Information
Training Tools Repository
CIO0361911SYN
Introduction Information Management COEs Skills Implications
There is no one ideal
model for an analytics
Where Should An Analytics COE Reside?
center of excellence as
the optimal location Factors Determining Organizational Location of Analytics Center of Excellence (COE)
depends on a number
of organizational factors.
CIO0361911SYN
Introduction Information Management COEs Skills Implications
Appendix 119
Overcoming the Insight Deficit 120
Information Architecture
Data Warehousing
Reporting
Analytics
Information Stewardship
Examples
IT–Owned Activities
Owned Elsewhere in the Corporate Center
A. E
ach major business unit
is represented on the Data
Governance Council; the CIO
council was set up as a
business council, and IT’s
participation on the council
A
is purely as one of the Enterprise
represented business units. Data Governance Business Factory
Business
Council Applications Systems
Intelligence (EBI)1
B. EDW and BI user support
team addresses questions
related to both business EBI Asia
intelligence tools and content. EDW Content and Business
Support and
Data Governance Intelligence
Development
C. B
I solutions team develops
reports and tools for users; as C
EDW
there is a strong focus on user Asia Business
Content and BI Solutions
self-service, the team typically Intelligence
Architecture
builds templates for users to
adapt rather than providing D D
ongoing maintenance and Real-Time and Real-Time and
PMO and Data BI Strategy and Asia EDW
support. Application- Application-
Modeling Architecture Content
Specific Specific
D. A
pplication-specific and Reporting Reporting
B
factory-related reporting is
run outside the EBI group to EDW and BI Advanced
cater to special requirements User Support Analytics
of these groups.
Reports to IT
1 Quality and engineering applications support teams are not shown on this chart but they also report into the executive director,
Enterprise Business Intelligence.
CIO Executive Board
IT Practice
www.cio.executiveboard.com
CIO0361911SYN
Introduction Information Management COEs Skills Implications
Appendix 121
Overcoming the Insight Deficit 122
Information management
and analytics
II. PAIRED IT AND BUSINESS COEs
responsibilities are
divided with a business Information Management and Analytics Organization Structure
COE acting as an
interface between IT
and the rest of business. Analytics Resource Center (ARC) Data Analysis COE
Enterprise Information
■■ At point A, touchpoints
Committee
with IT are maintained
through representation in Analytic and Information
committees and through Information Management
Management
regular meetings with the IT and Analytic Services
Advisory Committee
staff and CIO.
Analytics
■■ At point B, a few technical Center of Analytic
information management– Excellence Analytic Continuous
Services and A
related roles such as data Delivery Improvement
Solutions
integration remain outside
B
of the COE; information Information
architecture responsibilities Informatics Analytic Strategy and
are represented on both Solutions Services Architecture
sides.
B
Solutions Informatics Portfolio and Data Integration
■■ BCBSNC’s COE is being Informatics
Portfolio Solutions Process and Production
built iteratively, first by Analysts
Analyst Analysts Analyst(s) Support
including the transactional
activities (e.g., data Informatics
Support
sourcing, reporting, Solutions
training) and some critical Developers
pure analytics functions Technology
(e.g., group analytics, care Architect
management).
Appendix 123
Overcoming the Insight Deficit 124
Info. Architecture
Info. Integration
Business Analysis
Info. Visualization
Analysis/Insight Generation
For a description of the key information management and analytics skills, please see page 125.
CIO0361911SYN
Introduction Information Management COEs Skills Implications
Implementation Guide: SKILLS Descriptions
Analytics Service The skills required manage analytics as an end-to-end service from service strategy to service delivery
Management and improvement
Database Design The skills required to produce a detailed data model for a database
Database Management The skills required to create, access, and maintain a database
Information System Design The skills required to analyze and design information systems, concentrating on entities and their attributes and
interrelationships; also includes data modeling for individual databases and the corporate data model, as well as coordinating
the definition of data across multiple distinct databases
Information Architecture
The skills required to define how information and information systems support the objectives of the business
and Development
Information Integration The ability to integrate information and content from multiple sources and channels
Business Analysis The skills required to identify business needs and determine solutions to business problems
Information Visualization The ability to depict information in intuitive and insightful ways
Information Policy
The skills required to establish guidelines for information dissemination and usage
Formulation
Note: For detailed analysis on the emerging skill gaps and role changes for IT, please ask the CIO Executive Board staff or your account manager for a copy of our recent study on The IT Talent
Implications of the Future of Corporate IT. You can also download this study from www.cio.executiveboard.com.
Appendix 125
List of Questions Asked in
the Insight IQ Diagnostic
CIO0361911SYN
126
I. Information Management Structures and Processes
CIO0361911SYN
I. Information Management Structures and Processes (Continued) II. Training and Support
18. Do you use cloud applications for generating information? maintained in pockets of solutions or documents, typically 1. Is there an analyst available to you to help you find
at a project level. information or conduct analysis?
19. Do you use cloud applications for storing information?
–– Consolidation of some business definitions or technical –– There is an analyst embedded directly in my team.
20. Do you use cloud applications for analyzing information? information occurs, but governance processes are –– There is a team of analysts in my business unit or function.
21. What capabilities are in place at your organization to ease immature. –– There is a companywide central team of analysts.
the capture of unstructured information in shared systems? . –– Business and IT accountabilities and processes are –– There are informal analytical experts on my team.
–– Automated processes (e.g., automated tagging, document starting to be formed to manage repositories of metadata. –– There is no one available to me for help in finding
uploads) –– Business and IT are utilizing common metadata across information or conducting analysis.
–– Rules and guidelines on what information to capture and departments and projects to improve understanding.
2. Does your organization provide training on finding
tag –– Formal processes and accountabilities are in place for
information or conducting analysis?
–– Rules and guidelines on how to capture information and managing official repositories of metadata utilized for
guiding information use. 3. If you attended training provided by your organization on
tag
finding information or conducting analysis, how effective or
–– Dedicated team or individual responsible for capturing, 25. Do rules exist around types of information that could
ineffective was that training in the following areas?
tagging and transferring information into shared systems be shared between levels and/or different parts of the
–– Finding information I need
–– Other organization?
–– Information definitions and terminology
–– None
26. How would you assess your organization’s current level –– Conducting analysis
22. For which of the following does your organization have of proficiency in the following Enterprise Architecture –– Communicating and displaying information
defined processes in place? disciplines? –– Using the functionality of the tools provided
–– Create/Acquire –– Business Architecture –– Where/how to get help
–– Store/Share –– Information Architecture
4. How formalized is the process for making decisions in your
–– Find/Use –– Applications Architecture
organization?
–– Archive/Retain –– Technical Architecture
–– Non-existent: No decision making processes exist.
–– Dispose
–– Ad hoc: Individuals and/or teams have processes that are
–– Manage/Control
followed irregularly.
–– None
–– Undocumented: Individuals and/or teams follow processes
23. How do you define metadata in your data warehouse? that are not standardized nor documented.
–– For all information types –– Defined: Processes used across business units and/or
–– For information that is shared between functions functions are documented but not enforced.
–– For information that is shared between business units –– Managed: Processes are documented and enforced.
–– For information that is deemed critical to the business –– Optimized: Processes are documented, enforced, and
–– For information that is used most frequently measured to drive improvement.
–– For information that is stored in multiple systems or
5. Which of the following are in place to improve decision
repositories
making within your organization?
–– For information where there is frequent confusion
–– Automated decisions
regarding the definition
–– Collective intelligence tools
–– For unstructured information
–– Decision making team structure based around diversity
–– For no types of information
and independence
24. Which best summarizes your current level of metadata –– Formal decision frameworks or guidelines
management maturity? –– Other
–– Incomplete business and technical metadata are –– None
CIO0361911SYN
III. Knowledge Worker Attitudes and Incentives
1. Are you evaluated in your performance review on your ability 11. How significant a contributor to your personal analysis 15. How important is information and/or analytics in enabling
to conduct analysis effectively? and decision making are each of the following types of you to do the following?
information? –– Make Better Decisions
2. To what extent do you agree with the following: I enjoy
–– Centrally provided information available to and used –– Make Faster Decisions
conducting analysis?
across the organization –– Improve Your Job Performance
3. To what extent do you agree with the following: If I conduct –– Centrally provided information available to and used
16. How frequently do you regret decisions you’ve made where
analysis and disagree with the findings I will rely on my own across my business unit/function
information/analytics contributed to the decision?
judgment instead? –– Information generated by and managed by my
subfunction or team 17. To what extent do you agree with the following: All of the
4. To what extent do you agree with the following: I have
–– Information I personally obtain through external sources information I need to do my job is available to me?
enough time to conduct analysis that is valuable to me in
making decisions in my daily work? 12. Which characteristics do you value most when using 18. To what extent do you agree with the following: In my
information and analytics in your daily work? daily work, I rely heavily on the results of analysis (charts,
5. To what extent do you agree with the following: Expertise
–– Information Accuracy graphics, reports, dashboards, etc.) to support my business
in analytics increases my ability to advance in my desired
–– Information Timeliness decisions, processes and deliverables?
career path?
–– Information Consistency
19. To what extent do you agree with the following: The reports
6. To what extent do you agree with the following: I feel very –– Information Relevance
and dashboards that are pushed to me are very valuable and
comfortable conducting analysis and am able to meet all my
13. How important are the following capabilities when you I read, analyze and use them extensively?
information and analytical needs without external help?
make decisions?
20. To what extent do you agree with the following: The reports
7. When making important decisions in your organization, how –– Visualization—The graphical or visual presentation of
and dashboards that I pulled are very valuable and I read,
critical are each of the following considerations to you, your information (e.g., through static or interactive graphs,
analyze and use them extensively?
direct manager, and your organization’s leadership? charts, maps, images) to make it easier for the users to
–– Information and Analytics understand the information contents and attributes 21. If information you would like to have as an input to making
–– Intuition –– Analysis—The process of examining information with the a decision is not available in a standard report or dashboard,
–– Precedent purpose of drawing conclusions about that information how willing are you to go look for the information?
–– External Advice –– Filtering—The process by which redundant or unwanted
8. How effective is your direct manager in using analytics in information from an information stream is removed and
carrying out his/her job responsibilities? only information that meets specific criteria (e.g., time,
value, user groups) is displayed or presented
9. To what extent do you agree with the following: My –– Aggregation—The process by which information is
performance review is based largely on metrics and gathered and expressed in a summary form, typically
achievement of measurable goals and objectives? combining two or more information attributes or
10. To what extent do you agree with the following: Performance occurrences
metrics and progress against my business unit/function’s 14. How effective a contributor is information and/or analytics
goals and objectives are reported to me on a regular basis? to your organization’s ability to do the following activities?
–– Product or service development
–– Improving product or service profitability
–– Evaluating and prioritizing investment proposals
–– Developing a corporate or business unit strategy
–– New market identification and market strategy
development
–– Identifying and targeting customer segments
–– Forecasting demand for products, services, or resources
CIO0361911SYN
1. What is your organization’s total budget for the following 1. To what extent do you agree with the following: –– There are no means to assess or verify information quality
projects/initiatives/capabilities in 2011? Include all spend “The information from corporate sources is in a usable –– Other
both inside and outside of IT. format and does not require modification or manipulation?” 9. For which of the following has your organization established
–– Business Intelligence standard definitions?
2. To what extent do you agree with the following:
–– Information Management –– Information that is shared between functions
“I have a good understanding of the definitions of the
–– Collaboration –– Information that is shared between business units
information from corporate sources?”
2. At which groups are your analytics investments and –– Information that is deemed critical to the business
3. To what extent do you agree with the following statement: –– Information that is used most frequently
initiatives targeted?
“There is a single version of the truth across information –– Information that is stored in multiple systems or
3. How many staff are part of the information management from all corporate sources?” repositories
group? –– Information where there is frequent confusion regarding
4. To what extent do you agree with the following:
4. Rank the following four capabilities in terms of effort and “The information from corporate sources is directly relevant the definition
investment. to business decisions, processes, and deliverables in my daily 10. Is each of the following information types defined in a data
–– Visualization work?” dictionary?
–– Analysis
5. To what extent do you agree with the following: 11. What steps does IT take to remedy information quality
–– Filtering
“The information from corporate sources is always timely issues (such as duplicates, errors, outliers, contradictions,
–– Aggregation
and up-to-date enough to be useful in my daily work?” inconsistencies, missing values) in relation to the information
5. How are analytics projects and initiatives approved? 6. To what extent do you agree with the following: that is entered or maintained in the data warehouse?
–– Very formal business case review “The information from corporate sources is always precise –– There is an information quality framework that establishes
–– Somewhat formal business case review and accurate enough to be useful in my daily work?” quality standards and guidelines for information to be
–– No formal business case review added or kept in the data warehouse or data marts.
–– Piggyback on other IT/business initiatives 7. What level of confidence do you have in the quality
–– The information is fixed in the source systems before
–– Other (cleanliness and accuracy) of information from each of the
entering it into the data warehouse or data marts.
following?
6. Which of the following investment approval processes do –– The information is fixed both in the source systems and
–– Human Resources
you use for analytics investments and initiatives? during information movement.
–– Finance
–– A business-enabling capital investment –– The information is fixed in the data warehouse or data
–– Legal and Compliance
–– An IT infrastructure investment mart with the fixes fed back to the source systems.
–– Marketing
–– An R&D investment –– The quality of information in our data warehouse is
–– Sales
reviewed or audited on an ongoing basis.
7. How are information management projects and initiatives –– Customer Service
–– The quality of information in our data warehouse or data
funded? –– Supply Chain and Operations
mart is reviewed or improved as and when business users
–– As a business-enabling capital investment –– Research and Development
raise quality concerns.
–– As an IT infrastructure investment 8. For functions where you have low or no confidence in the –– The information quality process is integrated into the ETL
–– As an R&D investment quality of their information, what are the reasons behind process to maintain information quality thresholds.
8. How are information management projects and initiatives your lack of confidence?
12. To what degree are information quality governance and
approved? –– Lack of transparency in how information was collected
standards automated through technology?
–– Very formal business case review –– Lack of transparency in how information was stored
–– Somewhat formal business case review –– Lack of transparency in how information was analyzed 13. Which of the following describes the information quality
–– No formal business case review –– Multiple versions of the same information exist standards within your data warehouse or data marts?
–– Piggyback on other IT/business initiatives –– Lack of trust in systems –– All information has the same standards for information
–– Other, please indicate –– Lack of common vocabulary/standard definitions quality.
CIO0361911SYN
V. Quality of Information (Continued) VI. Information Usage Patterns
–– Information for certain functions (e.g., Finance, HR, supply 16. Who in your organization is primarily responsible 1. How frequently do you use each of the following?
chain) has higher standards for information quality. for enforcing information quality standards and policies? –– Workforce Information
–– Information that is used more frequently has higher –– Business leaders –– Financial Information
standards for information quality. –– Subject matter experts in the business –– Risk Information
–– Information that has been deemed to be more valuable to –– IT –– Customer Information
the business has higher standards for information quality. –– Information Management group –– Product, Materials, and Suppliers Information
–– Information that will be used for certain types of analysis –– Analytics group –– Research, Product Development and Testing Information
has higher standards for information quality. –– Shared services group –– Supply Chain Information
–– Information that will be used by more senior or executive –– No one –– Demand Information
staff has higher standards for information quality. –– Web Information
17. On which of the following characteristics do you tag
–– Information from certain corporate systems has higher –– Consumer and Marketing Information
information in your data warehouse or data marts?
standards for information quality. –– Customer Service Information
–– Individual author of the information
–– Sales Operations Information
14. Which of the following describes the information timeliness –– Function that created the information
–– Economic and Industry Information
standards within your data warehouse or data marts? . –– Date and time the information was created
–– Financial Market and Trading Information
–– All information has the same standards for information –– Cleanliness and timeliness standards for the information
–– Assets and Maintenance Information
quality. –– Relevant governance processes
–– Business Process Efficiency, Service Levels, and Quality
–– Information for certain functions (e.g., Finance, HR, supply –– Frequency information is used
Information
chain) has higher standards for information quality. –– Purpose of the information
–– Information that is used more frequently has higher –– Types of analysis information is used for 2. Which of information types are most important to you in
standards for information quality. –– Types of decisions information is used to make your daily work?
–– Information that has been deemed to be more valuable to –– Standards reports or dashboards information is used in
3. Is your team or function responsible for owning and/
the business has higher standards for information quality. –– Other
or generating a significant proportion of the following
–– Information that will be used for certain types of analysis –– None
information types?
has higher standards for information quality.
–– Information that will be used by more senior or executive 4. How frequently do you consult each of the following in your
staff has higher standards for information quality. day-to-day work?
–– Information from certain corporate systems has higher –– Operational Structured Information
standards for information quality. –– Analytical Information
–– Unstructured Information
15. Who in your organization is primarily responsible for
–– Undocumented information from colleagues or experts
defining information quality standards and policies?
–– External company-endorsed sources
–– Business leaders
–– External non-company endorsed sources
–– Subject matter experts in the business
–– IT 5. How many different sources of information/analysis do you
–– Information Management group use on a weekly or more frequent basis?
–– Analytics group –– Corporate operational systems
–– Shared services group –– Corporate unstructured information systems and
–– No one document repositories, excluding e-mail
–– Reports, dashboards, data/information warehouses, or
data marts On average, how much of your time at work
do you spend finding information or conducting analysis?
CIO0361911SYN
VI. Information Usage Patterns (Continued) VII. Staff Knowledge and Competencies
6. On average, how much of your time at work do you spend –– Significantly higher value due to speed/convenience 1. Is conducting analysis explicitly mentioned in your job
finding information or conducting analysis? of mobile device accessibility description?
–– Somewhat higher value
7. Of the time you spend finding information and conducting 2. Which of the following skills do you most value in staff
–– Same value
analysis, what percentage is spent finding information? responsible for conducting and providing analysis to you and
–– Lower value, as some information is missing
your team?
8. How frequently do you undertake each of the following –– Significantly lower value, as key information is missing
–– Business knowledge and understanding
types? –– N/A—I do not have mobile access.
–– Quantitative skills
–– Descriptive Analytics—The process of using analytics to
15. Do you analyze information obtained from any of the –– Technical skills
describe what happened in the business (using reports
following sources? –– Relationship and advisory skills
and dashboards)
–– Smart Networks and Sensors –– Training and coaching skills
–– Predictive Analytics—The process of using analytics to
–– Radio-Frequency Identification
predict what will happen in the business (using data 3. How frequently do you interact with staff from each of the
–– Global Positioning System
mining, pattern recognition and alerts, Monte-Carlo following functions for work-related purposes?
–– Collaboration Tools
simulation, forecasting, predictive modeling) –– Human Resources
–– External Social Media
–– Finance
9. How much time do you spend obtaining information or
16. Do you provide information to corporate systems for others’ –– Legal and Compliance
conducting analysis that does not ultimately lead to a
consumption (e.g., inputting information into a CRM system –– Marketing
decision or productively inform your or your colleagues’
or sharing information/knowledge within a collaboration –– Sales
work?
system or document repository)? –– Customer Service
10. To what extent do you agree with the following: When I need –– Supply Chain and Operations
17. Do you have the opportunity to provide feedback on
specific information, I know where to find it? –– Research and Development
information sources and business intelligence/analytics tools
11. To what extent do you agree with the following: In instances for improvement or functionality enhancements? 4. To what degree are your decisions made independently?
where someone in my organization has conducted similar –– I make the majority of decisions completely independently.
18. How does your organization determine end-users’ needs for
analysis before, I am able to find documentation, lessons –– I take my peers’ perspectives into account but ultimately
information with regard to tools, capabilities, quality, etc.?
learned, or notes on their process and experiences? make decisions independently.
–– Link information needs to strategic decisions.
–– I make decisions in conjunction with peers to incorporate
12. To what extent do you agree with the following: For –– Derive information needs from strategic goals or targets/
the group’s perspectives.
projects/tasks where someone in my organization has KPIs.
–– Decisions are largely dictated by my manager or other
worked on something similar before, I am able to find –– Define information needs based on workflow processes.
leaders.
documentation, lessons learned, or notes on their process –– Keep up to date of market/industry/functional trends.
and experiences? –– Directly ask knowledge workers via survey. 5. Which of the following best describes the methods you use
–– Directly ask knowledge workers via focus groups. to conduct analysis?
13. Which of the following formats/channels do you use to
–– Track information requests from knowledge workers to –– I develop specific theories that I then test using analysis.
access each information type?
understand trends in information needs. –– I have some theories in mind that I test using analysis, but
–– PC-based web browser
–– Take in feedback from knowledge workers in terms of their also manipulate the information in other ways to see if
–– Mobile devices
satisfaction with information (to inform future needs). interesting trends or insights emerge.
–– E-mail
–– I manipulate the information without a clear idea of what
–– Online system
I expect to find, and see if interesting trends or insights
–– Paper
emerge.
14. If you access information from mobile devices, how do you
rate their value and functionality relative to when accessing
them on a standard desktop or laptop?
CIO0361911SYN
VIII. Information Strategy and Governance
1. Does your organization have a data steward role? 7. Which components and initiatives are included in your 11. Rank order the business motivations for making analytics
strategic plan for analytics and information management? investments .
2. If you responded “Yes, formal data steward role,” how many
–– Formalizing the structure, roles, and responsibilities for –– Product or service development
data stewards does your organization have?
analytics –– Improving product or service profitability
3. Indicate if the following are available to knowledge workers –– Implementing information architecture requirements –– Evaluating and prioritizing investment proposals
through self service. –– Implementing information governance requirements –– Developing a corporate or business unit strategy
–– Data definition –– Implementing information quality standards –– New market identification and market strategy
–– Name of the data steward –– Implementing new analytical tools or capabilities development
–– Source of the data –– Conducting training on using analytical tools –– Identifying and targeting customer segments
–– Details of transformations performed on the data –– Developing knowledge worker analytical skills –– Forecasting demand for products, services, or resources
–– None –– Developing an analytical culture –– Other
4. To what degree do information security and access policies –– Measuring analytical performance
12. Does your Enterprise Architecture group have information
inhibit your ability to obtain information that is valuable/ 8. Are your analytics and information management initiatives architects?
important to your analysis or decision making? aligned to business strategic goals?
13. How many information architects does your EA group have?
5. How mature are business processes within your function/ 9. Which groups are involved in developing the strategic plan
14. How would you assess your organization’s current level of
business unit? for analytics and information management?
proficiency in the following disciplines?
–– Non-existent: No business processes exist. –– IT
–– Data Strategy and Governance
–– Ad hoc: Individuals and/or teams have processes that are –– Finance
–– Data Stewardship
followed irregularly. –– Marketing
–– Data Standardization
–– Undocumented: Individuals and/or teams follow processes –– Corporate Planning
–– Master Data Management
that are not standardized nor documented. –– Business Shared Services
–– Data Quality Management
–– Defined: Processes used across business units and/or –– Business Unit
–– Reporting and Analytics
functions are documented but not enforced. –– Standalone Information Management Group
–– Information Lifecycle Management
–– Managed: Processes are documented and enforced. –– Sales
–– Optimized: Processes are documented, enforced, and –– Supply Chain 15. When IT provides a new report or analytics tool, does
measured to drive improvement. –– Other, please indicate the project plan include formal steps to change business
–– N/A—No Strategic Plan processes or workflow to make better use of it?
6. Does your organization have a strategic plan for analytics
and information management? 10. If IT is involved in developing the strategic plan for analytics 16. Are the implications of information and analytics
–– Yes, we have a standalone strategic plan for analytics and and information management, what role do the following requirements incorporated into other IT projects?
information management. play? –– Yes, before the project has been approved
–– Yes, analytics and information management are –– CIO –– Yes, when building out requirements for the project
incorporated into a broader IT strategic plan. –– Project Management Office –– Yes, after the project has been deployed
–– Yes, analytics and information management are –– Enterprise Architecture
17. Which of the following best describes your organization’s
incorporated into a broader business strategic plan. –– Infrastructure
approach to master data management?
–– No, we do not have a strategic plan for analytics and –– Applications
–– No formal initiative is currently in place or planned around
information management. –– Security
master data management.
–– Dedicated Information Management Team
–– Master data is defined and managed at the individual
–– Other, please indicate:
system level.
–– N/A—IT not involved in strategic plan
–– Siloed, department or group-level master data
management strategy has been defined and established.
CIO0361911SYN
–– A unified, enterprise-level master data management –– Yes, after they are developed or purchased 1. Do you use any document repositories that are not provided
strategy has been defined and established. –– No and supported by corporate IT (e.g., Google documents,
–– N/A—no data model Dropbox, iDisk) for business purposes?
18. Which of the following do you use as part of your master
data management? 24. Does your organization have an information architecture in 2. Do you use any business intelligence/analytics tools that are
–– Source identification place? not provided and supported by corporate IT?
–– Data collection –– Yes—We have documented both the current and future
3. If you answered “Yes,” why do you use tools not provided
–– Data transformation state architectures for the enterprise and have a roadmap
and supported by corporate IT?
–– Normalization in place against which we are moving toward the future
–– IT does not provide an alternative tool.
–– Rule administration state.
–– The non IT provided tool is easier to use.
–– Error detection and correction –– Yes—We have architected the current and future states for
–– The non IT provided tool is more sophisticated.
–– Data consolidation different business units or areas of the enterprise but have
–– The rest of my team uses the non IT provided tool.
–– Data storage yet to aggregate into a unified whole.
–– Other
–– Data distribution –– Yes—We have an initiative in place but are still in the early
–– Data governance stages. 4. Which of the following best describes the analytical tools
–– None –– No—We have a vision for what we want to do but no available to you?
formal initiative. –– There is no tool available to me.
19. Who owns or stewards master data subjects (e.g., customer,
–– No—We don’t have an information architecture and don’t –– I only have access to Excel.
product)?
plan to create one. –– There is a single tool I have access to that is consistent
–– IT
across the entire company.
–– Business unit 25. Which of the following best describes your data governance
–– There is a single tool I have access to that is consistent
–– A central or cross-functional master data management or processes?
across my business unit/function.
governance group –– We have formal governance structures in place that are
–– There are multiple tools with similar functionality I have
–– Other understood and adhered to across the enterprise.
access to and I am free to use the one I prefer.
–– None –– We have formal governance structures in place that
are understood and adhered to, but only in parts of the 5. Which of the following collective intelligence tools are
20. Does your organization have an enterprise data model for
organization. currently being used at your organization?
organizing information?
–– We have some formal and some informal governance –– Discussion forms
21. Which information types are included in the enterprise data processes in place; adherence is mixed. –– Blogs
model? –– Governance processes are generally ad hoc and –– Wikis
adherence is generally low across the organization. –– Prediction markets
22. If your organization does use a data model, which of the
–– There are no formal and few informal governance –– Ratings and online surveys
following components are included in your data model?
processes in place. –– Other, please indicate
–– Data architecture
–– Data modeling 26. Rate the following end user requirements that the 6. Which of the following best describes your organization’s
–– Data properties Information Management group is working to solve in order approach to selecting analytical tools?
–– Data organization of urgency). –– Tools are centrally selected and deployed across the
–– Data structure –– Information quality or accuracy organization.
–– Data model theory –– Information timeliness –– Tools are centrally selected and deployed across the
–– N/A—no data model –– Information consistency organization, but we customize capabilities to meet the
–– Information relevance needs of different functions/business units.
23. If your organization does use a data model, is there a
–– Information accessibility or ability to search –– Tools are centrally selected, but different tools are
process for reviewing new applications, tools, and databases
–– Usability of reporting and analytical tools deployed to different business units/functions based on
for alignment with the enterprise data model?
–– Flexibility to use personally preferred tools, e.g. via cloud their unique needs.
–– Yes, before they are developed or purchased
or mash-ups
CIO0361911SYN
IX. Technical Environment (Continued)
–– Business units/functions choose their own tools with 13. Which of the emerging technologies and capabilities are in contributing to its removal? Please check all that apply.
guidance from IT and the information management and/ use (either broadly or narrowly)? –– No longer required to retain the information for regulatory
or analytics group. and compliance purposes
14. To what extent are analytics and information
–– Business units/functions choose their own tools and IT/ –– Information no longer has business value.
management solutions deployed using a test and learn
Information Management are not/rarely involved in the –– Too costly to continue storing the information
approach?
decision process.
20. Does IT integrate external sources of information into the
–– Individual knowledge workers choose their own tools. 15. Does your organization have an enterprise data warehouse?
data warehouse?
–– Yes—data from both structured corporate systems and
7. How many different analytical tools are in use? –– Yes—we proactively search for external sources of
unstructured information repositories are all stored in a
–– Across the company information that could be valuable to the business and
single data warehouse.
–– Maximum within a business unit integrate it into the data warehouse.
–– Yes—structured information from all corporate systems
–– Maximum within a function –– Yes—when the business recurrently uses an external
are stored in a single data warehouse, but Unstructured
–– Per information type source of information, we integrate it into the data
information is stored elsewhere.
warehouse.
8. How many different operational systems are in use? –– Yes—data from some corporate systems are stored in a
–– Partially—we make some external sources of information
–– Across the company single data warehouse, but there is a significant amount of
available internally, but it is kept separate from the data
–– Maximum within a business unit data outside the data warehouse.
warehouse.
–– Maximum within a function –– No—data is stored in several data marts.
–– No—we do not integrate external sources of information
–– Per information type
16. What is the total number of data warehouses within your into our environment.
9. How many different corporate information repositories are organization?
21. Do you have a master data management tool?
provided?
17. What is the total size of data warehouses within your
–– Across the company
organization in TB?
–– Maximum within a business unit
–– Maximum within a function 18. Which of the following best describes the information that is
–– Per information type included in your organization’s data warehouse?
–– All information generated by our organization is included
10. What is the total volume of storage in your organization
in the data warehouse indefinitely.
(include primary, backup, and archive) in TB?
–– All information generated by our organization is included
11. What is the annual percentage growth rate of total storage in the data warehouse, but it is retired and removed after
volume in your organization (include primary, backup, and a defined period of time.
archive)? –– Only information deemed valuable to the business is
included in the data warehouse.
12. Which of the following emerging technologies and
–– Only certain information is included based on technical
capabilities have you implemented?
feasibility.
–– In-memory analytics
–– Only certain information is included based on business
–– Data mash-ups
requests.
–– Cloud/SaaS BI or analytics
–– N/A—not in data warehouse
–– Columnar databases
–– Mobile BI 19. If your organization removes information from the data
–– Interactive visualization warehouse after a period of time, what are the factors
–– Enterprise search
–– Semantic technologies
–– Complex event processing
–– Advanced analytics
CIO0361911SYN
X. Demographics
1. What is your age? 15. How high is the regulatory burden faced by your
organization?
2. What is the highest level of education you obtained?
16. What is the level of business integration at your
3. In which subject or discipline did you achieve an
organization?
undergraduate- or graduate-level university degree?
17. What is the level of information intensity of each of the
4. What is your tenure at your current organization?
following at your organization?
5. Which of the following most closely describes your current
18. What is the overall annual revenue (in millions of your
level in the organization?
currency) for your organization?
6. Which of the following most closely describes your current
19. To what degree does your organization handle sensitive or
function in the organization?
confidential information?
–– Communications
–– Corporate Strategy 20. How many employees does your organization employ?
–– Customer Service/Contact Center
21. In how many countries does your organization have
–– Finance and Accounting
operations?
–– Financial Services
–– General Management Functions 22. Which of the following best describes your IT
–– Human Resources organization’s structure?
–– Information Technology
23. How many IT users does your organization support?
–– Legal and Compliance
–– Marketing/Market Research 24. How significant a part of your job responsibilities is
–– Production/Operations/Procurement conducting analysis?
–– Professional Services
–– Real Estate and Facilities
–– Research, Development, and Engineering
–– Retail
–– Sales
CIO0361911SYN