Practical Predictive Analytics

The Stepping Stones to Success

Dan Putler (Alteryx)
Cornelius Kaestner (The Boston Consulting Group)
March 6, 2013

We have shaped business thinking for 50 years...

Growth- Share Matrix

Experience Curve

Time-based Competition

Change Monster

Adaptive Advantage

Trading up/Trading down

Emotional

Positional durability

Information flow

High
1950

1955

Low
1960

1965

1970

Manageable
1975

1985

1980

1990

1995

Exabytes 1800

Functional

2000

2005

Industry boundaries
Data overload

Clear

Blurred

Digital Information

1600
1400

DVD,

1200

Digital TV,

RFID,
MP3 players,

1000

Digital cameras,
Camera phones, VoIP,

800

Moores Law (indexed)1

Available Storage

Medical imaging, Laptops,

600

Datacenter applications, Games,

Satellite images, GPS, ATMs, Scanners,

400

Sensors, Digital radio, DLP theaters, Telematics,

Peer-to-peer, Email, Instant messaging, Videoconferencing,

200

Technical

0
2005

CAD/CAM, Toys, Industrial machines, Security systems, Appliances

2006

2007

2008

2009

2010

2011

...including thinking on the value of Big Data

Our thought leadership resonates with our clients

Global revenue (Indexed, 1990=100)
2,000

1,500
+15%

1,000

500

0
1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

2010

2012

Capturing value from Big Data: our framework

Big Data
Strategy

Navigation

Advanced
analytics

Big Data
transformation

Strategic
Analytics

Enterprise
Information
Management

Platform
Analytics

Business Model
Transformation

Alteryx supports our Strategic Analytics efforts

Data Business
Creation

Key principles for our Strategic Analytics efforts

Work with clients who are open to new analytical methods

Follow demand: known challenges our clients are looking to solve

Focus on challenges with significant upside potential
Start where decent data exists
Invest where we can learn the most
Seek pragmatic, implementable solutions instead of perfectly pure analytics

Retail Example: Optimizing circulars

Approach: SKU / event promo analysis

Insight and impact

Incremental Margin

Up to 50% of promotions have no impact

on sales or margin
Significant opportunities to improve value
creation, e.g.,
4% sales opportunity
7% margin opportunity
10% flyer cost reduction

Incremental Sales

Additional insight from the analysis

At one retailer, stores were not consistently
executing promotions
At another, we could identified vendors
who consistently underfunded promotions

Our approach for Strategic Analytics at BCG

Identify talent to
drive Strategic
Analytics
Enable our
organization

Encourage
experimentation

Codify and
share wins

Small team to drive the effort, each with combination of skills

Business understanding to recognize actionable solutions
Analytical aptitude and technological savvy to leverage tools
Critical tools available to all (Alteryx, Tableau)
125 Alteryx users enabled in the last 6 months
Remote processing available for larger data sets
Seek out opportunities to test new methods
Invest in learning opportunities
Involve clients in the experimentation
Formalize our lessons learned into products
Look for opportunities to apply products at other clients

The Lay of the Land

Predictive analytics is GREAT!...
but predictive analytics is a scary thought for a lot of managers
Lots of math
Potentially a lot of expense
Can you believe the numbers that come out of the fancy models?
How do you even get started?

Two Approaches to Getting Started

Hire an outside firm
No fixed costs
Take advantage of the outside firms expertise
Do-it-yourself
Lower variable costs
Greater opportunities to learn and understand the capabilities and limitations of
predictive analytics
Allows for a closer connection and integration with existing business processes
Many organizations conduct a mixture of in-house and outsourced predictive analytics
projects

Four Steps to In-House Predictive Analytics Success

Start small and take a learning by doing approach
Develop an initial list of possible predictive analytics projects that address frequent and
important business decision in your organization
Select projects from the initial list that make use of well-known metrics for predicting
outcomes
Compare the results of a new predictive analytics-based business process to the
incumbent process used to make a decision

Four Steps to In-House Predictive Analytics Success

Start small and take a learning by doing approach
Develop an initial list of possible predictive analytics projects that address frequent and
important business decision in your organization
Select projects from the initial list that make use of well-known metrics for predicting
outcomes
Compare the results of a new predictive analytics-based business process to the
incumbent process used to make a decision

The Virtues of Starting Small

An initial low financial commitment with respect to both software and personnel
You likely already have Alteryx licenses if you are in this room
The organization is able to develop internal expertise in predictive analytics that it can
leverage in the future
The organization develops a better understanding of what is and is not possible with
predictive analytics
It provides the ability to assess the possible benefits from using predictive analytics to
drive business processes, but in a limited way that limits the downside risk
Several successful small projects builds managerial confidence in the approach,
enhancing organizational buy-in

What do you Need to Start Small?

One or two current staff members with the willingness to take on a new challenge, have
a basic set of computer skills, and are given some time to experiment with the methods
Appropriate software
You likely already have Alteryx licenses if you are in this room
Our Predictive Analytics Essentials online course can provide a jump-start
The analysis tool pack in Excel has been used by a number of organizations to get
started
What about advanced statistical and data mining training?
It helps, but an understanding of the business and the willingness to learn matters
more
Asking the right question is a lot more valuable than using the best analysis method

Develop a List of Business Questions PA can Inform

A useful way to start is with your organizations key performance indicators (KPIs) and
then determine how predictive analytics can help address the business decision that
underlie the KPIs
OK lets use an example to make this concrete
Congratulations you are now the General Manager of a major league baseball team
In this job, what are your KPIs?
What decisions can you make in order to deliver on those KPIs?
What information can we use to inform these decisions?

Use Well-Known Metrics to Select Projects

In many cases there are (fairly) well-known metrics that can be taken advantage of to
select projects from the list of potential projects
Relying on others past experience in selecting predictor variables can really shorten
the time it takes to develop a useful predictive analytics model
Recency, frequency, and monetary value (or RFM) is a well known example from direct
marketing that works well for cross selling applications to existing customers
Web searches to find relevant articles, blog posts, slide decks, and other resources can
really help
Should web searches fail, thinking through the information that is available at the time a
decision is made (as opposed to what is available with 20-20 hindsight) is a useful thought
experiment that can be used to develop possible metrics

Back to Baseball
We know that scoring runs is a critical element in winning baseball games, and we know
we can draft or acquire players based on statistics (metrics) that, as a team, will lead to
scored runs. What are the available statistics?
Common baseball batting statistics on individual players available on a historical basis:
Hits
Walks (Base on balls and hit by a pitch)
Strikeouts
Singles
Doubles
Triples
Homeruns
RBIs
Batting average / plate appearances (at bats)

Back to Baseball
The statistics are for individuals, but what happens when we combine them into a team?
What is better, having a set of players in the batting order that can only get to first
base on a walk or hitting a single, but do so at every at bat, or a set of players in the
batting order that only hit home runs, but have a one in three chance of doing so at
each at bat?
We just saw the common statistics, are there metrics we can construct from them that
can be more informative?
The wisdom of Bill James and SABRmetrics: On Base Percentage + Slugging Percentage
The question: Can we use this information as the basis for drafting or acquiring players?

Compare Models to Traditional Business Processes

Testing and experimentation are an essential part of the use of predictive analytics tools
The goal of these tests is to objectively compare the performance of the predictive
analytics-based business process to the traditional business process
Why?
Many managers dont trust models, but they are very comfortable with comparing one
group with another to see if there is a noticeable difference between the two of them
Favorable results in these tests increase managers trust in predictive analytics
How?
A/B testing: Explicitly creating treatment (those who are addressed using a predictive
analytics-based process) and control (those who are addressed using traditional
business processes) groups and then compare results
Retrospective testing: Use two time periods and compare differences in outcomes
based on traditional business processes and those predicted as best by a model

Thank You!