But First, Something Fun

1. 2. 3. 4.

Pull Out Your Phone Open Your Texting App Prepare to Send a Text to 22333 Here are the Possible Text Responses
223-33

Big Data - Big Expectations

Todays Agenda

Big Reality:
Real Solutions

Big Data Framework

Real Customer Examples Why How What

What Makes it Big Data?

SOCIAL

BLOG

SMART METER

101100101001 001001101010 101011100101 010100100101

VOLUME

VELOCITY

VARIETY

VALUE/ VERACITY

Data that cannot be turned into business value fast enough

Where Does Big Data Come From?

Process Mediated Data

Structured: aka "Process-mediated": Examples: ERP, CRM , POS Characteristics: transactional, referential, relational, Traditionally IT managed
Semi-Structured: aka "Machine-generated Examples: XML, JSON, Network Logs, Sensor Data Characteristics: Well suited to computer processing but massive in Volume and accumulation, often too large for EDW Unstructured: aka "Human-sourced": Examples: CDR, Doctor's Notes, Social Media, Audio, Video, comment fields Characteristics: subjective record of personal experiences, structuring required to realize value

Machine Generated Data

Human Sourced Data

This is your data. This is placeholder text for whatever best represents the structured and unstructured data you want to query. tis, justo e pellentesque metus, et sollicitudin diam lectus eu sapien. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculu s Do 40% more of this rhon mus cus aliquam, massa mauris por ta tortor, vel porta odio ligula a elit. Sed laoreet, lectus ac dapibus placerat, nulla turpis lacin ia lectus, eget hendrerit n ibh mauris vel ligula. Sed a nisl dolor, a tincidunt leo. ab

Where is the Value?

Gain Market Share (Social Scrapes) Identify Cost Savings (Streaming Fraud Detection, Doctors Notes) Customer Retention (Call Detail Records) Real Time Business Automation (High Freq. Trading, Smart Utility Distribution) Business Process Re-invention (Personalized Risk vs Statistical Averages) Modernization and Competitive Technological Advantage

How Does it Get to the User?

Process Mediated Data

(Traditional) EDW In-Memory

Data Integration Human Sourced Data
This is your data. This is placeholder text for whatever best represents the structured and unstructured data you want to query. tis, justo e pellentesque metus, et sollicitudin diam lectus eu sapien. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculu s Do 40% more of this rhon mus cus aliquam, massa mauris por ta tortor, vel porta odio ligula a elit. Sed laoreet, lectus ac dapibus placerat, nulla turpis lacin ia lectus, eget hendrerit n ibh mauris vel ligula. Sed a nisl dolor, a tincidunt leo. ab

Users

Data Virtualization

BI / Analytics

NoSQL Analytics Mart

Machine Generated Data

Hadoop

Disparate, non-performing

Integrated, performance-optimized

Where Do We Put It?

Human Information

Hadoop

NoSQL

(Traditional) EDW

Analytics Mart

In-Memory

Java /Open Source

SQL, Windows, Linux

Vendor Specific

A Few Customer Examples

Volume - Financial Analysis

1T Rows

10

Variety & Veracity

Maintenance
Maximo SAPPM Oracle Passport DataStream

Executive Management / Plant Management ISA-95 Asset Hierarchical Data Model

Manual Input
DB Excel

Reliability
Meridium Capstone DB / Excel

Capital Projects Outputs:

Hundreds Standard KPIs Reporting with Drill Down Root Cause Analysis Role Based Dashboards Management of Change Multiple Site Role up Cross Functional Collaboration Platform
Primavera MS Project Sharepoint

Operation IP21 OSI Pi s

Supply Chain
Aspen EDO Scheduling

Honeywell Yokogawa Siemens

Engineering
Intergraph AspenTech AutoCAD Bentley Aveva

Finance
SAP JDE Oracle

Health Safety & Environment

SAP ESS

Variety & Veracity

Using ALL the Data Volume & Variety eBay

I was thinking if we created a new search engine with added functionality to help me find that obscure R2-D2 action figure faster and easier?

Bob, what if we could answer these questions What did someone buy?, What did they bid on it? Its also Where were they at the time? Its also Who influenced them within their social circle? All that data is amassed. cool idea? We can build a Big Datas infrastructure challenge first test run of a Hadoop cluster consisted of 400 nodes and Two 24-petabyte clusters this will allow us to bring the processing to where the data sits, removing the need for time-consuming data transfers Larger Index than Voyager More descriptions, history & metadata in indexes 100 engineers, all new codebase, 18 months

Variety Financial Structured and Unstructured Data

Select Customers with < 150K in Assets pull demographics

Reference check 21 image DB Clickstream data from banking web site Meaning based positive comments

From a database get me all matches from the CRM and Call Detail Records that match the query

From unstructured sources get me all matches for calls, chat, email that were positive for the structured results

Pull < total 30% of net worth from Check 21 Image database
Customers who conducted net worth report from our banking web site

Structured Data
Columnar

10,015,664,356,165 rows (10 trillion) 22B-43B rows loaded daily 20 node x86 cluster 1.794 petabytes raw data 7:1 compression

Unstructured Data

Velocity - Smart Meters

Meter Data Management (MDM) System that 1) Customer segmentation 2) Anomaly Detection 3) Data volume

Variety - Public Health

Can we find any relationship between FDA recall actions & conversations within the social media universe?

How to avoid unintended economic consequences of recall actions?

In 2009
Peanut butter recall cost producers $1,000,000,000
48 million people get sick 128,000 are hospitalized and 3000 die each year from foodborne diseases
What where people talking about two weeks before the recall? How did they feel? Where were they? What were they eating?

16

How Do I Start?
Data Analytics Gap Analysis

Proof of Value

Vendor Agnostic Assessment of Current State and Go-Forward Strategy Benchmark your Data Management Infrastructure against industry

Business Value Assessment

Technical Assess. of targeted products

System Design and Architecture POC Installation/configuration Integration Testing Performance Benchmarking System Tuning/Optimization Performance Evaluation

Identify Critical Business and Technology Gaps

Matrix/SWOT of product capabilities and environmental applicability Strategic and Technical Execution and Deployment Planning

ROI Assessment
17

Thank You

18