You are on page 1of 73

Lecture:

Trends and Concepts in the


Software Industry
Introduction
Dr.-Ing. Jrgen Mller

Course Statistics

W
ee
k
W "1"
ee
k
W "2"
ee
k
W "3"
ee
k
W "4"
ee
k
W "5"
ee
k"6
Ex "
am
"

h#p://www.openHPI.de
13,126 par8cipants, 4.161 ac8ve
contributors in discussions
4000"
100"
6 course modules for 6 weeks
80"
3000"
60"
35 individual module items (videos /
2000"
40"
slides / reading material)
1000"
20"
300 exam / assignment / self-test
0"
0"
ques8ons
309 slides explicitly created for this course
215 pages book reading material
Number"of"Par8cipants"in"Assignments"&"Exam"
8h cut video material (12h raw material) Average"Results"in"%"
Teaching Team: 4 colleagues

Goals
Deep technical understanding of a column-oriented dic8onary-
encoded in-memory database and its applica8on in enterprise
compu8ng

The future of enterprise compu8ng


Founda8ons of database storage techniques
In-memory database operators
Advanced database storage techniques
Founda8ons for a new enterprise applica8on development era

December 2006: The Basic Idea

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Chapter 1:
The Future of
Enterprise Computing

New Requirements for


Enterprise Computing

Dr.-Ing. Jrgen Mller

Sensors
Tracing Pharmaceu(cal Packages in Europe
15 billion packages / 34 billion read events per year
Distributed repositories for storing read events
References to read events are stored in central discovery service

Monitoring F1 car Performance


Between 300 and 600 sensors per car
Mul8ple events per second per sensor
Tracking every Grand Prix lap or test run

Combination of Structured
and Unstructured Data
Airplane Maintenance at Boeing
Maintenance workers at Boeing write reports aYer repairs
Reports and men8oned part numbers get indexed
Analy8cs reveal which parts in other planes may be defec8ve

Medical Diagnosis at Charit


Doctors write medical reports aYer every diagnosis
Diagnosis and men8oned symptoms get indexed
Comparison with similar cases for op8mal treatment

Mobile
Mobile inverts tradi(onal corporate structures

Enables customer-facing
Consumers / Customers
personnel
Sales, Service

Response 8mes < 1 second

Opera8ons
Controlling

Example: Dunning

Consolida8on
Strategy
CEO

10

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Enterprise Application
Characteristics

Dr.-Ing. Jrgen Mller

Challenge:
Diverse Applications
Transac(onal
Data Entry

Real-(me Analy(cs,
Structured Data


Sources: Machines, Transac8onal
Apps, User Interac8on, etc.

Sources: Repor8ng, Classical


Analy8cs, planning, simula8on
CPUs
(mul*-Core + Cache)
Main Memory

Event Processing,
Stream Data

Data
Management


Sources: machines, sensors,
high volume systems

Text Analy(cs,
Unstructured Data


Sources: web, social, logs,
support systems, etc.

13

OLTP vs. OLAP


Online Transac8on
Processing

Modern enterprise resource planning (ERP) systems are


challenged by mixed workloads, including OLAP-style
queries. For example:

14

Online Analy8cal
Processing

OLTP-style: create sales order, invoice, accoun8ng documents,


display customer master data or sales order
OLAP-style: dunning, available-to-promise, cross selling,
opera8onal repor8ng (list open sales orders)

But: Todays data management systems are op8mized


either for daily transac(onal or analy(cal workloads
storing their data along rows or columns

Drawbacks of
the Separation

OLAP system does not have the latest data

OLAP system does only have predened subset of the data

Cost-intensive ETL process has to synch both systems

There is a lot of redundancy

Dierent data schemas introduce complexity for


applica8ons combining sources

15

OLTP Access Pattern


Myth

16

Workload analysis of enterprise systems shows:


OLTP and OLAP workloads are not that dierent

Vision
Combine OLTP and OLAP data
using modern hardware and database systems
to create a single source of truth,
enable real-(me analy(cs and
simplify applica8ons and database structures.


Addi8onally,
extrac8on, transforma8on, and loading (ETL) processes

pre-computed aggregates and materialized views become


obsolete.

17

Enterprise Data
Characteristics

Many columns are not used even once

Many columns have a low cardinality of values

NULL values/default values are dominant

Sparse distribu8on facilitates high compression

Standard enterprise soYware data is sparse and wide


18

Low Cardinality of Values


Within Many Columns
Results from analyzing nancials

Dis8nct values in accoun8ng document headers


(99 a#ributes)

CPG

Banking

Logis8cs

Discrete
manufacturing

High tech

19

Many Columns are not


Used Even Once
55% unused columns per company in average
40% unused columns across all companies

80%

% of Columns

60%

Inventory Management
Financial Accounting

78 %

70%
64 %

50%
40%
30%
20%

24 %

10%
0%
20

13 %
9%
33 - 1023
1024 - 100000000
Number of Distinct Values
12 %

1 - 32

Wide Tables
Analysis of width of 144 most used* tables
30

# Tables

25
20
15
10
0

1-9
10-19
20-29
30-39
40-49
50-59
60-69
70-79
82
99
110-119
120-129
138
140-149
156
180-189
200-209
230
312
399

# Columns

21

* Largest in terms of cardinality

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Changes in Hardware

Dr.-Ing. Jrgen Mller

Advances in Hardware
A

Mul8-Core Architecture
8 x (8-16) core CPU per blade

64 bit address space 4TB


in current server boards

Parallel scaling across blades

One blade $50.000 =


1 Enterprise Class Server

25GB/s data throughput,


CPU - DRAM

Cost-performance ra8o
rapidly declining

Copy 50GB data via Inniband 10s


24

Memory Hierarchy
CPU
Registers

Lower Price /
Higher Latency

Higher
Performance

CPU
Caches

Main Memory

Flash
Hard Disk
25

Latency Numbers
L1 cache reference (cached data word)
Branch mispredict

5ns

L2 cache reference

7ns

Mutex lock/unlock

25ns

Main memory reference

100ns

0.1s

20,000ns

20s

SSD random read

150,000ns

150s

Read 1 MB sequen8ally from memory

250,000ns

250s

10,000,000ns

10ms

150,000,000ns

150ms

Send 2K bytes over 1 Gb/s network

Disk seek
Send packet CA to Netherlands to CA
26

0.5ns

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

A Blueprint of SanssouciDB

Dr.-Ing. Jrgen Mller

SanssouciDB: An In-Memory
Database for Enterprise Applications

Main memory access is


the new bo\leneck

Cache-conscious algorithms/
data structures are crucial
(locality is king)

TA Manager

Time
travel

Combined
Column

Column

Differential
Store

Main Store

Data
aging

Distribution Layer
Layer
Distribution
at Blade
Serveri i
at

Main Memory
at Blade
Serveri i

Active Data

Column

S8ll: logging to disk/


recovery from disk

Metadata

Merge

Main Memory is the primary


persistence

Query Execution

Combined
Column

Data resides permanently in


main memory

Column

Interface Services and Session Management

Column

In-Memory Database (IMDB)

Indexes
Inverted

Object
Data Guide

Logging

Recovery

Log
Passive Data (History)

Non-Volatile
Memory

Snapshots

Chapter 2:
Foundations of
Database Storage Techniques

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Dictionary Encoding

Dr.-Ing. Jrgen Mller

Motivation

Main memory access is the new bo#leneck


Compression reduces number of I/O opera8ons to
main memory
Opera8on directly on compressed data
Osenng with bit-encoded xed-length data types
Based on limited value domain

33

Dictionary Encoding
Example

8 billion humans

A#ributes:
rst name
last name
gender
country
city
birthday
200 byte per tuple

34

Each a#ribute is dic8onary encoded

Sample Data
Table: world_popula8on
recID

fname

lname

gender

city

country

birthday

39

John

Smith

Chicago

USA

12.03.1964

40

Mary

Brown

London

UK

12.05.1964

41

Jane

Doe

Palo Alto

USA

23.04.1976

42

John

Doe

Palo Alto

USA

17.06.1952

43

Peter

Schmidt

Potsdam

GER

11.11.1975

35

Dictionary Encoding
a Column

36

A column is split into a dic8onary and an a#ribute vector


Dic8onary stores all dis8nct values with implicit valueID
A#ribute vector stores valueIDs for all entries in the column
Posi8on is stored implicitly
Enables osenng with bit-encoded xed-length data types
Dic(onary for fname

A\ribute Vector for fname

recID

fname

valueID

Value

posi(on

39

John

40

Mary

23

John

39

23

41

Jane

24

Mary

40

24

42

John

25

Jane

41

25

43

Peter

26

Peter

42

23

43

26

valueID

Querying Data using


Dictionaries
Search for A#ribute Value
(i.e. retrieve all persons with fname Mary)

1. Search valueIDs for requested value (Mary)
2. Scan A#ribute Vector for valueID (24)
3. Replace valueIDs in result with corresponding
dic8onary value

37

Sorted Dictionary

Dic8onary entries are sorted either by their numeric value or


lexicographically
Dic8onary lookup complexity: O(log(n)) instead of O(n)

Dic8onary entries can be compressed to reduce the amount


of required storage

Selec8on criteria with ranges are less expensive

38

Compression Rate

Depends on cardinality / entropy


Cardinality

Table cardinality: number of tuples in a rela8on


Column cardinality: number of dis8nct values in a column

Entropy

measure for informa8on density


Entropy = column cardinality / table cardinality

39

Data Size Examples


Column

Cardi-
nality

Bits
Item Size Plain Size Size with Dic(onary Compression
Needed
(Dic(onary + Column)
Factor

First
names

5 millions

23 bit

50 Byte

373GB

238.4MB + 21.4GB

17

Last
names

8 millions

23 bit

50 Byte

373GB

381.5MB + 21.4GB

17

1 bit

1 Byte

7GB

2.0b + 953.7MB

1 million

20 bit

50 Byte

373GB

47.7MB + 18.6GB

20

Country

200

8 bit

47 Byte

350GB

9.2KB + 7.5GB

47

Birthday

40000

16 bit

2 Byte

15GB

78.1KB + 14.9GB

200 Byte

1.6TB

92GB

17

Gender
City

Totals
40

Data Layout in Main Memory

Dr.-Ing. Jrgen Mller

Basics
...
0x0

0xFFFF FFFF FFFF FFFF


FFFF FFFF FFFF FFFF
FFFF FFFF FFFF FFFF

The memory layout is only linear, every higher-dimensional


access is mapped to this linear band.

42

Row Data Layout

Data is stored tuple-wise


Leverage co-loca8on of a#ributes for a single tuple
Low cost for reconstruc8on, but higher cost for sequen8al
scan of a single a#ribute
Column Operation
A B C A B C A B C A B C
Row

Row

Row

Row Operation
A B C A B C A B C A B C
43

Row

Row

Row

Row

Columnar Data Layout

Data is stored a#ribute-wise


Leverage sequen8al scan-speed in main memory
Tuple reconstruc8on is expensive
Column Operation
A A A A B B B B C C C C
Column

Column

Column

Row Operation
A A A A B B B B C C C C
44

Column

Column

Column

Chapter 3:
In-Memory Database Operators

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Insert

Dr.-Ing. Jrgen Mller

Example
Example Table: world_popula8on
recID

fname

lname

gen country
der

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg 04-04-1968

Anton

Meyer

AUT

Innsbruck 10-20-1992

Sophie

Schulze

GER

Potsdam 09-03-1977

...

...

...

...

...

...

INSERT INTO world_popula8on


VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
48

...

INSERT Without
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

...

...

...

...

...

...

...

Albrecht

1 1

Berg

2 3

Meyer

3 2

Schulze

4 3

A#ribute Vector (AV)


Dic8onary (D)

49

INSERT Without
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

Albrecht

1 1

Berg

2 3

Meyer

3 2

Schulze

4 3

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

...

...

...

...

...

...

...

A#ribute Vector (AV)

1.

50

Look-up on dic8onary entry found

Dic8onary (D)

INSERT Without
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

4 3

Sophie

Schulze

GER

Potsdam

09-03-1977

5 3

...

...

...

...

0 0

Albrecht

1 1

Berg

2 3

Meyer

3 2

Schulze

...

1.
2.

Schulze
...

...

A#ribute Vector (AV)

Look-up on dic8onary entry found


Append valueID to a#ribute vector

Dic8onary (D)

51

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

Berlin

1 0

Hamburg

2 1

Innsbruck

3 2

Potsdam

4 3

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

...

...

...

...

5
...

Schulze
...

...

A#ribute Vector (AV)


Dic8onary (D)

52

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

...

...

...

...

Berlin

1 0

Hamburg

2 1

Innsbruck

3 2

Potsdam

4 3

5
...

1.

Schulze
...

...

A#ribute Vector (AV)

Look-up on dic8onary no entry found

Dic8onary (D)

53

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

Berlin

1 0

Hamburg

2 1

Innsbruck

3 2

Potsdam

4 3

Rostock

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

...

...

...

...

5
...

1.
2.

54

Schulze
...

...

Look-up on dic8onary no entry found


Append new value to dic8onary (no re-sor8ng necessary)

A#ribute Vector (AV)


Dic8onary (D)

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 0

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

Berlin

1 0

Hamburg

2 1

Innsbruck

3 2

Potsdam

4 3

Rostock

5 4

5
...

1.
2.
3.

Schulze
...

...

Rostock
...

...

...

...

A#ribute Vector (AV)

Look-up on dic8onary no entry found


Append new value to dic8onary (no re-sor8ng necessary)
Append valueID to a#ribute vector

Dic8onary (D)

55

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 2

Anton

1 3

Hanna

2 1

Mar8n

3 0

Michael

4 4

Sophie

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

5
...

Schulze
...

...

Rostock
...

...

...

...

A#ribute Vector (AV)


Dic8onary (D)

56

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 2

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

Anton

1 3

Hanna

2 1

Mar8n

3 0

Michael

4 4

Sophie

5
...

1.

Schulze
...

...

Rostock
...

...

...

...

A#ribute Vector (AV)

Look-up on dic8onary no entry found

Dic8onary (D)

57

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Sophie

Schulze

GER

Potsdam

09-03-1977

Karen

0 2

Anton

1 3

Hanna

2 1

Mar8n

3 0

Michael

4 4

4
5

...

1.
2.

58

Schulze
...

Look-up on dic8onary no entry found


Append new value to dic8onary

...

Rostock
...

...

...

...

A#ribute Vector (AV)


Dic8onary (D)

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D (old)

AV
0 2

Anton

D (new)
0

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Sophie

Schulze

GER

Potsdam

09-03-1977

Anton

1 3

Hanna

Hanna

2 1

Mar8n

Karen

3 0

Michael

Mar8n

4 4

Sophie

Michael

Karen

Sophie

5
...

1.
2.
3.

Schulze
...

...

Rostock
...

...

...

...

A#ribute Vector (AV)

Look-up on dic8onary no entry found


Append new value to dic8onary
Sort dic8onary

Dic8onary (D)

59

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
AV (old)

AV (new)

D (new)
fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Michael

Sophie

Schulze

GER

Potsdam

09-03-1977

Sophie

0 2

0 3

Anton

1 3

1 4

Hanna

2 1

2 1

Karen

3 0

3 0

Mar8n

4 4

4 5

4
5

...

1.
2.
3.
4.

60

Schulze
...

Look-up on dic8onary no entry found


Append new value to dic8onary
Sort dic8onary
Change valueIDs in a#ribute vector

...

Rostock
...

...

...

...

A#ribute Vector (AV)


Dic8onary (D)

INSERT With
New Dictionary Entry
INSERT INTO world_popula8on VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
D

AV
0 3

1.
2.
3.
4.
5.

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

GER

Potsdam

09-03-1977

Anton

1 4

Hanna

2 1

Karen

3 0

Mar8n

4 5

Michael

Sophie

Schulze

5 2

Sophie

Karen

Schulze

...

...

...

Rostock
...

...

...

...

A#ribute Vector (AV)

Look-up on dic8onary no entry found


Append new value to dic8onary
Sort dic8onary
Change valueIDs in a#ribute vector
Append new valueID to a#ribute vector

Dic8onary (D)

61

Result
Example Table: world_popula8on
recID

fname

lname

gender country

city

birthday

Mar8n Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg 04-04-1968

Anton

Meyer

AUT

Innsbruck 10-20-1992

Ulrike

Schulze

GER

Potsdam 09-03-1977

Karen

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

INSERT INTO world_popula8on


VALUES (Karen, Schulze, f, GER, Rostock, 06-20-2012)
62

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Scan Performance

Dr.-Ing. Jrgen Mller

Scan Performance (1)


8 billion humans
A#ributes:
rst name
last name
gender
country
city
birthday
200 byte

Ques8on: How many women, how many men?


Assumed scan speed: 2MB/ms/core

65

Scan Performance (2)


Row Store Layout
Table:
orld_popula(on
Table:whumans
Last Name
Country
First Name
Gender
Row
1
Row
2
Row
3

...

Row
9
8 x 10

66

Birthday
City

Scan Performance (3)


Row Store Layout
Table:
orld_popula(on
Table:whumans
Last Name
Country
First Name
Gender
Row
1

Birthday
City

Row
2

Table size:
8 billion tuples
200 bytes per
tuple 1.6 TB

Row
3

...

Row
9
8 x 10

67

Scan Performance (4)


Row Store Full Table Scan
Table:
orld_popula(on
Table:whumans
Last Name
Country
First Name
Gender
Row
1

Table size:
8 billion tuples
200 bytes per
tuple 1.6 TB

Scan through
all rows with
2MB/ms/core
800 s
with 1 core

Birthday
City

Row
2
Row
3

...

Row
9
8 x 10

68

Data loaded
and used

Data loaded
but not used

Scan Performance (5)


Row Store Stride Access Gender
Table:
orld_popula(on
Table:whumans
Last Name
Country
First Name
Gender
Row
1

8 billion cache
accesses 64 byte
512 GB

Read with
2MB/ms/core
256 s
with 1 core

Birthday
City

Row
2
Row
3

...

Row
9
8 x 10

69

Data
not loaded

Data loaded
and used

Data loaded
but not used

Scan Performance (6)


Column Store Layout
Table:
orld_popula(on
Table:whumans
Last Name
First Name
Gender

70

Country

A#ribute vector
size 91 GB

Dic8onary size
700 MB

Table size 92 GB

Compression
factor 17

Birthday
City

Scan Performance (7)


Column Store Full Column Scan Gender
Table:
orld_popula(on
Table:whumans
Last Name
First Name
Gender

71

Data
not loaded

Country

Size of a#ribute
vector Gender:
8 billion tuples
1 bit per tuple
1 GB

Scan through
column with
2MB/ms/core
0.5 s with 1 core

Birthday
City

Data loaded
and used

Data loaded
but not used

Scan Performance (8)


Column Store Full Column Scan Birthday
Table:
orld_popula(on
Table:whumans
Last Name
First Name
Gender

72

Data
not loaded

Country

Size of a#ribute
vector Birthday:
8 billion tuples
2 byte per tuple
16 GB

Scan through
column with
2MB/ms/core
8 s with 1 core

Birthday
City

Data loaded
and used

Data loaded
but not used

Scan Performance
Summary



How many women, how many men?


Row Store

Full table scan


Stride access

Column Store

Time in seconds

800

256

0.5

Factors compared
to best in class

1,600

512

73

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Tuple Reconstruction

Dr.-Ing. Jrgen Mller

Tuple Reconstruction (1)


Accessing a record in a row store
Table: world_popula8on

76

All a#ributes
are stored
consecu8vely

200 byte 4 cache


accesses 64 byte
256 byte

Read with
2MB/ms/core
0.128 s
with 1 core

Tuple Reconstruction (2)


Virtual record IDs
Table: world_popula8on

All a#ributes are


stored in separate
columns

Implicit record IDs


are used to
reconstruct rows

77

Tuple Reconstruction (3)


Virtual record IDs
Table: world_popula8on

78

1 cache access for


each a#ribute

6 cache accesses
64 byte
384 byte

Read with
2MB/ms/core
0.192 s
with 1 core

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Select

Dr.-Ing. Jrgen Mller

SELECT Example
SELECT fname, lname FROM world_population
WHERE country=Italy and gender=m

id

fname

country

gender

2394

Gianluigi Buon

lname

Italy

3010

Lena

Gercke

Germany

3040

Mario

Balotelli

Italy

3949

Manuel

Neuer

Germany

4902

Lukas

Podolski

Germany

20102

Klaas-Jan Huntelaar

Netherlands

81

Query Plan
country = "Italy"

gender = "m"

position
list

position
list

positional
AND

82

fname,
lname

Predicates are evaluated and generate posi8on lists


Intermediate posi8on lists are logically combined
Final posi8on list is used for materializa8on

Query Execution
ValueID

Dic(onary
for country

id

fname

lname

country

gender

2394

Gianluigi

Buon

3010

Lena

Gercke

Algeria

France

3040

Mario

Balotelli

Germany

3949

Manuel

Neuer

Italy

4902

Lukas

Podolski

20102

Klaas-Jan Huntelaar

Netherlands

country = 3 (Italy)

ValueI
D

Dic(onary
for gender

gender = 1 (m)
Posi(on
0
2

Posi(on

AND

Posi(on
0
2
83

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Chapter 4:
Advanced Database
Storage Techniques

Differential Buffer

Dr.-Ing. Jrgen Mller

Motivation

Inser8ng new tuples directly into a compressed structure can be


expensive

New values may require a reorganiza8on of the dic8onary


Number of bits required to encode all dic8onary values may change,
a#ribute vector has to be reorganized

Dele8on of tuples is expensive

All a#ribute vectors have to be reorganized, valueIDs of following tuples


have to be adjusted

87

Differential Buffer

Read

Write

world_population

Table

A#ribute Vector

4
5

0
1
2
3

Anton

Hanna

Michael

Sophie
Differential

1
3
2

8 Billion entries
88

fname

Dic8onary

Main Store

A#ribute Vector

Main
0
Store
1
Table
2
3

Read Operations

fname

Data Modifying
Operations

New values are wri#en to a dedicated dieren8al buer (delta)


Cache Sensi8ve B+ tree (CSB+) used for faster search on delta

(compressed)

Dic8onary

0
1
1

Angela

Klaus

Andre

Buffer
up to 50,000
entries

Dieren8al Buer

CSB+

Differential Buffer

Inserts of new values are faster, because dic8onary and


a#ribute vector does not need to be resorted
Range select on dieren8al buer expensive, based on
unsorted dic8onary
Dieren8al Buer requires more memory:

no a#ribute vector compression


addi8onal CSB+ tree for dic8onary

89

Tuple Lifetime
Michael moves from Berlin to Potsdam
Table: world_popula8on
recId

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Ulrike

Schulze

GER

Potsdam

09-03-1977

Sophie

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

8 109

Zacharias

Perdopolus

GRE

Athen

03-12-1979

Main Store

Dieren8al Buer
UPDATE world_popula*on
SET
city=Potsdam
WHERE fname= Michael AND lname=Berg
90

Tuple Lifetime
Michael moves from Berlin to Potsdam
Table: world_popula8on
recId

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Ulrike

Schulze

GER

Potsdam

09-03-1977

Sophie

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

8 109

Zacharias

Perdopolus

GRE

Athen

03-12-1979

Main Store

Dieren8al Buer
UPDATE world_popula*on
SET
city=Potsdam
WHERE fname= Michael AND lname=Berg
91

Tuple Lifetime
Michael moves from Berlin to Potsdam
Table: world_popula8on
recId

fname

lname

gender

country

city

birthday

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Ulrike

Schulze

GER

Potsdam

09-03-1977

Sophie

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

8 109

Zacharias

Perdopolus

GRE

Athen

03-12-1979

Michael

Berg

GER

Potsdam

03-05-1970

UPDATE world_popula*on
SET
city=Potsdam
WHERE fname= Michael AND lname=Berg
92

Main Store

Dieren8al Buer

Tuple Lifetime

Problem: Tuples are now available in Main Store and Dieren8al


Buer
Tuples of a table are marked by a validity vector to reduce the
required amount of reorganiza8on steps

Like an a#ribute vector for validity

Invalidated tuples stay in the database table, un8l the next


reorganiza8on takes place
Search results are reduced using the validity vector
1 bit required per database tuple

93

Tuple Lifetime
Michael moves from Berlin to Potsdam
Table: world_popula8on
recId

fname

lname

gender

country

city

birthday

valid

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Ulrike

Schulze

GER

Potsdam

09-03-1977

Sophie

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

8 109

Zacharias

Perdopolus

GRE

Athen

03-12-1979

Michael

Berg

GER

Potsdam

03-05-1970

UPDATE world_popula*on
SET
city=Potsdam
WHERE fname= Michael AND lname=Berg
94

Main Store

Dieren8al
Buer

Tuple Lifetime
Michael moves from Berlin to Potsdam
Table: world_popula8on
recId

fname

lname

gender

country

city

birthday

valid

Mar8n

Albrecht

GER

Berlin

08-05-1955

Michael

Berg

GER

Berlin

03-05-1970

Hanna

Schulze

GER

Hamburg

04-04-1968

Anton

Meyer

AUT

Innsbruck

10-20-1992

Ulrike

Schulze

GER

Potsdam

09-03-1977

Sophie

Schulze

GER

Rostock

06-20-2012

...

...

...

...

...

...

...

8 109

Zacharias

Perdopolus

GRE

Athen

03-12-1979

Michael

Berg

GER

Potsdam

03-05-1970

UPDATE world_popula*on
SET
city=Potsdam
WHERE fname= Michael AND lname=Berg

Main Store

Dieren8al
Buer

95

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Merge

Dr.-Ing. Jrgen Mller

Handling Write
Operations

98

All Write opera8ons (INSERT, UPDATE) are stored within a


dieren8al buer (delta) rst
Read-opera8ons on dieren8al buer are more expensive than
on main store
Outdated tuples values in main store are invalidated
To guarantee op8mal performance the dieren8al buer needs
to be merged periodically with the main store

Merge Process
Overview

The merge process is working table wise

Consists of:
Prepare merge
A#ribute merge
Commit merge

1.
2.
3.

Is triggered by:

Amount of tuples in buer


Cost model to
Schedule
Take query cost into account

99

Online Merge
Working on data copies allows asynchronous merge
Very limited interrup8on due to short lock
At least twice the memory needed!

Before

Table
Main
Store

Merge Operation

Main
Store

Read Operations
100

Table
Differential
Buffer

Main
Store
(new)

Read Operations

Prepare

Data
Modifying
Operations

Data
Modifying
Operations

Table
Differential
Buffer

After

During the Merge Process

Data
Modifying
Operations

Attribute Merge

Differential
Buffer
(new)

Main
Store
(new)

Differential
Buffer
(new)

Read Operations

Commit

Attribute Merge
Step 1:
Extract delta dic8onary from delta column
Merge main and delta dic8onary, op8onally remove unused
values
Create mapping if valueID changed

Step 2:
Create new main column (if needed)
Main store column a#ribute vector needs to be rewri#en if

Dic8onary increases over #bit value range

Sort order changes

101

Example
Inputs Step 1

Outputs Step 1

Inputs Step 2

Output Step 2

AVm

Dm

Dc

AUX m

AVm

AVc

valueID

Dictionary Entries

Dictionary Entries

new valueID

valueID

valueID

anna

charlie

daniel

frank

hans

isabell

bernd

bernd

hans

frank

102

B1

D1

anna

bernd

charlie

daniel

frank

5
6

0
1
2
3
4
5

hans

isabell

10

11

AUX d

Delta Position
CSB+ Leafs
Lists
bernd

0
1

frank
hans

3
2

0
2
3
4
5
6

new valueID

old valueID

AVd

uncompressed

M1

old valueID

new valueID

old valueID

0
1
2

1
4
5

UM1

AVds

valueID

10

11

UM2

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Logging and Recovery

Dr.-Ing. Jrgen Mller

What is Logging and Recovery?

In-memory databases need to provide durability guarantees


to be used in produc8ve enterprise applica8ons
Poten8al strategies

Logging: wri8ng log informa8on to backup storage (covered here)


Server replica(on: run transac8ons simultaneously in primary and
replicated database

Recovery is the process of bringing a system back to a


produc8ve mode aYer a system failure
Desirable characteris8cs of any recovery mechanism:

Minimal impact on running system


Fast recovery without any data loss

105

Logging Infrastructure for an


In-Memory Database

106

Snapshots of primary
data stores are wri#en
periodically (e.g. during
merge process)
Log les for updates
are wri#en between
snapshots to recover
the database
Dic8onary-compressed
column logging allows
parallel log le wri8ng
and recovery

Snapshots

A snapshot writes the primary data of the in-memory


database to persistent memory
Purpose of snapshots:

Speed-up of recovery process, as only the log les aYer the


snapshot need to be replayed
Crea8on of a system copy

A snapshot is ideally wri#en aYer each merge process

Snapshot contains new read-op8mized store


Changes in the delta store are logged in log les

107

Chapter 5:
Foundations for a New
Enterprise Application
Development Era

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Implications on Application
Development

Dr.-Ing. Jrgen Mller

Simplified Application
Development
Tradi8onal

Column-oriented

Applica8on
cache

No caches needed

No redundant data/
objects

No maintenance of
indexes or aggregates

Data movements are


minimized

Database
cache

Prebuilt
aggregates

Raw data

111

SAP ERP Financials on


In-Memory Technology
In-memory column database for an enterprise system

Combined workload (parallel OLTP/OLAP queries)

Leverage in-memory capabili8es to

Reduce amount of data

Aggregate on-the-y

Run analy8c-style queries (to replace materialized views)

Execute stored procedures

112

Use Case: SAP ERP Financials solu8on

Post and change documents

Display open items

Run dunning job

Analy8cal queries, such as balance sheet

Current Financials
Solutions

113

The Target
Financials Solution
Only base tables, algorithms, and some indexes

114

In-Memory Financials on
SAP ERP
accounting documents

BKPF

secondary indices

BSEG

BSAD

BSAK

BSAS

BSID

dunning data

MHNK

MHND

CDHDR

CDPOS

BSIS

115

In-Memory Financials on
SAP ERP (prototype)
BKPF

BSEG

116

GLT0

change documents

BSIK

accounting documents

sum tables

LFC1

KNC1

Feasibility of Financials on
In-Memory Technology
Modica8ons on SAP Financials

Removed secondary indices, sum tables and pre-calculated and materialized


tables

Reduce code complexity and simplify locks

Insert Only to enable history (change document replacement)

Added stored procedures with business func8onality

European division of a retailer

ERP 2005 ECC 6.0 EhP3

5.5 TB system database size

Financials:

23 million headers / 8 GB in main memory

252 million items / 50 GB in main memory


(including inverted indices for join a#ributes and insert only extension)

117

Reduction by a Factor 10

118

DBMS

IMDB

BKPF

8.7 GB

1.5 GB

BSEG

255 GB

50 GB

263.7 GB

51.5 GB

Secondary Indices

255 GB

Sum Tables

0.55 GB

Complete

519.25 GB

51.5 GB

Dunning Run

Dunning run determines all open and due invoices


Customer dened queries on 250M records
Current system: 20 min
New logic: 1.5 sec

In-memory column store


Parallelized stored procedures
Simplied Financials

119

Why?

120

Being able to perform the dunning run in such a short 8me


lowers TCO
Add more func8onality!
Run other jobs in the mean8me! - in a mul8-tenancy cloud
setup hardware must be used wisely

Bring Application Logic Closer


to the Storage Layer

Select accounts to be dunned, for each:

Select open account items from BSID, for each:


Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Create and write dunning item tables


121

Bring Application Logic Closer


to the Storage Layer

Select accounts to be dunned, for each:

Create and write dunning item tables


122

Select open account items from BSID, for each:


Calculate due date
Select dunning procedure, level and area
Create MHNK entries

1 SELECT
10,000 SELECTs
10,000 SELECTs
31,000 Entries

Bring Application Logic Closer


to the Storage Layer

Select accounts to be dunned, for each:

1 SELECT
One
single
Select open account items from BSID, for each: stored procedure
10,000 SELECTs
Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Create and write dunning item tables

10,000 SELECTs
31,000 Entries


123

Bring Application Logic Closer


to the Storage Layer

Select accounts to be dunned, for each:

1 SELECT
One
single
Select open account items from BSID, for each: stored procedure
10,000 SELECTs
Calculate due date
Select dunning procedure, level and area
Create MHNK entries

Create and write dunning item tables


124

10,000 SELECTs
31,000 Entries

Calculated on-the-y

Results
Original Version needed about 20 minutes
Factor 800x accelera8on achieved
#
Opera(on

HANA2 Version

Select Open Items


0.63s

Due date, dunning level


Variant 2

Variant 3

1.01s
0.6s
(incl. T047 & KNB5 Join)
(incl. T047 & KNB5 Join)

27s

deferred to aggrega8on

0.5s

Filter 1 (Verify Dunning levels)


19s

1.1s

0.5s

Filter 2 (Check Last Dunning)


15s

0.8s

0.4s

Generate MHNK (Aggregate)


done in #1

1.2s

done in #1

Generate MHND (Execute Filters)


done in #1

140ms

done in #1

Total

1 Minute

3.0s

(#3, #4 exec. in parallel)


1.5s

(#3, #4 exec. in parallel)


Hardware: 4 CPUs x 6 Cores (Intel Dunnington), 256GB RAM



Customer Data: 250mio line items, 380k open, 200k due

125

Dunning Application

126

Dunning Application

127

Available-to-Promise
Check

Can I get enough quan88es of a requested product on a


desired delivery date?
Goal: Analyze and validate the poten8al of in-memory and
highly parallel data processing for Available-to-Promise (ATP)
Challenges

Dynamic aggrega8on
Instant rescheduling in minutes vs. nightly batch runs
Real-8me and historical analy8cs

Outcome

128

Real-8me ATP checks without materialized views


Ad-hoc rescheduling
No materialized aggregates

In-Memory Available-to-Promise

129

Demand Planning

Flexible analysis of demand


planning data
Zooming to choose granularity
Filter by certain products or
customers
Browse through 8me spans
Combina8on of loca8on-based
geo data with planning data in an
in-memory database
External factors such as the
temperature, or the level of
cloudiness can be overlaid to
incorporate them in planning
decisions

130

GORFID

HANA for Streaming Data Processing


Use Case: In-Memory RFID Data Management
Evalua8on of SAP OER
Prototypical implementa8on of:

RFID Read Event Repository on HANA


Discovery Service on HANA (10 billion
data records with ca. 3 seconds response 8me)
Frontends for iPhone, iPad2

Key Findings:

HANA is suited for streaming data


(using bulk inserts)
Analy8cs on streaming data is now possible

131

Near Real-Time as a
Concept

ReadEvent
Event
Read
Read
Event
Repositories
Repositories
Repositories

Verification
Verification
Verification
Services
Services
Services

8,000read
read
upupto
toto8.000
8.000
up
read
eventnotifications
notifications
event
event
notifications
persecond
second
per
per
second

Bulk load every


2-3 seconds:
> 50,000 inserts/s

upto
to2.000
2,000
up
up
to 2.000
requests
requests
requests
persecond
second
per
per
second
DiscoveryService
Service
Discovery
Discovery
Service

SAP HANA
HANA
SAP
SAP HANA

PPA
132

A
A

Learning Map
The Future of
Enterprise
Compu(ng
Founda(ons of
Database Storage
Techniques

In-Memory
Database
Operators

Founda(ons for a
New Enterprise
Applica(on
Development Era

Advanced
Database
Storage
Tech-
niques

Views

Dr.-Ing. Jrgen Mller

Dynamic Views
Presentation Layer

Excel

Business
Objects
Explorer

SAP

View

View Layer
(Calculations, Filter, ...)

Any Software

View

View

...

View

View

Node Tables

Persistency Layer

View

View

Other DB

Node Tables

Node Tables

Node Tables

Store

(Main Memory)

DB
Persistence

Restart

Object Hierarchy
Write Complete
Objects

Recovery

Logical
Log

135

Learning Map
The Future of
Enterprise
Compu(ng

In-Memory
Database
Operators

Advanced
Database
Storage
Tech-
niques

Founda(ons of
Database Storage
Techniques

Founda(ons for a
New Enterprise
Applica(on
Development Era

Bypass Solution

Dr.-Ing. Jrgen Mller

Bypass Solution Allows a Smooth


Transition to In-Memory Technology

140

Millions of old un-op8mized lines of code at the


customers site Transi8on required
Row-store replacement
Part-for-part replacement with bypass
Transform row-store to column-store on the y
Change of applica8on code

Bypass Solution for Transition

Todays System

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB

Excel

w/ Cubes

Traditional DB

141

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel

Todays System with IMDB

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB
w/ Cubes

Traditional DB

IMDB

SSD

Excel

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on

Todays System with IMDB

Traditional BI
SAP

ERP

ETL

OLAP
Engine

BIA

Business
Objects

...

Traditional DB

Excel

w/ Cubes

Traditional DB

IMDB

SSD

New
Applications

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB

Todays System with IMDB

Traditional BI with IMDB


SAP

ERP

EL

OLAP
Engine

BIA

Business
Objects

...

IMDB
w/o cubes

Traditional DB

IMDB

SSD

New
Applications

Excel

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views

Todays System with IMDB

Traditional BI with IMDB


SAP

ERP

EL

OLAP
Engine

BIA

Business
Objects

...

IMDB
w/o Cubes

Traditional DB

IMDB

Excel

BI 2.0

OLAP

Engine

Business
Objects

New
Applications

...

SSD

SAP

Excel

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views
STEP 5: Eliminate all the traditional BI, virtualize all in-memory BI, using non-materialized views

Todays System with IMDB


ERP

Traditional DB

BI 2.0

IMDB
OLAP

Engine

New
Applications

Business
Objects

...

SSD

SAP

Excel

Bypass Solution for Transition


STEP 1: Install and run the in-memory database in parallel
STEP 2: Generate business value from the first day on
STEP 3: Re-create traditional BI w/o cubes in IMDB
STEP 4: Introduce next-gen BI running in parallel without materialized views
STEP 5: Eliminate all the traditional BI, virtualize all in-memory BI, using non-materialized views
STEP 6: Run ERP and BI on IMDB

Future System with IMDB


ERP Future
Releases
OLTP & OLAP
BI 2.0

IMDB
OLAP

Engine

New
Applications

Business
Objects

...

SSD

SAP

Excel

You might also like