US008555288B2

(12) Ulllted States Patent

Brown et a].
(54) MANAGING DATABASE UTILITIES TO
IMPROVE THROUGHPUT AND

(10) Patent N0.:

(45) Date of Patent:
6,243,736 B1 *
6,249,800 B1 *
6,330,583 B1*

US 8,555,288 B2
Oct. 8, 2013

6/2001 Diepstraten et a1. ........ .. 718/108

6/2001
12/2001

Aman et a1. ..... ..

Reif?n ......... ..

718/105
718/105

CONCURRENCY
(75) Inventors: Douglas P. Brown, Rancho Santa Fe, CA (US); Anita Richards, San Juan Capistrano CA (Us). Bruce Wayne
.

6,560,717 6,950,848 7,020,373 7,082,463 7,093,250

7,552,171

B1* B1* B1 * B1 * B1*

B2 *

5/2003 9/2005 3/2006 7/2006 8/2006

6/2009

Scott et a1. Youse?Zadeh Rhee et a1, ,,,,,,, u Bradley et a1. . Rector ................. ..
Chidambaran et a1.

714/4.1 709/203 713/104 709/223 718/100

.. 709/203

Bnttonvalencla CA ms) Todd A

Walten POWaY CA (Us)

2002/0174379 A1* 11/2002 Korenevsky et a1. ..

2004/0193699 A1* 9/2004 Heymann et a1. ..
10/2004
2/2005

714/19
709/218

2003/0212660 A1* 11/2003 Kerwin ....................... .. 707/1

(73)
( * )

Assignee: Teradata US, Inc., Dayton, OH (US)

Notice: Patent Subject15 ' toextended any disclaimer, or adlusted ' the term under of this 35

2004/0205473 A1* 10/2004 Fisher et a1. ..... ..

2004/0205761 A1*
2005/0038834 A1* i

715/500
718/105
707/203

PaI-tanen .............. ..
ghidambtlrin Souder 0 ram et e a1. a . ........ et . . . .a1~ . . . ..

U-S-C- 154(1)) by 1664 days(21) Appl. N0.: 11/435,523

2005/0055446 A1*
2005/0256971 A1*

3/2005 Chidambaran et a1.

11/2005
6/2008

709/226
709/238

Colrain et a1. ....... ..

2007/0157206 A1*
2008/0133608 A1*

7/2007 Rakvic et a1. .... ..

718/102

Brown et a1. ............... .. 707/200

(22)
(65)

Filed:

May 17, 2006

Prior Publication Data

* Cited by examiner
_ _

Primary Examiner * Emerson Puente

Us 2007/0271570 A1

NOV 22 2007

Assistant Examiner * Willy W Huaracha

(51)

Int Cl

(74) Attorney, Agent, or Firm * Howard Speight

G06F 9/46

(2006.01)

57

ABSTRACT

(52) US. Cl.

USPC .......... .. 718/105; 718/100; 718/ 101; 718/ 102

( )
A method, computer program, and system are disclosed for

(58)

Field of Classi?cation Search

USPC ................................ .. 7 18/ 102, 104, 105, 100

executing a utility on a database system having a plurality of

database system nodes. Each database system node has an

See application ?le for complete Search history,

(56) References Cited
US. PATENT DOCUMENTS
5,603,029 A * 6,041,384 A *
2/1997 Aman et a1. ................ .. 718/105

existing load. An increased load on the database system required to execute the utility is determined. The existing load
on each of the database system nodes is determined. The
increased load is distributed in such a Way as to balance the

load among the database system nodes.

3/2000

Waddington et a1. ....... .. 710/200

24 Claims, 54 Drawing Sheets

REQU ESTS

605
610

l
l
AsslsNEn

[-415
WORKLOAD RULES

ASSIGNMENT OF REQUESTS TO WORKLOAD GROUPS AND PRIORITY CLASSES

REQUESTS

WORKLOAD QUERY (DELAY) MANAGER

WORKGRQUP PERFORMANCE AGAlNsT WORKLOAD RULES

FRIORII'Y cLAssEs

REQUEST PROCESSOR UNDER CONTROL OF PRIORITY SCHEDULER

INFORMATION
EXCEPTION MONITOR
SYSTEM ALLocATI AmusTME

FACILITV (PSF)

625 /

630
EXCEPT. LOGIQUEUE

PROCESSING

REGULATOR

US. Patent

0a. 8, 2013

Sheet 1 0f 54

US 8,555,288 B2

mou z 38?

m2|w_>\<Ez 02K

FI L

Al
_oz2am|\
" 2o: a: No: no:

_owza ml

_ .55302 :0 02

03 N3 .5 N2: |_
208 69% 69M 69 _

0 m

US. Patent

0a. 8, 2013

Sheet 3 0f 54

US 8,555,288 B2

407

WORKLOAD PROFILE

/'

PERFORMANCE TUNING

___>

CAPACITY

PLANNING

ADMINISTRATOR

WORKLOAD RULES

409

REGULATOR

FIG. 4

US. Patent

0a. 8, 2013

Sheet 4 0f 54

US 8,555,288 B2

[- 405
505

QUERY LOG

GUIDE FOR
DBA

515

GUIDANCE CREATION OF /_ <

WORKLOAD RULES

510
EXCEPT LOG/QUEUE

WORKLOAD RULES

409

(INCLUDING SLGs)

FIG. 5
ADMINISTRATOR

US. Patent

0a. 8, 2013

Sheet 5 0f 54

US 8,555,288 B2

REQUESTS

[- 415

605 \

I
WORKLOAD RULES

409

ASSIGNMENT OF REQUESTS
TO WORKLOAD GROUPS AND < PRIORITY CLASSES
ASSIGNED

_\

" REQUESTS

WORKLOAD QUERY (DELAY) MANAGER

_<6M62y\ fMFSZOS
PRIORITY CLASS ES

WORKGROUP PERFORMANCE AGAINST WORKLOAD RU LES

REQUEST PROCESSOR UNDER CONTROL OF PRIORITY SCHEDULER

THROUGHPUT INFORMATION

Y
EXCEPTION MONITOR

FACILITY (PSF)

625 -/

SYSTEM RESOURCE ALLOCATION ADJUSTMENTS

630

PROCESSING

EXCEPT. LOG/QUEUE

FIG. 6
REGULATOR

US. Patent

Oct. 8, 2013

Sheet 6 0f 54

US 8,555,288 B2

ASSIGNED
REQUEST

/_

610

/ 705
wclq'i?gw
A

COMPARATOR FOR WORKGROUP PERFORMANCE DETERMINING IF REQUEST < AGAINST SHOULD BE QUEUED OR

RELEASED FOR EXECUTION

QUEUED REQUEST

409

WORKLOAD

QUEUE

RULES

710 -/
RELEASED

REQUEST

DISPATCHER OF REQUESTS TO PRIORITY CLASSES

620a

/- 715

(A/$62M /620c
l l I I

I_|

(-6205

FIG. 7
WORKLOAD QUERY

(DELAY) MANAGER

US. Patent

0a. 8, 2013

Sheet 7 0f 54

US 8,555,288 B2

409
WORKLOAD THROUGHPUT

[-615
INFORMATION

,___-________

_____________

> WORKLOAD PERFORMANCE | TO WORKLOAD RULES '>

' COMPARATOR

WORKLOAD PERFORMANCE
AGAINST
WORKLOAD

\ 805

RULES

PERFORMANCE GOAL INDEX

(PGI) FOR EACH

g
EXCEPT. LOG/QUEUE

WORKGROUP CALCULATOR

\810 i

SYSTEM RESOURCE ALLOCATION TO REDUCE MAX PGI CALCULATOR

SYSTEM RESOURCE ALLOCATION ADJUSTMENTS

FIG. 8
EXCEPTION MONITOR

US. Patent

0a. 8, 2013

Sheet 9 0f 54

US 8,555,288 B2

mMEg@skma?

US. Patent

0a. 8, 2013

Sheet 12 0f 54

US 8,555,288 B2

Em,gui.,m?w

uQ,mgW5sEwx?m

US. Patent

Oct. 8, 2013

Sheet 14 of 54

US 8,555,288 B2

Bypass Complex Classi?cation

I
New

Request
1 505

I
Workload
Delay Processing

-\
I

Occurs
Here

Classi?cation
R . t ejec Request
v

Execute K
(log warnings)

Prior to .[ixcculion Execution under comm! ofPSF) / /I 1 51 0

/_
_

Exception

'

Monitoring
I

+
Abort

i
Continue

+
Continue

i
Change

Request

(send alert)

(log exception)

Workload 1'

FIG. 15

US. Patent

0a. 8, 2013

Sheet 15 0f 54

US 8,555,288 B2

/1605
REQUEST DETAILS

[-515
1610

I
GIVE DBA: SNAPSHOT OF SYSTEM USAGE AGGREGATING ON, E.G.,

ACCouNTs, APPLICATIONS, ETC.

~

/_

MAPPING OF EXISTING WD-LIKE DEFINITIONS TO WDS

EXISTING WDs

505

QUERY L06

EXCEPT. LOG/QUEUE

510

1615

WD
CLASSIF. OK?
Y 1620

/- 1635
SPLIT/MERGE

REVISED WD

/_
DEFINE SLGs FOR WDs

CLASSIFICATIONS
AND EXCEPTION CRITERIA

/- 1625
1630

DEFINE PSF SETTINGS FOR WDs

FIG. 16

US. Patent

0a. 8, 2013

Sheet 16 0f 54

US 8,555,288 B2

FIG. 18
AFTER MERGE

FIG. 19
AFTER SPLIT

US. Patent

Oct. 8, 2013

Sheet 17 0f 54

US 8,555,288 B2

[- 635
/ 2005
MERGE OR SPLIT FOR ACCOUNTING PURPOSES

QUERY LOG
505

51

LOOK FOR CLUSTERS OF REQUESTS WITHIN WDs

BASED ON WHO, WHAT,

__> WHERE

/ 2025

SPLIT BY CHOSEN CLUSTERS

1
FIG. 20

US. Patent

Oct. 8, 2013

Sheet 18 0f 54

US 8,555,288 B2

/
MAP REQUEST WHO, WHAT, WHERE INFO INTO N-GRID

2105

/_

1020

QUERY LOG
505

EXCEPT. LOG/QUEUE

510

2110

LET DBA DEFlNE CLUSTERS

FIND CLUSTERS OF REQUESTS WITH LEAST IN COMMON WITH OTHER

GROUPS OF REQUESTS

FIG. 21

US. Patent

Oct. 8, 2013

Sheet 19 0f 54

US 8,555,288 B2

/ 620

l
>

/ 2205

PROVIDE HISTORICAL

PERFORMANCE, E.G.,
THROUGHPUT, RESPONSE TIME, AND DEFAULTS (E.G.
95TH PERCENTILE WITH 25% BOOST) AS A STARTING POINT

/_
QUERY
LOG
EXCEPT. LOG/QUEUE

2210

510 LET DBA DEFINE AND REFINE SLGs WITH LIMITS

505

I
FIG. 22