Professional Documents
Culture Documents
Carl Dudley
University of Wolverhampton, UK
Introduction
Working with Oracle since 1986
Oracle DBA - OCP Oracle7, 8, 9, 10
Oracle DBA of the Year 2002
Oracle ACE Director
Regular Presenter at Oracle Conferences
Consultant and Trainer
Technical Editor for a number of Oracle texts
UK Oracle User Group Director
Member of IOUC
Day job University of Wolverhampton, UK
Carl Dudley University of Wolverhampton, UK
The Drivers
Regulatory Compliance
SOX, Basel II, HIPAA, J-SOX, GLB, Privacy Laws
Adequate ITDATE
controls
RECORDS
ORGANISATIONS
Separation of
Duty
130,000,000
2009-01-20
Heartland Payment Systems
Proof of compliance
94,000,000
2007-01-17
TJX Companies Inc.
Risk assessment and monitoring
30,000,000
2004-06-24
America Online
Insider Threat Estimates
26,500,000
2006-06-24
U.S. Department of Veterans Affairs
60% of threats come from insiders
25,000,000
2007-11-20
HM Revenue and Customs, TNT
65% of internal
threats are undetected
17,000,000
2008-10-06
T-Mobile, Deutsche Telekom
Source -- http://datalossdb.org
Carl Dudley University of Wolverhampton, UK
1
1
1
Decrypt
table key
Decrypted
table key
2
Master
Key
External to
the database
Database
ID
NAME
JOB
SAL
345
COX
263
FORD
481
WOOD
981
WARD
572
BELL
668
HALL
10
11
12
Job Separation
STARTUP
Security DBA
Opens wallet
13
14
Algorithm
Key Size
Parameter
Name
168 bits
3DES168
128 bits
AES128
AES
AES
256 bits
AES256
15
Type
------------------VARCHAR2(10)
VARCHAR2(9)
VARCHAR2(9) ENCRYPT
VARCHAR2(9)
VARCHAR2(9) ENCRYPT
VARCHAR2(10)
TABLE_NAME
---------EMPE
EMPE
EMPE
EMPE
EMPE
EMPE
EMPENC
EMPT
EMPT
COLUMN_NAME
--------------JOB_SALT
JOB_NOSALT
HIREDATE_SALT
HIREDATE_NOSALT
EMPNO_SALT
EMPNO_NOSALT
JOB
JOBSALT
JOBNOSALT
ENCRYPTION_ALG
----------------------------AES 192 bits key
AES 192 bits key
AES 192 bits key
AES 192 bits key
AES 192 bits key
AES 192 bits key
3 Key Triple DES 168 bits key
AES 192 bits key
AES 192 bits key
SALT
---YES
NO
YES
NO
YES
NO
YES
YES
NO
16
17
18
19
Transparent Encryption
SELECT DUMP(jobsalt) FROM empe WHERE ROWNUM < 6;
DUMP(JOBSALT)
---------------------------------------Typ=1 Len=5: 67,76,69,82,75
Typ=1 Len=8: 83,65,76,69,83,77,65,78
Typ=1 Len=8: 83,65,76,69,83,77,65,78
Typ=1 Len=7: 77,65,78,65,71,69,82
Typ=1 Len=8: 83,65,76,69,83,77,65,78
Jobsalt
column
encrypted
SELECT VSIZE(jobsalt)
FROM empe WHERE ROWNUM < 6;
VSIZE(JOBSALT)
-------------5
8
8
7
8
No evidence of encryption
Optimizer is aware of encryption
Underestimates load on CPU
20
Conventional
table
Number of
blocks
22
29
563
173
125
21
Encryption
empjob
empnosalt
empsalt
22
JOB
--------CLERK
SALESMAN
SALESMAN
MANAGER
SALESMAN
MANAGER
MANAGER
ANALYST
PRESIDENT
JOBNOSALT
--------CLERK
SALESMAN
SALESMAN
MANAGER
SALESMAN
MANAGER
MANAGER
ANALYST
PRESIDENT
JOBMID
--------CLERK
SALESMAN
SALESMAN
MANAGER
SALESMAN
MANAGER
MANAGER
ANALYST
PRESIDENT
JOBSALT
--------CLERK
SALESMAN
SALESMAN
MANAGER
SALESMAN
MANAGER
MANAGER
ANALYST
PRESIDENT
LASTENAME
--------SMITH
ALLEN
WARD
JONES
MARTIN
BLAKE
CLARK
SCOTT
KING
23
07
34
Length byte
showing 52
encrypted
characters
Length byte
showing 7
characters
24
Values
--------AAA
BBB
CCC
DDD
EEE
FFF
GGG
X
HHH
X Encrypted without salt
III
X Encrypted with salt
JJJ
25
SELECT file_id,block_id
FROM dba_extents
WHERE segment_name = 'EMPTEST';
FILE_ID BLOCK_ID
------- -------7
11
7
13
7
15
7
17
26
13012C4A
4D75C624
B765BEE0
C93670DF
17382796
84AA4E41
109A3A5D
77074444
9DD941A3
B994DE07
03279521
678B5DB8
9701FDF0
321F8109
48035801
DE2F4385
5286A4C4
324022B6
8AAC1F31
D2259BBC
B3901ABF
41414103
263D8301
033E427D
43430349
EEF700DA
0F7920AE
EFF599BC
011602B5
4969D6E9
BFF989E4
34464646
73213CB4
FFD84631
E3D9212E
E9244848
68FE6297
C74DF1B9
FDE49833
CD9CD845
48E690DD
4A4A4A03
164CC203
4A122CA6
18210733
F28E3443
1464E07F
E7B98C9E
53666A32
45030101
0327759C
47E3D46F
19353C33
EFD4E5C0
CB625933
61369397
2840AB8D
06B36BDA
038C9DBD
32BB1A66
72073567
EB4CD0BC
0313012C
42424203
85E65358
0CBDC26B
D3F2D41F
851B2241
92788311
4403D183
36244545
104F0370
89396194
AFEBEF0A
47AD0BD4
EA0659BF
47474703
2333F7A5
3223540D
34494949
A58D0037
B70430E3
1C926222
03414141
[J,...AAA..L..BBB]
[$.uM..=&.,.JXS..]
[..e.}B>.3.!.k...]
[.p6.I.CCC4......]
[.'8.......d.A"..]
[AN... y.......x.]
[]:......2jfS...D]
[DD.w.......EEE$6]
[.A....iI.u'.p.O.]
[........o..G.a9.]
[!.'.FFF43<5.....]
[.].g.<!s.......G]
[....1F..3Yb..Y..]
[...2.!....6a.GGG]
[.X.HHH$...@(..3#]
[.C/..b.h.k...T#2]
[...R..M.....III4]
[."@23...f..27...]
[1...E...g5.r.0..]
[..%....H..L."b..]
[.....JJJ,...AAA.]
Unencrypted
DATE column
Encrypted DATE
without salt
Encrypted DATE
with salt
27
Actual NO SALT
length AES192
SALT
AES192
NO SALT
3DES168
SALT
3DES168
NUMBER
any
36
52
28
36
DATE
36
52
28
36
VARCHAR2(1)
36
52
28
36
VARCHAR2(9)
36
52
36
44
VARCHAR2(100)
1-7
36
52
28
36
VARCHAR2(100)
8-15
36
52
36
44
VARCHAR2(100)
16-23
52
68
44
52
VARCHAR2(100)
24-31
52
68
52
60
VARCHAR2(100)
45
68
84
68
76
VARCHAR2(100)
63
84
100
84
92
VARCHAR2(100)
72
100
116
100
108
VARCHAR2(100)
81
116
132
108
116
28
29
Presence of Histograms
Encrypted values (tablespace or column) can be seen in histogram views
Oracle automatically creates histograms by default
VARCHAR2 columns are translated into large numerics that are then
truncated
CREATE OR
REPLACE
FUNCTION
fn_hist_str(pi_str_hist
IN NUMBER)
User
written
functions
can reconstruct the number
and then decode it
RETURN
VARCHAR2
Dates
are shown as Julian dates
julian dates
AS
vc_hist
Numbers
not translated
:= are
TO_CHAR(pi_str_hist,'fm'||RPAD('x',50,'x'));
vc_return
VARCHAR2(4000);
TABLE_NAME COLNAME
END_VAL
REAL_VAL MOD_REAL
BEGIN
----------------- ------------------------------------ -------- -------WHILE (ENAME
vc_hist 339046801723754000000000000000000000
is not null )
EMP_HISTO
ALLEM ALLEN
EMP_HISTO
ENAME
344238228918531000000000000000000000
BLAKD BLAKE
LOOP
EMP_HISTO
ENAME:= vc_return||CHR(TO_NUMBER(SUBSTR(vc_hist,1,2),'xx'));
365069608210433000000000000000000000 FORC FORD
vc_return
EMP_HISTO
ENAME
385838479141748000000000000000000000
JONER JONES
vc_hist := SUBSTR(
vc_hist, 3 );
EMP_HISTO HIREDATE
2444591 2444591 2444591
END
LOOP;
EMP_HISTO HIREDATE
2444656 2444656 2444656
RETURN vc_return;
EMP_HISTO
SAL
2500 2500
2500
END;
EMP_HISTO
SAL
3000 3000
3000
V
SELECT TO_DATE(2444591,'j') FROM dual; dicalue sh
tion ow
ary n in
Encrypted data still available in Result Cache after wallet
TO_DATE(244
by is closed
def
aul
----------t
17-DEC-1980
3.8584E+35
Carl Dudley University of Wolverhampton, UK
30
Candidate Columns
Credit card Numbers
First name
Last name
Driver Licence number
Medical and health information (images that include PII)
31
32
33
different ciphertexts
34
35
36
Tablespace Encryption
Physical data files are encrypted but with NO storage overhead
Uses cipher feedback mode (CFB) for space preservation
Oracle will automatically decrypt data before it arrives in the SGA
Does not apply to column-level TDE
Only when data is written to the file system will it be encrypted
True for all disk-based structures
11g Release 2 same master key for tablespace and wallet encryption
37
38
39
40
41
ALGORITHM_
DESCRIPTION
--------------AES 128-bit key
AES 192-bit key
AES 256-bit key
IS_DEFAULT RESTORE_ONLY
---------YES
NO
NO
-----------NO
NO
NO
42
43
3. Dual Mode
RMAN> SET ENCRYPTION IDENTIFIED BY pwd ON FOR ALL TABLESPACES;
44
45
Backup with
encryption
Backup with
compression and
encryption
Not encrypted
Data compressed
Data encrypted
Encrypted with
TDE column
encryption
Data compressed;
encrypted columns are
treated as if they were
not encrypted
Data encrypted;
double encryption of
encrypted columns
Encrypted with
TDE tablespace
encryption
Encrypted tablespaces
decrypted, compressed,
and re-encrypted
Encrypted
tablespaces passed
through to backup
unchanged
Encrypted tablespaces
decrypted, compressed,
and re-encrypted
46
47
Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Starting "SCOTT"."SYS_EXPORT_TABLE_01": SCOTT/******** DUMPFILE = ENC.DMP DIREC
TORY = DP TABLES = EMPT
Estimate in progress using BLOCKS method...
Only Data Pump can
Processing object type TABLE_EXPORT/TABLE/TABLE_DATA
export encrypted data
Total estimation using BLOCKS method: 992 KB
Processing object type TABLE_EXPORT/TABLE/TABLE
Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX
Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported "SCOTT"."EMPT"
338.4 KB
7168 rows
ORA-39173: Encrypted data has been stored unencrypted in dump file set.
Master table "SCOTT"."SYS_EXPORT_TABLE_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SCOTT.SYS_EXPORT_TABLE_01 is:
C:\ENC.DMP
Job "SCOTT"."SYS_EXPORT_TABLE_01" completed with 1 error(s) at 19:21:41
48
49
50
ENCRYPTION value
Action
ENCRYPTED_COLUMNS_ONLY
10g behaviour
METADATA_ONLY
DATA_ONLY
ALL
NONE
No encryption performed
51
ENCRYPTION_MODE
Can be DUAL, TRANSPARENT, PASSWORD as for RMAN
TRANSPARENT prevents setting of ENCRYPTION_PASSWORD
ENCRYPTION_ALGORITHM
Can be AES128, AES2192, AES256
Can be specified only if ENCRYPTION or ENCRYPTION_PASSWORD is set
Tablespaces can be transported only if endianness is same on target
Wallet must be copied to target
Not possible if target already has wallet
52
SQL_REDO
--------------------------------------------------INSERT INTO emp VALUES(1234,'COX',Unsupported type,
7902,'01-JAN-2002',2500,500,30);
SQL_REDO
--------------------------------------------------------update "SCOTT"."ENCEMP" set "ENC_COL" = Unsupported Type,
"NOENC_COL" = 'DDDD' where "ENC_COL" = Unsupported Type
and "NOENC_COL" = 'CCCC' and ROWID = 'AAAOibAAEAAAIsvABe';
53
54
55
ENAME
-----SMITH
ALLEN
WARD
:
JOB
--------CLERK
SALESMAN
SALESMAN
:
Carl Dudley University of Wolverhampton, UK
56
57
58
59
60
61
62
63
64
65
52 characters
66
SELECT COUNT(SUBSTR(y,1,1))
FROM enc;
COUNT(Y)
---------131072
COUNT(SUBSTR(Y,1,1))
-------------------131072
Elapsed: 00:00:01.65
Elapsed: 00:00:01.91
SELECT COUNT(x)
FROM noenc;
SELECT COUNT(SUBSTR(x,1,1))
FROM noenc;
COUNT(X)
---------131072
COUNT(SUBSTR(X,1,1))
-------------------131072
Elapsed: 00:00:00.03
Elapsed: 00:00:00.09
67
Trace Results
SELECT COUNT(X) FROM NOENC
call
count
------- ----Parse
1
Execute
1
Fetch
2
------- ----total
4
Rows
------1
131072
cpu
elapsed
disk
query
current
-------- ---------- -------- ---------- ---------0.01
0.01
0
1
0
0.00
0.00
0
0
0
0.07
0.07
0
1067
0
-------- ---------- -------- ---------- ---------0.08
0.08
0
1068
0
rows
-------0
0
1
-------1
***************************************************************************
SELECT COUNT(Y) FROM ENC
call
count
------- ----Parse
1
Execute
1
Fetch
2
------- ----total
4
Rows
------1
131072
cpu
elapsed
disk
query
current
-------- ---------- -------- ---------- ---------0.00
0.00
0
0
0
0.00
0.00
0
0
0
1.69
1.70
0
1067
0
-------- ---------- -------- ---------- ---------1.69
1.70
0
1067
0
rows
-------0
0
1
-------1
68
69
cpu
elapsed
disk
query
current
-------- ---------- -------- ---------- ---------0.01
0.00
0
0
0
0.00
0.00
0
0
0
0.02
0.02
0
1064
0
-------- ---------- -------- ---------- ---------0.03
0.03
0
1064
0
rows
-------0
0
1
-------1
cpu
elapsed
disk
query
current
-------- ---------- -------- ---------- ---------0.00
0.00
0
0
0
0.00
0.00
0
0
0
1.69
1.69
0
1068
0
-------- ---------- -------- ---------- ---------1.69
1.70
0
1068
0
rows
-------0
0
1
-------1
70
Column
Enc
Tablespace
Enc
Bytes
of data
per row
Storage
(Blocks)
n1
NO
NO
n52
NO
NO
etbs1
NO
Insert Times
Scan Times
CPU
Elapsed
CPU
Elapsed
1664
0.82
1.77
0.03
0.35
52
8192
1.26
7.52
0.05
1.81
YES
1664
1.81
2.04
0.43
1.04
etbs52 NO
YES
52
8192
2.12
14.35
1.18
2.01
ecol
NO
8192
11.53
14.26
5.45
5.73
YES
Every table has one VARCHAR2 column and 1000000 (1M) rows
Tablespace encryption incurs no storage overhead
Encryption is at the block level
Oracle claim performance overhead of around 5%
71
72
Block header
clear data
free space containing data in clear
(to be encrypted)
Workaround:
1. perform column encryption
2. create a new tablespace
3. move ALL objects from the
old to the new tablespace
4. drop the old tablespace from
the database (without
removing the file)
5. use OS to shred the old file
6. remove the file
73
74
75
Carl Dudley
University of Wolverhampton, UK
UKOUG Director
Oracle ACE Director
carl.dudley@wlv.ac.uk
Data Masking
Majority of organisations use customer production data for testing
Often over 10M records
Around use credit card numbers
Common practice to transfer live data to third parties
Live data used as test may be lost
Often no way of knowing due to lack of measures
Non-production environments more susceptible to breaches
Breaches must be disclosed - $240 per record
Tension between requirements of test situations and data privacy (laws)
77
De Identifying Information
Irreversible process
Options available to re-construct
Data is scrambled but still appears realistic
RI is maintained
Implicitly database enforced
Explicitly application enforced (correlated columns)
Uses masking formats and templates
Pre-built format libraries
Randomising facilities
Shuffles
Substrings
User-defined functions
78
Pre-Masking Validation
Formats match column datatypes
Selective masking
SSNO, NINO
Credit card numbers
4929......
3773......
Maintenance of uniqueness
Checks for available space
Take care with partitioned tables
Could suffer row migration
Random masking of six columns in 100M row table
80 minutes
79
80
Column masking
81
Masking Definitions
82
New Features
Support for e-Business suite structures
83
Tutorial
http://www.oracle.com/technology/obe/11gr1_db/security/datamask/datam
ask.htm#o
Encryption versus data masking video
http://www.youtube.com/watch?v=ZDKJysfyGUs
Slide Show
http://www.slideshare.net/oracleil/data-masking-using-enterprise-manager
-4262943
Oracle Data Masking Demo
http://download.oracle.com/technology/products/oem/screenwatches/data
_masking/index.html
84