SecureFiIe Performance Page 2 1. INTRODUCTION .............................................................................................. 3 2. USING SECUREFILES..................................................................................... 3 REQUIREMENTS .......................................................................................................... 3 SPECIFYING SECUREFILE.......................................................................................... 34 LOB Storage Clause ............................................................................................. 4 XMLTvpe Storage Clause ..................................................................................... 4 JARRAY Storage................................................................................................... 4 3. MIGRATION TECHNIQUES .......................................................................... 4 A SIMPLE EXAMPLE ................................................................................................... 5 Sample Table ........................................................................................................ 5 4. MIGRATION WITH ONLINE REDEFINITION .......................................... 5 MIGRATING AN ENTIRE TABLE.................................................................................... 6 MIGRATING AN ENTIRE TABLE AND ENABLING ADVANCED SECUREFILE FEATURES. 7 ONLINE REDEFINITION OF A PARTITIONED TABLE ONE PARTITION AT A TIME........... 8 5. MOVING TO SECUREFILES WITHOUT MIGRATION ......................... 11 ALTER TABLE . ADD PARTITION.................................................................. 11 CREATING PARTITIONS FROM A NON-PARTITIONED TABLE...................................... 12 6. CONCLUSION................................................................................................. 13 SecureFiIe Performance Page 3 SecureFiles Migration
1. NTRODUCTON In RDBMS 8i, Oracle introduced the ability to store large unstructured and semi- structured data as Large Objects ,LOBs,. In 11gR1, we introduce a new LOB storage mechanism Oracle Secureliles. It enables lile system-like perormance or LOBs. 1he old LOB storage mechanism is still aailable and is now call Basicliles. In general, Oracle customers hae seen perormance improements o 3-6x, with some seeing een larger gains, by migrating to Secureliles rom Basicliles. In order to get these gains, data stored in the older ,Basiclile, ormat needs to be migrated to the Securelile storage ormat. Secureliles represents a completely new storage paradigm or Large Objects stored in the Oracle RDBMS. It is a complete redesign and reimplementation o LOB storage. Because o the signiicant changes that were required to all structures and algorithms in all leels o the RDBMS code stack to create the signiicant perormance gains Secureliles demonstrate, it was impossible to keep on-disk compatibility with the original LOB implementation. 1hus, it is impossible to perorm an in-place` migration rom Basicliles to Secureliles. Secureliles are 100 backward compatible with the arious LOB APIs in the system, and migrating to Secureliles requires no code changes to any application. 1his document is intended to help Database Administrators and other technical proessionals decide the best way to migrate rom Basicliles to Secureliles. 2. USNG SECUREFLES Requirements 1o use Secureliles, the Oracle RDBMS parameter compatible` must be set to 11.1.0.0.0 or higher. Also, the tablespace the Securelile LOB segment ,column,partition,sub-partition, resides in must be conigured as SLGMLN1 SPACL MANAGLMLN1 AU1O` ,an ASSM tablespace,. Adanced Secureliles eatures mentioned in this document may require the Adanced Compression Option or Adanced Lncryption Option or use. Specifying SecureFiIe 1o speciy that a LOB segment be a stored as a Securelile, you use the SLCURLlILL` keyword as part o the LOB storage clause. 1he SLCURLlILL` SecureFiIe Performance Page 4 keyword is tightly bound to the S1ORL AS` phrase. So, in all cases where LOB storage is being speciied, SLCURLlILL` will directly ollow S1ORL AS` and come beore any other additional keywords or speciiers that might otherwise ollow S1ORL AS`. Let`s look at some SQL DDL snippets to clariy this. LOB Storage CIause 1he SLCURLlILL` keyword comes ater S1ORL AS` and beore the optional segment name: LOB(c_lob) STORE AS SECUREFILE myclob XMLType Storage CIause lor XML1ype storage, the SLCURLlILL` keyword comes ater S1ORL AS` and beore CLOB` or BINAR\ XML`. CLOB storage: XMLTYPE myxmltype STORE AS SECUREFILE CLOB BINAR\ XML ,BLOB, storage: XMLTYPE myxmltype STORE AS SECUREFILE BINARY XML VARRAY Storage lor VARRA\ Storage, again, the SLCURLlILL` keyword ollows S1ORL AS`, and either a LOB segment name or other storage options must be speciied: VARRAY arr1 STORE AS SECUREFILE LOB arr1lob Or alternatiely: VARRAY arr1 STORE AS SECUREFILE LOB (CACHE)
3. MGRATON TECHNQUES 1here are seeral ways to migrate your applications to use Secureliles. \ou can migrate entire tables to use Securelile storage or you can migrate a partition at a time. 1he methods to migrate to Secureliles are as ollows: 1. Online Redeinition - \e recommend Online Redeinition as the preerred method o migration as it allows the database and the table being migrated to be online during the migration process. 1here are two ways o migrating using Online Redeinition: a. Redeine the entire table b. Redeine a partition at a time SecureFiIe Performance Page 5 2. AL1LR 1ABLL ADD PAR1I1ION - I you do not hae a need to migrate your existing data to Secureliles, but want the perormance going orward, you can create new partitions with Securelile storage going orward. 3. Creating a partitioned table rom a non-partitioned table - I your table currently isn`t partitioned, it is possible to leae all the current data as-is and partition your table so that new data is stored as Secureliles. \e highly recommend one o the two Online Redeinition methods o migrating to Secureliles. A SimpIe ExampIe 1hroughout this document, we will be using a simple partitioned table with one LOB column or our examples. 1he example is kept simple or clarity o explaining the principles behind the arious ways to migrate rom Basicliles to Secureliles. loweer, the prescribed methodologies should apply to all schemas. SampIe TabIe create table table1( c_id number primary key, c_lob clob ) lob (c_lob) store as (tablespace tbs_1) partition by range(c_id) ( partition tab1_p1 values less than (1000000), partition tab1_p2 values less than (2000000), partition tab1_p3 values less than (3000000), partition tab1_p4 values less than (4000000));
4. MGRATON WTH ONLNE REDEFNTON 1he simplest and recommended method or migrating rom Basicliles ,or LONGs, to Secureliles is to use Online Redeinition. 1here are many beneits to using Online Redeinition or migration: 1., 1he data is migrated while this system is up and aailable. 2., \hen complete all LOB data is stored in the Securelile storage ormat, and all the eatures and perormance o Secureliles are immediately aailable to your applications. 3., Addition o eatures such as Securelile Compression, Securelile Deduplication and Securelile Lncryption can be done as the data is being migrated. SecureFiIe Performance Page 6 Online Redeinition requires ree space equal to o the table,partition being migrated. loweer, migrating a single partition at a time can mitigate this. Online redeinition uses materialized iews to make the migration seamless. As the data is migrated, any changes to the source table will be picked up in the destination table. At the end o the migration, the source and destination table names are swapped so that all new updates and queries will operate against the new table. Upon completion, the destination table now holds the original data and can be dropped. Migrating an entire tabIe 1his method works whether or not your table is partitioned. 1o migrate an entire table ia online redeinition, you simply create a new table with the properties you want, in our case the LOB columns are Secureliles, and migrate the data to that table. On completion, you can drop this newly created table, as it will be holding the original, pre- migration, data. During the migration more than 2x the space is required or the original table and LOBs. 1his is because the data will be stored in two locations, the original table and the migration table. Upon completion o the migration, the original data can be dropped and the space can be reused. I compression and deduplication are used on the destination Securelile columns, the amount o space required will be reduced appropriately. Please see the Administrator`s Guide or more inormation,restrictions o online redeinition. Let`s look at a simple example o Online Redeinition o a simple table with a Single LOB column. Our source table deinition: create table table1( c_id number primary key, c_lob clob ) lob (c_lob) store as (tablespace tbs_1) partition by range(c_id) ( partition tab1_p1 values less than (1000000), partition tab1_p2 values less than (2000000), partition tab1_p3 values less than (3000000), partition tab1_p4 values less than (4000000));
Our destination table deinition: create table table1_int( c_id number primary key, c_lob clob ) lob(c_lob) store as SecureFile (tablespace tbs_sf1) partition by range(c_id) ( SecureFiIe Performance Page 7 partition tab1_sf_p1 values less than (1000000), partition tab1_sf_p2 values less than (2000000), partition tab1_sf_p3 values less than (3000000), partition tab1_sf_p4 values less than (4000000)); -- Note: tbs_sf1 is an ASSM tablespace. Notice two signiicant changes to the original table deinition. 1hey are in bold aboe. 1he tablespace is in bold to note that Secureliles must be created in an ASSM tablespace. I you are still using MSSM tablespaces or your ,Basiclile, LOB data, you will need to create or allocate ASSM tablespaces to implement the use o Secureliles. Perorming the actual redeinition is simple. lor our example, we`ll assume the table aboe, and user Scott`. 1. Create the destination table ,table1_int rom aboe,. 2. Start the redeinition: dbms_redefinition.start_redef_table( 'SCOTT', 'TABLE1', 'TABLE1_INT', 'C_ID C_ID, C_LOB C_LOB'); -- <SCHEMA>, <Source table>, <dest_table>, -- <Column mapping> source.col1 dest.col1.
3. Copy the table dependents: dbms_redefinition.copy_table_dependents( 'SCOTT', 'TABLE1', 'TABLE1_INT', 1, true, true, true, false, error_count); 4. linish the redeinition ,swaps the table names,: dbms_redefinition.finish_redef_table('SCOTT', 'TABLE1', 'TABLE1_INT'); 5. ,Optionally, Drop the destination table: DROP TABLE SCOTT.TABLE1_INT; \hen this process completes, 1ABLL1`s LOB column will be a Securelile LOB column, 1ABLL1_IN1 will hae 1ABLL1`s original deinition. 1ABLL1_IN1 can now be dropped to remoe the space used by the original table. Migrating an Entire TabIe and EnabIing Advanced SecureFiIe Features \hen perorming Online Redeinition, it is possible to enable Securelile Compression, Deduplication and,or Lncryption ,with the Adanced Compression and,or Adanced Lncryption Options,. \ou can do this simply by adding one or more o theses options to the LOB storage clause on the destination table: create table table1_int( c_id number primary key, c_lob clob ) lob(c_lob) store as SecureFile ( tablespace tbs_sf1 SecureFiIe Performance Page 8 compress low encrypt deduplicate) partition by range(c_id) ( partition tab1_sf_p1 values less than (1000000), partition tab1_sf_p2 values less than (2000000), partition tab1_sf_p3 values less than (3000000), partition tab1_sf_p4 values less than (4000000)); -- Note: tbs_sf1 is an ASSM tablespace.
\hen Online Redeinition to this destination table is complete the CLOB column will be in stored in a compressed, deduplicated and encrypted Securelile column. \e recommend that you enable any adanced eatures that are desired or the resulting Securelile data during the migration. All o the other migration techniques discussed in this document can also hae the adanced eatures enabled during migration. OnIine Redefinition of a Partitioned TabIe One Partition at a Time I you don`t hae the disk capacity to migrate your entire table at one time, and the table you are migrating is partitioned, you can use online redeinition to redeine each partition one at a time. 1he additional space requirement here is 1x your largest partition 1hus the space required is signiicantly less than the space required to store your entire table and all LOB columns. Lxtra space is only required to hold 1 partition at a time. In this case, we are going to operate on a partitioned table with the ollowing deinition: create table scott.table1( c_id number primary key, c_lob clob ) lob (c_lob) store as (tablespace tbs_1) partition by range(c_id) ( partition tab1_p1 values less than (1000000), partition tab1_p2 values less than (2000000), partition tab1_p3 values less than (3000000), partition tab1_p4 values less than (4000000)); 1he basic methodology here is to create a non-partitioned table with the Securelile column and to migrate each partition indiidually. 1his allows us to do the migration and only requires enough excess space to store that partition`s data. 1he actual steps to perorm the redeinition are as ollows: create table scott.table1_int( c_id number primary key, c_lob clob ) SecureFiIe Performance Page 9 lob(c_lob) store as SecureFile (tablespace tbs_sf1);
declare i number; begin for i in 1 .. 4 loop dbms_redefinition.start_redef_table( 'SCOTT', 'TABLE1', 'TABLE1_INT', NULL, dbms_redefinition.cons_use_pk, '', 'tab1_p' || i); dbms_redefinition.finish_redef_table( 'SCOTT', 'TABLE1', 'TABLE1_INT', 'tab1_p' || i); execute immediate 'drop table scott.table1_int'; execute immediate 'create table scott.table1_int(' || ' c_id number primary key, ' || ' c_lob clob) ' || ' lob(c_lob) store as ' || ' SecureFile ' || ' (tablespace tbs_sf1)'; end loop; end; / Perorming the aboe steps will result in a table with the ollowing deinition: SQL> select dbms_metadata.get_ddl('TABLE','TABLE1','SCOTT') from dual;
CREATE TABLE "SCOTT"."TABLE1" ( "C_ID" NUMBER, "C_LOB" CLOB, PRIMARY KEY ("C_ID") USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE(INITIAL 16384 NEXT 16384 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 50 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" ALTER INDEX "SCOTT"."SYS_C004577" UNUSABLE ENABLE ) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE( BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" LOB ("C_LOB") STORE AS BASICFILE ( TABLESPACE "TBS_1" ENABLE STORAGE IN ROW CHUNK 8192 RETENTION NOCACHE LOGGING STORAGE( BUFFER_POOL DEFAULT)) PARTITION BY RANGE ("C_ID") (PARTITION "TAB1_P1" VALUES LESS THAN (1000000) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 SecureFiIe Performance Page 10 STORAGE(INITIAL 16384 NEXT 16384 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 50 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" LOB ("C_LOB") STORE AS SECUREFILE ( TABLESPACE "TBS_SF1" ENABLE STORAGE IN ROW CHUNK 8192 NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES STORAGE(INITIAL 106496 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 BUFFER_POOL DEFAULT)) NOCOMPRESS , PARTITION "TAB1_P2" VALUES LESS THAN (2000000) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 16384 NEXT 16384 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 50 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" LOB ("C_LOB") STORE AS SECUREFILE ( TABLESPACE "TBS_SF1" ENABLE STORAGE IN ROW CHUNK 8192 NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES STORAGE(INITIAL 106496 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 BUFFER_POOL DEFAULT)) NOCOMPRESS , PARTITION "TAB1_P3" VALUES LESS THAN (3000000) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 16384 NEXT 16384 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 50 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" LOB ("C_LOB") STORE AS SECUREFILE ( TABLESPACE "TBS_SF1" ENABLE STORAGE IN ROW CHUNK 8192 NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES STORAGE(INITIAL 106496 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 BUFFER_POOL DEFAULT)) NOCOMPRESS , PARTITION "TAB1_P4" VALUES LESS THAN (4000000) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 STORAGE(INITIAL 16384 NEXT 16384 MINEXTENTS 1 MAXEXTENTS 505 PCTINCREASE 50 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "SYSTEM" LOB ("C_LOB") STORE AS SECUREFILE ( TABLESPACE "TBS_SF1" ENABLE STORAGE IN ROW CHUNK 8192 NOCACHE LOGGING NOCOMPRESS KEEP_DUPLICATES STORAGE(INITIAL 106496 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 BUFFER_POOL DEFAULT)) NOCOMPRESS ) SecureFiIe Performance Page 11 It is important to note here that een though the partition deinitions changed, the cotvvv deinition did not. 1his means that the deault LOB storage or any uture partitions added will be BASIClILL and SLCURLlILL, as is desired. \ou will hae to explicitly speciy Securelile on any columns that you might add. As we hae demonstrated, the use o online redeinition is simple and can be perormed with the database online. 1his method also allows you to create the new partitions with adanced options such as compression, encryption or deduplication and hae the migration process also perorm those transormations on the data automatically. 1his is why we strongly recommend this method or migrating rom Basicliles to Secureliles. 5. MOVNG TO SECUREFLES WTHOUT MGRATON Another possible method o migrating to Secureliles is through the use o partitions. Secureliles can be enabled on new partitions only, leaing the data in the older ormat as it is. So, new data will be stored in the high perormance Securelile ormat and will be able to leerage adanced eatures. Old data will continue to reside in the old ,Basiclile, ormat and can be migrated at a later date, i desired. ALTER TABLE . ADD PARTITION 1his method can be o particular use i your data is already partitioned by some monotonically increasing alue ,or example by date or ia a sequence,. In this case, you can simply deine any new partitions you create to hae Securelile storage or the LOB columns. lor example, consider our simple range partitioned table: create table scott.table1( c_id number primary key, c_lob clob ) lob (c_lob) store as (tablespace tbs_1) partition by range(c_id) ( partition tab1_p1 values less than (1000000), partition tab1_p2 values less than (2000000), partition tab1_p3 values less than (3000000), partition tab1_p4 values less than (4000000)); \e can add a new partition or alues less than 5000000 with a simple AL1LR 1ABLL . ADD PAR1I1ION: ALTER TABLE table1 ADD PARTITION tab1_p5 VALUES LESS THAN (5000000) LOB(col3) STORE AS SecureFile (TABLESPACE tbs_sf1); Ater this is completed, all LOB data inserted with a c_id alue between 4000000 and 4999999 will be stored as a Securelile. 1his will allow you hae your data going SecureFiIe Performance Page 12 orward be stored in Secureliles. 1his single largest detractor rom this is the older data still has the perormance characteristics o Basicliles since the original partitions are not migrated to Secureliles. So, the perormance characteristics o indiidual LOB accesses will be dierent across partitions. Also, the table deinition didn`t change, so you also must speciy the LOB column is a Securelile eery time you add a partition. 1he adantages here are you make no signiicant changes to your system. Assuming the data that is in the older partitions is reerences less and less, your application will continue to perorm better as more o the data current working set is stored in Secureliles. Creating Partitions from a Non-Partitioned TabIe I you do not hae data that is already partitioned by date or some other strictly increasing or decreasing alue, then you can artiicially create a partitioning scheme by creating an artiicial key` and adding that key column to the existing table beore the partition exchange. lere`s our simple example again, but as a non-partitioned table: drop table table1; drop table table1_sf;
create table table1( c_id number, c_lob clob ) lob (c_lob) store as (tablespace tbs_1);
create sequence table1_sf_key;
create table table1_sf( c_id number, c_lob clob, partkey$ number default 1 ) lob (c_lob) store as SecureFile (tablespace tbs_sf1) partition by range (partkey$)( partition bf values less than (1) lob(c_lob) store as BasicFile, partition sf values less than (MAXVALUE) );
alter table table1 add (partkey$ number default 0);
ALTER TABLE table1_sf EXCHANGE PARTITION bf WITH TABLE table1 WITHOUT VALIDATION UPDATE GLOBAL INDEXES;
DROP TABLE table1; RENAME table1_sf TO table1; SecureFiIe Performance Page 13 \hen this is complete, partition b` contains all the data with the LOBs stored as Basicliles, and all data going orward will be stored as Secureliles. I your table isn`t partitioned, this could be an opportunity to add a more complicated partitioning scheme than the one proided here, perhaps een using the new Interal Partitioning added in Oracle RDBMS 11gR1. 1he key here is that whateer scheme is chosen, all the original LOB data must be all by deinition o the key partitioning into a single partition.
6. CONCLUSON Secureliles is a signiicant improement in perormance and eatures oer the preious RDBMS LOB storage mechanism. 1o take adantage o the perormance and new eatures, the LOB data must be conerted to the new Secureliles ormat. lor migration o existing data rom Basicliles to Secureliles, we recommend online redeinition. 1his can be perormed at the table or partition leel as is necessary or your particular situation. It can be perormed while the system is online and aailable to your applications. loweer, the space taken during the redeinition is roughly equialent to 2x the storage required to store the current tables, LOBs and dependent objects. I you are limited in the aailable disk space such that there isn`t enough space to use temporarily while the online redeinition is being perormed, or i you hae LOB data that is used less oer time, there are other methods o migrating to Secureliles that don`t require additional space.
SecureFiIe Migration May 2008 Author: Scott Lynn Contributing Authors: Vikram Kapoor, Ravi Rajamani
Copyright 2008, OracIe. AII rights reserved. This document is provided for information purposes onIy and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed oraIIy or impIied in Iaw, incIuding impIied warranties and conditions of merchantabiIity or fitness for a particuIar purpose. We specificaIIy discIaim any IiabiIity with respect to this document and no contractuaI obIigations are formed either directIy or indirectIy by this document. This document may not be reproduced or transmitted in any form or by any means, eIectronic or mechanicaI, for any purpose, without our prior written permission. OracIe is a registered trademark of OracIe Corporation and/or its affiIiates. Other names may be trademarks of their respective owners.