EXPDP and IMPDP || One Stop Reference (Oracle 10g + 11g)

Prepared By Ajinkya S. Halwalkar

Difference between Original export import utilities and Data Pump export import utilities:
EXPDP and IMPDP operates on group of files (dump file sets) rather than on a sequential dump file EXPDP and IMPDP access files on servers than on clients, which increases performance but also needs directory objects specifying locations EXPDP and IMPDP uses parallel executions which increases performance EXPDP and IMPDP are self tuning utilities, i.e. BUFFER and RECORDLENGTH parameter used in EXP and IMP are no longer needed EXPDP and IMPDP has not options to merge extents when you recreate tables (COMPRESS parameter is used in EXP and IMP to attain the same). When using Data Pump, the extents are allocated according to the storage parameters of the target table Medias such as taped and pipes are not supported in Data Pump Moving data between different versions of database is attained in a different manner in Data Pump (This topic is covered on later in the document) When you are importing using Data Pump, any data that is appended to the table, if it violates any active constraint, then the load is discontinued and data is not loaded. This is exactly opposite of IMP, which would continue the load of data in case a constraint is violated and logs the violated row If a compressed data is imported, then Data Pump ensures that the data that is imported is in compressed format after the import. IMP loads the same data but not in compressed format. Data Pump consumes more UNDO space as compared to EXP IMP. This is due to the additional metadata queries during the export and long running master table queries during the import. Its wise to increase the undo tablespace size and also if possible the undo retention parameter

Getting started with Data Pump:

A) Data Pump Components:
Command line clients expdp and impdp -- These clients invoke the Data Pump export and import utility respectively. DBMS_DATAPUMP PL/SQL package (Data Pump API) The procedures contained in this package are used by expdp and impdp clients to execute import and export commands DBMS_METADATA PL/SQL package (Metadata API) Whenever data or metadata is moved, Data Pump uses the functionality provided by metadata API. This package provides a central repository for extraction, manipulation and resubmission of dictionary metadata
Note -- Security limits for using DBMS_METADATA Non privileged users can see metadata of only their own objects, if they need to for all the objects then the user should be a sys or it should have a SELECT_CATALOG_ROLE assigned to it. Security limits for using DBMS_DATAPUMP Users should be having a EXP_FULL_DATABASE and IMP_FULL_DATABASE role associated with them.

B) Features of Data Pump EXPDP and IMPDP:

** Dumpfiles generated by Data Pump utility are not compatible with dump file generated by the original export utility

Data Pump operations happens on the Oracle database server, hence increasing performance unlike in exp which are client based Data Pump can estimate as to exactly how much space an export job can consume. This is achieved by using the parameter ESTIMATE_ONLY (More on this later on) Through Data Pump, you can also specify the version of the database objects to be moved using VERSION parameter. This parameter also applies during an import job, when you try to export the data from the source database over the network. (More on this later on) Has a command line interface which helps in monitoring and interacting with the ongoing Data Pump jobs. (More on this later on) Data Pump support export and import over the network using NETWORK_LINK parameter (More on this later on) Supports start and stop of Data Pump export import jobs using START_JOB and STOP_JOB Data Pump can specify the maximum number of threads of active executing jobs using PARALLEL parameter (More on this later on, however, PARALLEL option can only be used in Oracle Enterprise edition) Data Pump can detach or reattach to the long running jobs Data Pump can change the name of the source datafile to a different datafile in all the DDL commands referenced by using REMAP_DATAFILE parameter (More on this later on)

Data Pump supports remapping of tablespaces during imports by using REMAP_TABLESPACE parameter (More on this later on)

C) Data Pump accessing data:

Data Pump supports two access methods to load and unload table row data

1. Direct Path:
The default method that Data Pump uses for loading and unloading data is direct path, when the structure of a table allows it. If the table has any columns of datatype LONG, then direct path must be used.

When Direct Path Load Is Not Used:

A global index on multipartition tables exists during a single-partition load. This includes object tables that are partitioned. A domain index exists for a LOB column. A table is in a cluster. There is an active trigger on a pre-existing table. Fine-grained access control is enabled in insert mode on a pre-existing table. The table has encrypted columns The table into which data is being imported is a pre-existing table and at least one of the following conditions exists:

o o o o

There is an active trigger The table is partitioned A referential integrity constraint exists A unique index exists

Supplemental logging is enabled and the table has at least 1 LOB column.

When Direct Path Unload Is Not Used

Fine-grained access control for SELECT is enabled. The table contains encrypted columns. A column of type LONG or LONG RAW that is not last.

** These are just some of the examples wherein direct path load and unload is not used

2. External Table:
This allows reading of data sources external to the database. As of Oracle Database 10g, the external tables feature also supports writing database data to destinations external to the database. Data Pump provides an external tables access driver (ORACLE_DATAPUMP) that reads and writes files. The format of the files is the same format used with the direct path method. This allows for high-speed loading and unloading of database tables.

External Tables are used in the following situations:

Loading tables with global or domain indexes defined on them, including partitioned object tables Loading tables with active triggers or clustered tables Loading and unloading tables with encrypted columns Loading tables with fine-grained access control enabled for inserts

** These are just some of the examples wherein external tables is used

So now, enough about the theoretical definitions lets get down to business and start with the actual scenarios.

PRE-REQUISITES:
However before starting with the scenarios, lets get the pre-requisites ready, which are:
1. Test databases on Oracle 10g (preferably 10.2.0.4)(I have two test databases, ajinkya1 and ajinkya2 on 10.2.0.4, and I would be using these for the rest of the demonstrations)

2.

Listener and tnsnames.ora configured. Following are the details of the same on my machine
## Listener.ora

SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (GLOBAL_DBNAME = ajinkyas) (ORACLE_HOME = c:/oracle11g/11g) (SID_NAME = ajinkyas) ) (SID_DESC = (GLOBAL_DBNAME = ajinkya1) (ORACLE_HOME = c:/oracle10g) (SID_NAME = ajinkya1) ) (SID_DESC = (GLOBAL_DBNAME = ajinkya2) (ORACLE_HOME = c:/oracle10g) (SID_NAME = ajinkya2) ) ) LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521)) ) (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 127.0.0.1)(PORT = 1521)) ) ) ## Tnsnames.ora AJINKYA1 = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 127.0.0.1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = ajinkya1))) AJINKYA2 = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 127.0.0.1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = ajinkya2)))

3.

Creation of test schemas with following roles assigned EXP_FULL_DATABASE and IMP_FULL_DATABASE

4.

Creation of directories on my test databases registered to my backup locations and assigning read-write rights on those directories for the test schemas

Note -- READ or WRITE permission to a directory object means only that the Oracle database will read or write that file on your behalf. You are not given direct access to those files outside of the Oracle database unless you have the appropriate operating system privileges. Similarly, the Oracle database requires permission from the operating system to read and write files in the directories.

Since these pre-requisites are met, lets finally take a plunge into the scenarios of EXPDP and IMPDP ..

SCENARIOS:

Listed are some of the scenarios that we would be attempting during this documentation

FULL database export Schema export ESTIMATING export size Moving data across different versions using VERSION Export and Import to INCLUDE and EXCLUDE certain objects and privileges Using PARALLEL parameter Using REMAP_TABLESPACE parameter Using NETWORK_LINK parameter Performing Cross schema import Using REMAP_DATAFILES parameter Using TRANSFORM parameter Using CONTENT parameter Using TABLE_EXISTS_ACTIONS parameter

Lets start with EXPDP and IMPDP scenarios:

Scenario I:
FULL database export & import

Script used:

Table Export:

Scenario II:
Schema export

Script used:

Scenario III:
Estimating export Size (for full database)

Script used:

Script used:

Estimating export Size (for a schema)

Scenario IV:
Moving database across different versions

Earlier, whenever we try to import data from a higher version of database to a lower version using the old export/import utility, it was needed that we use the lower version of the export utility to take an export of the higher version and then import the same using the lower version of the import utility. However, when we use Data Pump utilities, these issues are handled automatically by using the VERSION parameter. Here is what I did, I had a schema AJ3 in my 11.1.0.6 database, I needed to move this schema in my 10.2.0.4 version database, so instead of using expdp of 10.2.0.4 version, I would still use the expdp of my 11.1.0.6 version by using the below mentioned scripts and when I would be needing to import I would import the same into my 10.2.0.4 database using the import utility of 10.2.0.4 version without any hassles. Script used:

For importing the schema in 10.2.0.4 database, I move the dumpfile in my directory used for taking backups, and issue the following command Script used:

Scenario V:
Export to include and exclude certain objects and privileges

If while taking a schema level export and we require to skip any tables during an export, we can use the below mentioned scripts. If there are more than 1 table, that needs to be skipped then you can use : expdp logfile=aj1_expdp:schema_aj1.log schemas=aj1 exclude=table:\"like \'TEST%\'\", table:\"like \'EMP%\'\", table:\"like \'T%\'\" Script used:

Scenario VI:
Export & Import using PARALLEL (can be used for schema or table or full database level export or import)

This is the feature is used in data pump to enhance performance per se. Data pump assigns the same number of active workers for the expdp or impdp jbs as specified in the PARALLEL parameter Script used: Full database export:

Table level export:

Schema level export:

Same way the parallel clause can also be used in IMPDP

Scenario VII:
Import using REMAP_TABLESPACE

Case I have a user ajinkya in database ajinkya1, whose default tablespace is ajinkya. Now my requirement is that I need to move this entire schema in database ajinkya2, under the tablespace users. Ideally, what we used to do prior to datapump, was to edit the create table statements and then modify the tablespace clause. Post data pump we can use REMAP_TABLESPACE clause. Following are the scripts that were used for the same: Output from my database ajinkya1:

Export script used:

Import script used:

Output from my database ajinkya2

Scenario VIII:
Import using REMAP_DATAFILE

This is generally used, when you are trying to move a database from one server to another, and the destination database has got a different file system. This allows the change of the name of the source filename contained in the dumpfile to a target environment in all SQL statements where the source file is referenced example: create tablespace, create directory. This is only used when you have to a do a full database import Following are my datafile location for my database ajinkya1

Following are my datafile location for my database ajinkya3

Import command used for the full database export of ajinkya1

Contents of parfile remap_datafile="'C:\ORADATA\AJINKYA1\SYSTEM01.DBF':'C:\ORADATA\AJINKYA3\SYSTEM 01.DBF'","'C:\ORADATA\AJINKYA1\UNDOTBS01.DBF':'C:\ORADATA\AJINKYA3\UNDOTBS01. DBF'","'C:\ORADATA\AJINKYA1\SYSAUX01.DBF':'C:\ORADATA\AJINKYA3\SYSAUX01.DBF'"," 'C:\ORADATA\AJINKYA1\USERS01.DBF':'C:\ORADATA\AJINKYA3\USERS01.DBF'","'C:\ORAD

ATA\AJINKYA1\AJINKYA01.DBF':'C:\ORADATA\AJINKYA3\AJINKYA01.DBF'" full=y table_exists_action=replace

Scenario IX:

Import using REMAP_SCHEMA

Used to load all the objects from one schema to some other schema, however the source tablespace has to be available on the destination database Script used

However, got the following error during import:

So created the tablespace at the destination database:

Started the import and it got completed successfully

Table ash.ash got imported into aj2 schema

Scenario X:
Export using NETWORK_LINK

I have 2 databases, ajinkya1 and ajinkya2, I logged on to ajinkya2 as aj2 user and create a dblink to database ajinkya1 which would connect as user aj1 on ajinkya1 database.

The error is because I have already created a database link by that name.

On my database ajinkya1, I create a new directory call aj which is mapped on to c:\oradata\ajinkya2\expdump, and the user aj1 had been given read write privileges on that directory

Now use expdp from database ajinkya2, to take an export backup of aj1 schema in database ajinkya1

Scenario XI:
Import using NETWORK_LINK

Have to move the objects of user aj1from ajinkya1 database to aj2 in ajinkya2 database.

The only point noting hear is that just give the directory to impdp command and not the actual dumpfile parameter, as giving it results in the following error:

Scenario XII: Using CONTENT in expdp and impdp

CONTENT has got the following values a. all (DEFAULT) . It unloads all the data and metadata b. data_only . It unloads on the data and not the object definitions c. metadata_only. It unloads only the object level definitions and not the data

expdp using content=metadata_only

expdp using content=data_only

impdp using content=metadata_only

Impdp using content=data_only

This error occurs as the export dump only had data to unload and no metadata, so when I try to import the same it errors out saying that object doesnt exists

Scenario XIII: Using TRANSFORM in impdp

This is used to nullify the PCTSPACE, storage structures and segment attributes that are present by default. Also when you have a table in T1 tablespace and while importing you want that table to be imported in T2 tablespace, then also you can use the transform parameter
Example: I have a table called ash in my database, following is its ddl: (select dbms_metadata.get_ddl('TABLE','ASH','ASH') from dual;) CREATE TABLE "ASH"."ASH" ( "OWNER" VARCHAR2(30), "OBJECT_NAME" VARCHAR2(128), "SUBOBJECT_NAME" VARCHAR2(30), "OBJECT_ID" NUMBER, "DATA_OBJECT_ID" NUMBER, "OBJECT_TYPE" VARCHAR2(19), "CREATED" DATE, "LAST_DDL_TIME" DATE, "TIMESTAMP" VARCHAR2(19), "STATUS" VARCHAR2(7), "TEMPORARY" VARCHAR2(1), "GENERATED" VARCHAR2(1), "SECONDARY" VARCHAR2(1) ) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "ASH"

I take an export and import it into another database, while importing I would be including transform parameter as segment_attributes:n:table (for ignoring the default settings of the table)

expdp

impdp

DDL for the imported table is CREATE TABLE "AJ2"."ASH" ( "OWNER" VARCHAR2(30), "OBJECT_NAME" VARCHAR2(128), "SUBOBJECT_NAME" VARCHAR2(30), "OBJECT_ID" NUMBER, "DATA_OBJECT_ID" NUMBER, "OBJECT_TYPE" VARCHAR2(19), "CREATED" DATE, "LAST_DDL_TIME" DATE, "TIMESTAMP" VARCHAR2(19), "STATUS" VARCHAR2(7), "TEMPORARY" VARCHAR2(1), "GENERATED" VARCHAR2(1), "SECONDARY" VARCHAR2(1) ) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "USERS"

Here if you can see the table got created in the default tablespace of user AJ2

Now lets check what happens when I use transform=pctspace:30

DDL for the imported table: CREATE TABLE "AJ2"."ASH" ( "OWNER" VARCHAR2(30),"OBJECT_NAME" VARCHAR2(128), "SUBOBJECT_NAME" VARCHAR2(30),"OBJECT_ID" NUMBER,"DATA_OBJECT_ID" NUMBER,"OBJECT_TYPE" VARCHAR2(19),"CREATED" DATE,"LAST_DDL_TIME" DATE,"TIMESTAMP" VARCHAR2(19), "STATUS" VARCHAR2(7),"TEMPORARY" VARCHAR2(1),"GENERATED" VARCHAR2(1),"SECONDARY" VARCHAR2(1)) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGINGSTORAGE(INITIAL 24576 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645 PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT) TABLESPACE "ASH"

Now if you check it out here, the table didnt get created in the default tablespace of the user AJ2. This is because we opted out the transform=segment_attributes:n:tables, however if we would have include it then impdp would have ignored the pctspace=30 parameter

Scenario XIV: Using table_exists_action in impdp

Options for table_exists_actions a. SKIP Leaves the table as is and move to the next object b. APPEND Loads rows from the source and leaves existing rows unchanged c. TRNCATE Deletes existing rows and loads new rows d. REPLACE Drops the existing table and then creates and loads it from the source Points to remember: If going for replace or truncate, there shouldnt be rows with referential integrities When using append, data is always inserted in new space, even if any old space is available it wont insert there When using append, skip, truncate all the dependent objects like index, constraint, triggers are ignored, however, when replacing the dependent objects are dropped and recreated impd using table_exists_action=skip

impdp using table_exists_action=append

Count of rows before the import:

Count of rows after the import:

impd using table_exists_action=truncate

impdp using table_exists_action=replace

Scenario XV:
Checking the DDL statements using impdp:

impdp dumpfile=directory_location/exp.dmp sqlfile=directory_location/ddl.sql

DataPump using interactive mode:

IMPDP -- In interactive-command mode, the current job continues running, but

logging to the terminal is stopped and the Import prompt is displayed. To start interactive-command mode, do one of the following: From an attached client, press Ctrl+C. (Just be a bit careful, pressing Ctrl+C more than once can cause the datapump operation to terminate also) From a terminal other than the one on which the job is running, use the ATTACH parameter to attach to the job. This is a useful feature in situations in which you start a job at one location and need to check on it at a later time from a different location. Lists of activities that can be performed for the current job from the Data Pump Import prompt in interactive-command mode:

CONTINUE_CLIENT: Changes the mode from interactive-command mode to

logging mode. In logging mode, the job status is continually output to the terminal. If the job is currently stopped, then CONTINUE_CLIENT will also cause the client to attempt to start the job

EXIT_CLIENT: Stops the import client session, exits Import, and

discontinues logging to the terminal, but leaves the current job running. Because EXIT_CLIENT stops the client session but leaves the job running, you can attach to the job at a later time if it is still executing or in a stopped state. To see the status of the job, you can monitor the log file for the job or you can query the USER_DATAPUMP_JOBS view or the V$SESSION_LONGOPS view.

I start an expdp process and name that job as ajinkya

I then enter the interactive mode, check the status of the job and then exit from the client executing the export

Now to again, check the status of my export job, I use ATTACH as one of my export parameter which makes me enter my earlier expdp process:

I check the status of my job by using status command:

 KILL_JOB - Detaches all currently attached client sessions and then kills
the current job. It exits Import and returns to the terminal prompt. A job that is killed using KILL_JOB cannot be restarted. All attached clients, including the one issuing the KILL_JOB command, receive a warning that the job is being killed by the current user and are then detached. After all clients are detached, the job's process structure is immediately run down and the master table and dump files are deleted. Log files are not deleted

STOP_JOB: Stops the current job either immediately or after an orderly

shutdown, and exits Import. If the master table and dump file set are not disturbed when or after the STOP_JOB command is issued, the job can be attached to and restarted at a later time with the START_JOB command. To perform an orderly shutdown, use STOP_JOB (without any associated value). A warning requiring confirmation will be issued. An orderly shutdown stops the job after worker processes have finished their current tasks. To perform an immediate shutdown, specify STOP_JOB=IMMEDIATE. A warning requiring confirmation will be issued. All attached clients, including the one issuing the STOP_JOB command, receive a warning that the job is being stopped by the current user and they will be detached. After all clients are detached, the process structure of the job is immediately run down. That is, the master process will not wait for the worker processes to finish their current tasks. There is no risk of corruption or data loss when you specify STOP_JOB=IMMEDIATE. However, some tasks that were incomplete at the time of shutdown may have to be redone at restart time.

I attach to the same job and check its status

START_JOB: The START_JOB command restarts the job to which you are
currently attached (the job cannot be currently executing). The job is restarted with no data loss or corruption after an unexpected failure or after you issue a STOP_JOB command, provided the dump file set and master table remain undisturbed. The SKIP_CURRENT option allows you to restart a job that previously failed to restart because execution of some DDL statement failed. The failing statement is skipped and the job is restarted from the next work item. Transportable-tablespace-mode imports are not restartable.

 EXPDP
Apart from the ones mentioned for IMPDP, EXPDP has additional following two parameters:

FILESIZE: This helps us to define the size of the export dump as per our
requirement.

ADD_FILE: Add additional dump files

Since, my dumpfile is of 10M only, once filled, my expdp jobs would go into an ideal state, so now, I once again get my session attached to that job and add another dumpfile to the same so that my expdp process reinitiate and completes itself

DICTIONARY VIEWS used for IMPDP and EXPDP:

There are several data dictionary views available that allow you to manage and monitor Data Pump operations. These views are listed and described in the following table: View Name DBA_DATAPUMP_JOBS DBA_DATAPUMP_SESSIONS DATAPUMP_PATHS Purpose/Description Displays information on running Data Pump jobs. Also comes in the USER_DATAPUMP_JOBS variety. Provides session-level information on Data Pump jobs. Provides a list of valid object types that you can associate with the include or exclude parameters of expdp or impdp.

DATAPUMP 11gR1 Features:

Dumpfile sets compressions: This is a licensed product for both Enterprise

and standard editions. During expdp COMPRESSION parameter is used to attain this feature. This would compress the entire dumpfile sets that have been generated

COMPRESSION parameters can have the following options: a. ALL Enables compression for the entire operations b. METADATA_ONLY Default. Only metadata is compressed c. DATA_ONLY Only data being written to the dumpfile set would be compressed d. NONE No compression takes place The COMPATIBLE initialization parameter should be set to "11.0.0" or higher to use these options, except for the METADATA_ONLY option, which is available with a COMPATIBLE setting of "10.2".

Datapump Encrytion Enhancements:

Data pump encryption is an Enterprise Edition feature. COMPATIBLE initialization parameter must be set to "11.0.0" or higher to use these features. Parameter used for these ENCRYPTION enhancements are:

ENCRYPTION = {ALL | DATA_ONLY | ENCRYPTED_COLUMNS_ONLY | METADATA_ONLY | NONE}

Where:

ALL: Both metadata and data are encrypted. DATA_ONLY: Only data is encrypted. ENCRYPTED_COLUMNS_ONLY: Only encrypted columns are written to the dump file in an encrypted format. METADATA_ONLY: Only metadata is encrypted. NONE: Nothing is encrypted ENCRYPTION_ALGORITHM

{ AES128 | AES192 | AES256 }

 ENCRYPTION_MODE

{ DUAL | PASSWORD | TRANSPARENT }

Where DUAL: This mode creates a dump file that can be imported using an Oracle Encryption Wallet, or the the ENCRYPTION_PASSWORD specified during the export operation. This is the default setting if the ENCRYPTION_PASSWORD parameter is set and there is an open wallet. PASSWORD: This mode creates a dump file that can only be imported using the ENCRYPTION_PASSWORD specified during the export operation. This is the default setting if the ENCRYPTION_PASSWORD parameter is set and there isn't an open wallet. TRANSPARENT: This mode creates an encrypted dump file using and open Oracle Encryption Wallet. If the ENCRYPTION_PASSWORD is specified while using this mode and error is produced. This is the default setting of only the ENCRYPTION parameter is set. Example:

REUSE_DUMPFILES: The REUSE_DUMPFILES parameter can be used to prevent errors

being issued if the export attempts to write to a dump file that already exists.

By using reuse_dumpfiles=y, we wont encounter this error

REMAP_TABLE: This parameter allows a table to be renamed during the import operations performed using the TRANSPORTABLE method

Took an export backup of a table called aj3test using

Now, I would import the same table and rename it to test1 using remap_table parameter, however existing tables are not renamed, only tables created by the import are.

***************** ***************** ***************** ***************** ***************** *****************