Netezza Interface Library

ii

Netezza Interface Library

Contents
Netezza Interface Library . . . . . . . 1
Netezza Interface write operator . Netezza data load methods . . nzload method . . . . . External table method . . . Write modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 2 Limitations of write operation Character set limitations . . Bad input records . . . . . Error logs . . . . . . . Syntax for nzwrite operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 3 3 3

iii

iv

Netezza Interface Library

Netezza Interface Library

Netezza Enterprise stage is a database stage. You can use this stage to write bulk data to Netezza Performance Server 8000 (NPS). The Netezza Enterprise stage uses the Netezza Interface write operator (nzwrite) to write data to Netezza Performance Server. This chapter provides a detailed description of the Netezza Interface write operator. It also describes how to use the Netezza Interface write operator (nzwrite) to write data to Netezza Performance Server. It is intended for users who are familiar with the Orchestrate (OSH) language utilized by the DataStage parallel engine.

Netezza Interface write operator

The Netezza Interface write operator, writes data to Netezza Performance Server. Because Netezza does not provide a native read operator, retrieved data is then saved in a WebSphere DataStage data set that the Netezza Inteface stage reads from. The data is then written to the Netezza Performance Server database.

Data flow

Netezza data load methods

The Netezza Interface write operator employs two alternate methods to load data to Netezza Performance Server such as nzload method and External Table method. You can write data to a Netezza Performance Server database either by using a native load utility or by writing data to an external table before writing it to the database.

nzload method
You can use this load utility if the data in the source database is consistent, contains no default values, and uses a predefined format. The prerequisite to use the nzload method is that, nzclient utilities and ODBC functionality must be installed in the DataStage server.

nzload is utility provided by Netezza to load data into Netezza Performance Server. The retrieved data from the source database is fed to the nzload utility to load the destination database on the Netezza Performance Server. This method supports only single-byte characters. By default, Netezza Enterprise stage uses this load method.

External table method

If the data source that you want to read contains default values for table columns and uses variable format for data encoding such as UTF-8, you can write the data to an external table before loading it into the Netezza Performance Server database. If you select the External table method, nzwrite creates an external table in the Netezza Performance Server. Data from the DataStage data set is written into the external table and from here it is streamed to the destination table of the Netezza Performance Server.

Write modes
You can specify a write mode to determine how the records of the DataStage data set are inserted into the destination table in the Netezza Performance Server database. append Appends new rows to the specified table. To use this mode, you must have TABLE CREATE and INSERT privileges on the database that is being written to. Also the table must exist and the record schema of the data set must be compatible with the table. This mode is the default mode. create Creates a new table in the database. To use this mode, you must have TABLE CREATE privileges. If a table already exists with the same name as the one that you want to create, the step that contains this mode ends in error. The table is created with simple default properties. To create a table that is partitioned, indexed, in a non-default table space, or to create a customized table, you must use the -createstmt option with your own create table statement. replace Drops the existing table and creates a new one in its place. To use this mode, you must have TABLE CREATE and TABLE DELETE privileges. If another table exists with the same name as the one that you want to create, the existing table is overwritten. truncate Retains all the attributes of a table (including its schema), but discards the existing records and appends new records into the table. To use this mode, you must have DELETE and INSERT privileges on that table.

Limitations of write operation

During the write operation, the rows in the source database are mapped and streamed into the destination database. The columns of the source database are not mapped individually. So you cannot interchange the order of the columns while writing to the destination database. The Netezza Interface write operator expects the column format in the destination database to be identical to the schema of the source.

Character set limitations

Netezza Performance Server supports only ASCII table names and column names. The following character sets are supported: v UTF-8 for NCHAR and NVARCHAR data types v LATIN9 for CHAR and VARCHAR data types

Netezza Interface Library

Bad input records

Bad input records are the records that might be corrupted in the source database. If the Netezza Interface write operator encounters such records during the write operation, the write operation will end in an error. To ignore the corrupted records and continue the write operation, you can specify the number of times to ignore the corrupt records before the write operation stops. The Netezza Interface write operator ignores the bad records and continues to write data into the Netezza Performance Server until the number of bad input records equals the number specified.

Error logs
You can view the error logs to identify errors that occur during any database operations, and view information about the success or failure of these operations. By default the log files are created in the /tmp directory. While writing data to the Netezza Performance Server by using the nzload method, the log files are created in the /tmp directory on the client computer. The following names are used for the log files: v /tmp/database name.table name.nzlog v /tmp/database name.table name.nzbad While writing data to the Netezza Performance Server by using the External Table method, the log files are created in the /tmp directory in the Netezza Performance Server. The following names are used for the log files: v /tmp/external table name.log v /tmp/external table name.bad Note: The log files are appended every time an error occurs during the write operation.

Syntax for nzwrite operation

Syntax for the Netezza Interface write operator is given below. The option values that you provide are shown in italics. When the specified value contains a space or a tab character, you must enclose it in single quotes. Following are the required options. -server <Server Name> Specify the name of the Netezza server to connect to. -data_source <data source name> Specify the data source to be used for all database connections. -table <tablename> Specify the table name in the database to write data to. The table name can be fully qualified. -loadmethod <ET/nzload> Specify a load method. -mode <append|create|replace|truncate> Specify a write mode: append Appends new records into an existing table. create Creates a table. replace Drops the existing table and create a new table in its place. The schema of the new table is determined by the schema of the Orchestrate data set.
Netezza Interface Library

truncate Deletes all the records from an existing table before new records are written. Refer Write Modes for more information about these modes. Following are the conditional Netezza Interface write operator options: -user <user name> Specify the user name used to connect to the data source. This parameter is required if the data source requires a user name. -password <password> Specify the password used to connect to the data source. This parameter is required if the data source requires a password. -dbname <database name> Specify the name of the database to which to write. This option is applicable only when you are writing data using nzload method. Following are the optional Netezza Interface write operator options: -delim <delimiter character> Specify the delimiter character. The default delimiter is @ (at sign) character. You can use any ASCII character except blank and hyphen (-). The hyphen is the default delimiter for the date/time/timestamp parameter. -open <open_command> Specify an SQL statement to run after the insert array is processed. The statements are run only once on the conductor node. -close <close_command> Specify an SQL statement to run before the insert array is processed. You cannot commit work by using this option. The statements are executed only once on the conductor node. -createstmt <create_statement> Specify the CREATE SQL statement for creating the table when - mode create is specified. Use this command when you want to override the default write statements of the Netezza Interface write operator. -truncate If you set this option, column names are truncated to the maximum size allowed by the ODBC driver. -truncatelen <n> You can specify the length of the column names to which they should be truncated when the data is written into the destination table. This option can be used only with the -truncate option. -drop If you set this option, the Netezza Interface write operator drops unmatched input fields of the DataStage data set. An unmatched field is a field for which there is no identically named field in the destination table.

Example: syntax to replace a table

The syntax for nzwrite is as follows. The option values that you provide are shown in italic. nzwrite -loadmethod nzload -mode replace

Netezza Interface Library

-data_source NZSQL -dbname tempdb -server netezza1 -user user1 -password password -table test1

Netezza Interface Library