Professional Documents
Culture Documents
BY ODI EXPERTS
V1.0 DOCUMENTATION
CONTENTS READING AND WRITING EXCEL WITHOUT ODBC ..................................................... 2 LIMITATION OF THIS DRIVER ............................................................................................ 2 PRELIMINARY STEPS .............................................................................................................. 3 REVERSING THE EXCEL FILE (RKM)................................................................................ 6 SINGLE SHEET HEADER PRESENT AT 5TH ROW ................................................... 7 MULTIPLE SHEETS ............................................................................................................... 9 SELECTIVE MULTIPLE SHEETS ................................................................................... 10 LOADING FROM EXCEL TO DB (LKM) .......................................................................... 11 LOADING EXCEL FILE INTO DB ................................................................................... 11 HANDLING EMPTY ROWS .............................................................................................. 13 INTEGRATION INTO EXCEL (IKM) ................................................................................. 14 WRITING INTO 2003 (XLS)............................................................................................ 16 REWRITING INTO EXISTING EXCEL FILE ............................................................... 20 WRITING HUGE DATA INTO EXCEL FILE ................................................................ 22 LOADING DIRECTLY FROM FILE ................................................................................. 24 ODI EXCEL JDBC DATATYPE ............................................................................................. 25 CURRENCY, NUMERIC, SCIENTIFIC AND PERCENTAGE DATATYPE
SCALE LEVEL........................................................................................................................ 25 HANDLING DATE & DATETIME ................................................................................... 26 HANDLING CURRENCY_SYMBOLS .............................................................................. 27 API ................................................................................................................................................ 28 COMMON ERROR/ISSUES .................................................................................................. 31
In ODI Data / View Data on the data store will not work Excel (XLS) format only allows 65,536 Rows per Sheet and Excel (XLSX) format allows 1,048,576 rows per sheet. Although not a limitation but while writing Rows greater than 100,000 should go for XLSX format, as the process will be Faster and File created will be compressed and smaller in size. Excel 2003 (xls) lacks such capabilities.
PRELIMINARY STEPS
Extract the ODI Excel JDBC Zip file and you will find these files KM_IKM_EXCEL_INSERT_V1.xml KM_LKM_EXCEL_to_SQL_V1.xml KM_RKM_EXCEL_V1.xml ODI_JDBC_Excel_V1.jar TECH_ODI_JDBC_Excel.xml
Copy the ODI_JDBC_Excel_v1.jar under your ODI Agents Driver Folder and restart your Agent .For local, please copy in to your respective USERLIB Folder.
Create a DataServer with just an appropriate Name for it. No Need to provide any other details.
Create a Physical Schema and provide the Directory where the Excel Files are present. [ Note: - You can create n Number of Physical Schema under the same Data Server ]
Finally create a Logical Schema and link them through the appropriate Context.
Login into Designer and Import the RKM, LKM and IKM respectively.
MASK - << File Name with Extension>> SHEET_NO 1, Specify the sheet No. HEADER_ROW 5 [Row No ,where the header is present, as you can see in the above screenshot ] Once reversed the datastore will be similar as shown below,
Name - << File Name with Extension >> Resource Name Excel File Name [Note Please dont modify anything in the First line of the description]
MULTIPLE SHEETS
SAMPLE FILE
MASK - << File Name with Extension>> SHEET_NO % HEADER_ROW 1 [NOTE: - The same header row Value will be used for all the Sheets. In case if different sheets have different values please reverse individually] As you can see from the below image, all the sheets are reversed.
In case you wish to reverse selective multiple sheets you can provide the Sheet-No separated by comma. A sample screenshot is shown below.
Note: - All the columns need to be mapped to staging only. Column mapped on Source is not supported and will throw error. Select the LKM Excel to SQL_V1 and appropriate IKM depending on your Target Technology.
Save and Execute and ones the Execution is successful. Please look into the Target Table for Data. Operator Session Execution of the above Interface
The different options of the IKM are HEADER_ALIGNMENT: You can set the Header Text Alignment .The valid values are LEFT, RIGHT and CENTER. By default LEFT is used. HEADER_BOLD: If true the Header will be bold else Normal text. HEADER_COLOR: You can set the Header color here. By default the plum color is used. The valid color values are AQUA, AUTOMATIC, BLACK, BLUE, BLUE_GREY, BRIGHT_GREEN, BROWN, CORAL, CORNFLOWER_BLUE, DARK_BLUE, DARK_GREEN, DARK_RED, DARK_TEAL, DARK_YELLOW, GOLD, GREEN, GREY_25_PERCENT, GREY_40_PERCENT, GREY_50_PERCENT, GREY_80_PERCENT, INDIGO, LAVENDER, LEMON_CHIFFON, LIGHT_BLUE, LIGHT_CORNFLOWER_BLUE, LIGHT_GREEN, LIGHT_ORANGE, LIGHT_TURQUOISE, LIGHT_YELLOW, LIME, MAROON, OLIVE_GREEN, ORANGE, ORCHID, PALE_BLUE, PINK, PLUM, RED, ROSE, ROYAL_BLUE, SEA_GREEN, SKY_BLUE, TAN, TEAL, TURQUOISE, VIOLET, WHITE, YELLOW REWRITE_EXISTING_FILE: Enable this option if you want to rewrite existing Excel Sheet. WRITE_LARGE_XLSX: If you are loading huge data then enable this option as then JDBC driver uses a different algorithm to load faster and with less memory. DATE_FORMAT: Date format while writing into Excel.
d- Day m - Month y - Year Some of the Valid Format ----------------------------- m/d/yy d-mmm-yy d-mmm mmm-yy DATETIME_FORMAT: Date time format while writing into Excel d - Day m - Month y - Year s - Second h - Hour mm - Minutes Some of the Valid Format ------------------------------ h:mm AM/PM h:mm:ss AM/PM h:mm h:mm:ss m/d/yy h:mm
CURRENCY_SYMBOL: The currency symbol to be used if the datatype is Currency. The default currency is $.
Select the appropriate Datatype and dont worry about the logical length but make sure you select the appropriate Scale as Excel created will use that for creating required decimals data.
Sample Target data for the above source data. Since my Scale for Salary and com_pct was 2, the decimals were added accordingly.
As you can see the Decimal are created with 3 Decimal points.
If Decimal points are not entered then data is create is with no Decimal Values. Make sure you provide the Options else Excel file will be created using default options.
As you can see from the below screenshot, the New sheet is created next to exiting Sheet in the same file.
WRITING_LARGE_XLSX will purge the existing data in the File. We are going to load the below table data (around 3.6 million) into an Excel file.
1048575 * 3 Sheets + 506247 = 3,651,972 Rows which matches the above total Count. [Note: - In above calculation Header row is not included, thats why its 506247]
In the Interface just select the Source File logical schema as Staging Area
In case if the decimal value in the database is more than the scale specified in the Excel the data is displayed accordingly. For example now all the scale is changed to 2.
Now as you can see from the screenshot, the decimal values are displayed to 2 points. Even though the original data is of different scale, but the data is displayed accordingly and no data is truncated as you can see in the formula bar. In case you wish to display the data as it is in the Database, then have the datatype as VARCHAR, which is shown in the below screenshot
While loading data from Excel (LKM), all the above datatype are read as NUMBER and accordingly loaded as number/Integer into Staging or Target Side.
HANDLING CURRENCY_SYMBOLS
For any currency symbol other than $ please provide the appropriate currency symbol in the options as shown in the below examples.
API
UnloadTables You can use this API to unload multiple tables into same or different Excel File. The method is as follows
In the Command on Source Specify the Technology and the Schema. In the Command on Target Select the Technology Jython. Here is an example to unload multiple tables of HR Schema.
import api.api as Excelapi conn=odiRef.getJDBCConnection("SRC") fileName='C:/HR_TABLES.xlsx' schema='HR' tableNames='COUNTRIES,DEPARTMENTS,EMPLOYEES,JOB_HISTORY,JOBS,LOCATIO NS,REGIONS' Excelapi.UnloadTables(fileName,conn,schema,tableNames)
Sample Output
In case you wish to have different schema / different excel File; you can repeat the steps in single or different Command.
COMMON ERROR/ISSUES
We have documented some of the common error/issues which can occur due to invalid entry/options in KM options or Excel File. FileNotFoundException: - This exception will occur when you provided an invalid File Name or File without correct Extension.
Sheet does not display in the Operator : - While trying to Reverse multiple or Individual Sheets and if the session in the operator is successful, yet you dont see the Sheet being reversed there is high chance that Sheet is empty or you have provide an Header Row which is empty . Sheet index is out of range: - If you have provided a sheet No which does not exists in Excel File Sheet index out of range error will occur. Cannot get a numeric value from a text cell: - In case if the Excel data type is Numeric, Currency, Percentage and Scientific but some of the data contains Non numeric values or String in nature this Error can occur. Change the datatype of the Excel to VARCHAR and try reloading the Content. Data does not get loaded from Excel (LKM):- While using LKM Excel to SQL, if your interface is successful yet you dont see any data in target, it means you have set the column mapped to Source. Please set the appropriate column mapping to Staging and try again Loading data into Excel is taking too much time: - If you are loading huge data and taking too much. Try enabling WRITING_LARGE_XLSX .
CONTACT US
We hope that this documentation has helped in the initial steps of using and understanding the ODI JDBC EXCEL driver and KMs. In case if you still have issues or question please feel free to contact us at www.odiexperts.com.