Professional Documents
Culture Documents
Purpose ......................................................................................................................................................... 2 Disclaimer...................................................................................................................................................... 2 Vendor Specific Information .......................................................................................................................... 2 Appliance Software Version Support History ................................................................................................ 3 Interface and Protocol Support Details ......................................................................................................... 3 Software Requirements for Use.................................................................................................................... 3 Earliest Version of NetVault: Backup Required ........................................................................................ 3 Additional Software Required ................................................................................................................... 3 Technical Implementation ............................................................................................................................. 3 Adding Data Domain Virtual Tape Library to NetVault: Backup Server ................................................... 3 Prerequisites......................................................................................................................................... 3 Data Domain VTL Sizing and Setup Guidelines .................................................................................. 3 NetVault: Backup and Data Domain VTL Setup Guidelines ................................................................ 4 Data Domain VTL Cleaning Cycles (Reclamation Cycles or Commonality Factoring)........................ 5 Adding NetVault: Backup Virtual Tape Library Hosted on Data Domain Appliance................................. 5 Prerequisites......................................................................................................................................... 5 NetVault: Backup Configuration Guidelines ......................................................................................... 5 NetVault: Backup Authentication Guidelines........................................................................................ 6 Interface and Protocol Guidelines ........................................................................................................ 7 NetVault: Backup VTL Sizing and Setup Guidelines............................................................................ 7 Using and Managing a NetVault: Backup VTL on Data Domain Unit .................................................. 9 NetVault: Backup VTL and Data Domain Cleaning Cycles................................................................ 10 Summary................................................................................................................................................. 11 Other information .................................................................................................................................... 11 Known Firmware Issues: .................................................................................................................... 11 Device Notes and Caveats................................................................................................................. 11 Appendix A: Sample scripts for reclamation of disk space ......................................................................... 16 NetVault: Backup Virtual Tape Library Hosted on DataDomain VTL / Gateway Series ............................. 16 Windows Platform................................................................................................................................... 16
Page 1 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs PURPOSE
This document details how to deploy a Data Domain Appliance Virtual Tape Library (VTL), or Gateway Series for use with NetVault: Backup (NVBU).
DISCLAIMER
The information in this publication is provided As Is and is intended to be for reference purposes only. BakBone Software, Inc. makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties or merchantability or functional applicability for a particular purpose. BakBone Software, Inc. believes the information in this publication is accurate as of its publication date. The information is subject to change without notice of any kind. Use, copying, and distribution of the software information contained within this document requires a valid and applicable license. This version supersedes and replaces all previous versions. The information serves only as a guide for those configurations and products which BakBone Software, Inc. has qualified and is not the sole resource for system and software configuration. For more information or answers to questions not found in this document, please see BakBone Software, Inc. website, other BakBone Software, Inc. documentation, such as BakBone Software, Inc. Users Guides, relevant vendor documentation, or contact your BakBone Software, Inc. Sales or BakBone Software, Inc. Customer Service representative.
Emulation Vendor: IBM Supported Models: Ultrium-TD3 Ultrium-TD2 Ultrium-TD1 Emulation Vendor: StorageTek Supported Models: L180 1 - 10000 1 - 64 1 - 128 Supported Virtual 0 10 Caps Supported
Slots Libraries Tape Drives Bar Code Type Tape Import/Export Slot
Page 2 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs APPLIANCE SOFTWARE VERSION SUPPORT HISTORY
Date Nov-Dec 2008 Version OS 4.6.0.1-76126
SCSI-LVD
SATA
SAS
Fibre X
iSCSI
CIFS X1
NFS X1
TECHNICAL IMPLEMENTATION
Prerequisites
Refer to the manufactures documentation for additional details on the following prerequisites. Ensure cabling and termination is configured correctly Attach Device to host system Configure interface protocol addressing Power ON Unit, if required, and allow for the completion of the devices POST (Power-On SelfTest) Power on and boot up host computer system
Page 3 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
For example; When using 2.5TB of physical disk space intended to cope with about 4TB of data on a 4 week retention period the options would be to create a VTL with 16TB of space allocated to retain data at any one time or to simply create a VTL with 25TB of space allocation. In the example above we have 2.5TB of physical disk space, so we can define the size of the VTL to 25TB, approximately 10x the physical disk space or 16TB which is enough to accommodate the data set size and respective retention period and does not exceed the physical capacity guidelines outlined above. With the size of the VTL determined, consideration needs to be given to how we create the VTL itself. Each product can handle a set number of streams or emulated drives. We would consider writing to a NVBU VTL virtual drive as a stream. As a rule of thumb, consider creating virtual drives in the VTL only using what is needed to run concurrent jobs. Having too many drives could lead to poor performance. Never exceed the number of streams supported by the product when creating the NVBU VTL. Another best practice is to create media that properly sized and is not too big or too small. Having a large number of small media puts a strain on the NVBU Media Manager process and can cause a Windows 32bit machine to run out of resources.
Drive Selection
1. 2. 3. 4. In the Add Library window, click the Drive Selection tab. Open the Server/Client node to which the device is connected. Open the corresponding drive type node. The available drives are listed under this node. For libraries with a single drive, double-click the drive or right-click it and choose Select. The drive picture, type and details are displayed under Selected Drives. 5. For libraries with multiple drives, perform the following steps to assign the correct drive bay to each drive: a. Enter 1 in the Select for Drive Bay box. b. Under Choose Drives, double-click drive 1 or right-click it and choose Select. The drive picture, type and details are displayed under Selected Drives. c. Repeat steps a and b for the remaining drives, changing the drive number and bay for each drive 6. A default name will be given to the device in the Library Name field. Edit the Library Name if desired. 7. Do NOT save the device. Configuration options should be set before the device is saved. The configuration options are covered in the following section.
Page 4 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
3. In the Performance tab, increase the default Transfer Memory Buffer setting from the default setting of 257 KB. Ensure the operating system can support the new value entered. Important: The amount of memory to assign to transfer buffers (KB) is what is often referred to as the shared memory setting. The shared memory is allocated in 32 KB blocks with one additional byte. The optimum figure is dependent on the drive model, but the minimum value to use is at least 4x the block size setting for the drive. Ensure there is sufficient RAM (Windows) or Kernel Memory (UNIX). Use more shared memory if available and as required. 4. 5. 6. 7. Repeat steps 1 3 for each drive as required. Save device configuration by clicking the Save Details button in the top left corner. Select OK in the Device Added Successfully window. Close the Add Library window.
Adding NetVault: Backup Virtual Tape Library Hosted on Data Domain Appliance
Prerequisites
Refer to the manufactures documentation for additional details on the following prerequisites. Ensure cabling and termination is configured correctly Attach Device to host system Configure interface protocol addressing Power ON Unit, if required, and allow for the completion of the devices POST (Power-On SelfTest) Power on and boot up host computer system NetVault: Backup v8.2 or Higher Data Domain unit configured with a network share of /backup exported via CIFS or NFS. IP address/DNS name of Data Domain device Domain name, username and password of user setup on Data Domain unit to provide full access to the /backup share.
Page 5 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Deselect Check available disk space before creating libraries option as shown below and apply the changes.
Page 6 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Important: If an account has not been previously configured to allow access to network shares for backup then you need to use a user account that is consistent across the NetVault: Backup Server, the network shares, and the Data Domain unit. 5. Restart the NetVault Process Manager Service. This will stop all current running jobs and close the NVBU GUI if it is open.
Page 7 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
For example; When using 2.5TB of physical disk space intended to cope with about 4TB of data on a 4 week retention period the options would be to create a VTL with 16TB of space allocated to retain data at any one time or to simply create a VTL with 25TB of space allocation. So in the example above we have 2.5TB of physical disk space, so we can define the size of the VTL to 25TB, approximately 10x the physical disk space, or 16TB which is enough to accommodates the data set size and retention period and does not exceed the physical capacity guidelines mentioned above. With the size of the VTL determined, consideration needs to be given to how we create the VTL itself. Each product can handle a set number of streams or emulated drives. We would consider writing to a NVBU VTL virtual drive as a stream. As a rule of thumb, consider creating virtual drives in the VTL only using what is needed to run concurrent jobs. Having too many drives could lead to poor performance. Never exceed the number of streams supported by the product when creating the NVBU VTL. Another best practice is to create media that properly sized and is not too big or too small. Having a large number of small media puts a strain on the NVBU Media Manager process and can cause a Windows 32bit machine to run out of resources. Before the library creation takes place create a directory in the /backup share on the Data Domain unit. In our example we have created a directory called VTL. In our example we can create a VTL with 10 drives and 250 slots with a media size of 100 GB each. This gives us approximately 25 TB of usable VTL space.
Page 8 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
In the previous figure, you can see how we use the UNC path to the Data Domain unit to create the NVBU VTL called DD150 in the \backup\VTL directory. Once the NVBU VTL has been created it can be added to the NetVault: Backup Server in the usual way.
In our example NVBU VTL, we should now begin to look into grouping the data to be backed up to work with media groups as target sets. For example, if we have 2 TB of data to backup, done on a weekly full backup basis, with retention of 4 weeks, we will need 8 TB of media. With media at 100 GB each, 80 pieces of media can be added to the target media group. The main reason for using media groups is so we can force the Data Domain unit to re-use media before using blank media. NVBUs media selection algorithm is
Page 9 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Target media in a drive Target media in a slot Blank media Media marked for re-use.
As you can see blank media would be used before media marked for re-use. If we do not re-use media, the Data Domain unit will never be able to recover the space. By re-using media NVBU overwrites the previous contents once the header has been updated and reuses available space which otherwise would not be reclaimable by the Data Domain clean cycle. Keeping spare media that is not grouped blank will keep the usage of the Data Domain unit as low as possible. Because the Automatically Label Blank Media option is disabled, the backup may run out of media before it completes, especially if the data set is growing. This is the trade off when using a Data Domain unit, media management become necessary to allow best use of the unit. To alert administrators that a job has run out of media, NVBUs Global Notification can be used to send an email to the sys-op detailing that there is no suitable media to complete the job. The media request can then be placed on hold in the Device Management window. Blank media can be added to the relevant target media group and the media request taken off hold. While this is not ideal, it will allow the job to complete. Another recommendation would be automating a media utilization report (job ID 13) to be sent to the sysop on a reoccurring basis to track the media usage. This will allow the sys-op to pre-add media to help prevent the above scenario.
Page 10 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Summary
With careful planning and adhering to the best practices, the Data Domain units can be used as an effective disk based backup technology with a small physical footprint offering a large usable capacity. There will always be a trade off between the administration required and the space usage. Although they may be sold as a fit and forget technology, as NAS this can be true, but as a NVBU VTL it will require a management overhead to ensure that continued protection is delivered through the NVBU product within these environments.
Other information
Known Firmware Issues:
None
Page 11 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
With Data Domain de-duplication it is possible to create NVBU VTLs that are larger than size of the physical disk space. To allow this in NVBU open the NVBU Configurator and select the Plugin Options tab. Select the Disk Devices sub tab. Deselect the Check available disk space before creating libraries option as shown below and apply the changes.
NetVault: Backup User Access to Data Domain CIFS Share On the NetVault: Backup Server: 1. Open Windows Control Panel > Administrative > Services. 2. Select the NetVault Process Manager Service, right click it and select Properties. 3. Select the Log On tab. 4. Select This Account to change the local system account to the account details previously setup to the Data Domain unit to allow access to the CIFS share. NOTE: if this has already been done to allow access to network shares for backup then you need to use a user that is consistent across the network shares and the Data Domain unit. Restart the NetVault Process Manager Service. Disk Space Reclamation Blanking of used media in NVBU VTL created on the Data Domain share will not reclaim physical disk space. To reclaim this space the nvmakemedia utility must be used to create a clean identical .media file, in name and size, to replace the old media file. Before this is done all drives must be unloaded and Open Door command issued in NetVault: Backup Device Manager. Once all the desired media have been cleaned, the Close Door option can issued. Check that all media appear correctly in Device Manager and Media Manager. The recovered space will be reflected after the next Data Domain clean cycle completes. Refer to the NetVault: Backup Command Line Interface Users Guide at
Page 12 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
www.bakbone.com/documentation for more information on performing disk space reclamation with a Data Domain unit.
Page 13 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Disk Space Availability Due to the nature of deduplication and various factors surrounding compression yield and disk space utilization, users must avoid running out of physical disk space. Failing to do so will result in erroneous behaviour with the backup software. Refer to NVBU Jira NVG-4306, NVG-4307, and NVG-4308 for details.
Page 14 of 17
NETVAULT: BACKUP APPLIANCE APPLICATION NOTES DATA DOMAIN VTL & GATEWAY SERIEs
Page 15 of 17
APPENDIX A: SAMPLE SCRIPTS FOR RECLAMATION OF DISK SPACE NETVAULT: BACKUP VIRTUAL TAPE LIBRARY HOSTED ON DATADOMAIN VTL / GATEWAY SERIES
Windows Platform
@echo off set nvutil=%ProgramFiles%\BakBone Software\NetVault\util set vtlpath=<path to the NVBU VTL directly> set logfile=%nvutil%\logdd430-1clean.log echo Stage 1 - Completed :: :: ==================================================================== :: All virtual medias in the VTL start with this prefix :: ==================================================================== set media_prefix=<input VTL media prefix> set library_name=<assign VTL name> set media_size=<assign media size> @echo. >> "%logfile%" @echo. >> "%logfile%" @echo ======================================================== >>"%logfile%" @echo Start Media Start>> "%logfile%" date/t >> "%logfile%" time/t >> "%logfile%" @echo ======================================================== >>"%logfile%" echo Stage 2 - Completed :: :: :: ================================================ :: Look for all media in the NVBU VTL with status :: "expired" and only print lines with media infos :: ================================================ "%nvutil%\nvreport.exe" -class "media" -include "%%reuse = yes" -format "%%Label %%MediaGroup" -sort "%%Label" | find "%media_prefix%" > "%tmp%\~media.txt FOR /F "tokens=1,2 delims= " %%i IN (%tmp%\~media.txt) DO ( SET ML=%%i SET GL=%%j call :_clean %%i ) echo Stage 3 - Completed goto :eof :: :: ****************************************************** :: * * :: * Main Routine for every found media * :: * Eventually adapt Timeout * :: * according to machine type * :: * * :: ****************************************************** :_clean @echo ===========================================
@echo Clean Media with Label %ML% @echo Clean Media with Label %ML% >> "%logfile%" @echo =========================================== :: ============================================== :: Open and close I/O Port, to remove a :: remaining media, wait 3 seconds :: ============================================== @echo "%nvutil%\nvopeneeport.exe" -libraryname "%library_name%" "%nvutil%\nvopeneeport.exe" -libraryname "%library_name%" choice /T 1 /C jn /N /D j >NUL: "%nvutil%\nvcloseeeport.exe" -libraryname "%library_name%" choice /T 3 /C jn /N /D j >NUL: echo Stage 4 - Completed :: :: :: ============================================== :: Export expired media and open :: I/O Port. Library cannot load media :: into a drive anymore :: ============================================== echo "%nvutil%\nvexportmedia.exe" -barcode %ML% "%nvutil%\nvexportmedia.exe" -barcode %ML% choice /T 1 /C jn /N /D j >NUL: "%nvutil%\nvopeneeport.exe" -libraryname "%library_name%" echo Stage 5 - Completed :: :: :: =============================================== :: Media is now unreachable for the library :: in I/O Slot, now we cheat NetVault ! :: :: For security reasons: Rename old media first :: After that, create new media with old name :: Deletion in next step :: =============================================== if exist "%vtlpath%\media\%ML%.disabled" erase "%vtlpath%\media\%ML%.disabled" ren "%vtlpath%\media\%ML%" %ML%.disabled "%nvutil%\nvmakemedia.exe" %media_size% mediafiles "%vtlpath%\media\%ML%" "%nvutil%\nvcloseeeport.exe" -libraryname "%library_name%" choice /T 3 /C jn /N /D j >NUL: "%nvutil%\nvblankmedia.exe" -barcode %ML% -wait if %GL% == None ("%nvutil%\nvlabelmedia.exe" -barcode %ML% -newlabel %ML%) else ("%nvutil%\nvlabelmedia.exe" -barcode %ML% -newlabel %ML% -newgroup %GL%) if exist "%vtlpath%\media\%ML%.disabled" erase %vtlpath%\media\%ML%.disabled" choice /T 3 /C jn /N /D j >NUL: echo Stage 6 - Completed goto :eof echo Stage 7 - CompleteD