Table of Contents
TSM Data Protection for Virtual Environment
https://localhost:9081/TsmVMwareUI
https://www.lascon.co.uk/tsm-vmware.php
TDP VE options: http://www-01.ibm.com/support/docview.wss?uid=swg21634961
Installation procedure: http://www.oucs.ox.ac.uk/hfs/help/tsm4ve/index.xml?splitLevel=-1
Where can we use TDP for VE
TDP for VE can be used to backup VM (Vmware or HyperV), and we can restore a full VM, drives or files. It highly simplify the backup operations and doesn't impact VM. This is also supported for Exchange and MSSQL backup.
Install a local TSM client if you need more granular restore (for example restore a single object into the Active Directory, or a single mail into exchange then use the specific agent).
TSM client with “VMware Backup Tools” component allows to backup and restore VM only full. If you want to use incremental backup, then you need the license file tdp4vmw.lic which is a TDP for VE component (“Enablement file” component into base TDP for VE package, v6.3.0).
How does it works
TDP for VE will be installed on a proxy node or data mover (windows or linux); virtual machine, for little environment or a physical machine connected to the SAN for a large environment. For a virtual machine the VMVSTORTRANSPORT is “HOTADD”; while it's SAN, NDB (network), or NDBSSL for a physical server. This transport mode is the way that the proxy will use to backup the datas from the ESX servers. Traffic from proxy to TSM server is LAN for a proxy into VM, and LAN or Lanfree for a physical proxy server. The proxy server doesn't need cache space like former VCB proxy.
What must be installed
On the proxy node you need the
- TSM baclient with the component “VMware Backup Tools”
- TDP for VE (base level contains license used for incremental backup of VM) and updates
You can have multiples proxy nodes or data movers per virtual center.
How to backup/restore
- Directly from the Virtual Center using the TDP for VE Plugin (which is distribute from proxy node).
- From the proxy node using the baclient (Action –> Backup VM)
- From the proxy node using command line:
dsmc ba vm tsmreport -vmbackuptype=fullvm -mode=full
- Schedules on TSM server.
TDP Plugin for Virtual Center
On the Proxy server you have to configure C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts\
tsm: ARWEN>grant proxy target=LABvc01 agent=LABtdpve
Policy for VE backups
http://www-01.ibm.com/support/docview.wss?uid=swg21659826
Best practice is to store datas on tapes (slow media) and meta data (control files) on disk. In case of restore control files will be load first and must stay on fast storage (copy can be on tape). Control files is about 0,05% of total datas.
You can specify a different Management Class (this mean a different target stgpool), for datas and control files using the following options from vmcliprofile file (C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts\vmcliprofile) and dsm.opt (C:\Program Files\Tivoli\TSM\baclient)
VMMC vmware_dat VMCTLMC vmware_ctl
TSM server definitions
DEFINE MGMTCLASS TRI STANDARD VE-D28-d desc='Disk class with 28 days retention' DEFINE COPYGROUP TRI STANDARD VE-D28-d type=backup vere=28 verdel=1 rete=28 reto=28 dest=FF-DATA-D DEFINE MGMTCLASS TRI STANDARD VE-CTL-d desc='Disk class with 28 days retention – CTL only' DEFINE COPYGROUP TRI STANDARD VE-CTL-d type=backup vere=28 verdel=1 rete=28 reto=28 dest=VE-D28-D
Debug a TSM backup from proxy node:
backup vm vm_name -vmbackuptype=fullvm -mode=Incremental -traceflag=vm,vm_detail,options,exit,enter,stats -tracefile=VM-trace.out
tsm: TSM1> def devc vmctrlfile devt=file mountl=5 maxcap=100M dire=/tsm/tsminst1/stgpool1/vmctrlfile tsm: TSM1> def devc vmdatafile devt=file mountl=100 maxcap=10G dire=/tsm/tsminst1/stgpool1/vmdatafile tsm: TSM1> def stg dskvmctl vmctrlfile po=prim COLl=no maxscratch=20 tsm: TSM1> def stg dskvmdata vmdatafile po=prim COLl=no maxscratch=200
To be able to schedule a job from Virtual Center, the TSM user register into the plugin must have policy authority for the domain that it belong to.
Register nodes to TSM server
tsm: TSM1> reg node vc vc dom=PRDLUX passe=9999 tsm: TSM1> reg node vc_dc1 vc_dc1 dom=PRDLUX passe=9999 keepmp=yes tsm: TSM1> reg node vc_dc1_dm1 vc_dc1_dm1 dom=PRDLUX passe=9999 keepmp=yes tsm: TSM1> reg node vc_vcli1 vc_vcli1 dom=PRDLUX passe=9999
tsm: TSM1> grant proxynode target=vc agent=vc_dc1,vc_vcli1 tsm: TSM1> grant proxynode target=vc_dc1 agent=vc_dc1_dm1,vc_vcli1
C:\Program Files\Tivoli\TSM\baclient>dsm -optfile=dsm_dm1.opt C:\Program Files\Tivoli\TSM\baclient>dsmc set password -type=vm 10.10.10.123 root passwdvc -optfile=dsm_dm1.opt IBM Tivoli Storage Manager Command Line Backup-Archive Client Interface Client Version 6, Release 3, Level 0.18 Client date/time: 11/15/2012 11:56:32 (c) Copyright by IBM Corporation and other(s) 1990, 2012. All Rights Reserved. ANS0302I Successfully done.
C:\Program Files\Tivoli\TSM\baclient\dsm_dm1.opt
NODENAME VC_DC1_DM1 TCPSERVERADDRESS tsmserver TCPCLIENTADDRESS backup_proxy SCHEDMODE PROMPTED PASSWORDACCESS GENERATE SCHEDLOGNAME "C:\Program Files\Tivoli\TSM\baclient\dsmsched_dm1.log" SCHEDLOGRETENTION 7 ERRORLOGNAME "C:\Program Files\Tivoli\TSM\baclient\dsmerror_dm1.log" ERRORLOGRETENTION 7 VMCHOST vc # Virual Center Linux appliance VMCUSER root VMFULLTYPE VSTOR DOMAIN.VMFULL "ALL-VM" MANAGEDSERVICES WEBCLIENT SCHEDULE VMMC "VM63" VMCTLMC "VM63CTL" VMBACKDIR "c:\vmbackup" VMMaxparallel 10 VMLimitperdatastor 10 VMlimitperhost 10 VMVSTORtransport HotAdd Vmprocessvmwithprdm YES VMPROCESSVMWITHINDEPENDENT YES VMenabletemplatebackups YES *ASNODENAME VC_DC1 * do not use this option in this file but only into a shortcut with -asnode to be able to backup and restore VM
Special note for TDP for VE v4
When a virtual machine is included by this option, Tivoli Storage Manager, provides application protection. That is, Tivoli Storage Manager freezes and thaws the VSS writers and truncates the application logs. If a virtual machine is not protected by this option, application protection is provided by VMware, and VMware freezes and thaws the VSS writers, but application logs are not truncated.
Add on the proxy node into dsm.opt file, so snapshot provider that will be used is Windows VSS instead of VMware snapshot:
INCLUDE.VMTSMVSS <vmname>
Then you have to register the admin password for each VM into VMware using the command line:
> dsmc set password -type=vmguest 'VM name' 'mydomain\admin' 'password' Or > vmcli -f set_password -type VMGuest -I password_file.txt
With password_file.txt:
datacentername:DataCenter3 username:Domain1\Administrator password:tdpvmwareuserYpwd
http://tivoli-expert.blogspot.be/2012/12/tsm-64-new-features.html http://www-01.ibm.com/support/docview.wss?uid=swg27036685&aid=3
Create the TSM services on the proxy node:
C:\Program Files\Tivoli\TSM\baclient>dsmcutil list TSM Windows NT Client Service Configuration Utility Command Line Interface - Version 6, Release 3, Level 0.18 (C) Copyright IBM Corporation, 1990, 2012, All Rights Reserved. Last Updated Sep 8 2012 TSM Api Version 6.3.0 Command: List Installed TSM Client Services Machine: IIT-LUITBACKUP(Local Machine) Installed TSM Client Services: 1. TSM Client Acceptor DM1 2. TSM Client Scheduler DM1 3. TSM Remote Client Agent DM1 4. vstor2-mntapi10-shared 4 TSM Client Services were located.
Now you are ready to do full backups of VM (not incremental). If you use hotadd don't change the default option into diskpart (automount must be set to enable).
tsm: TSM1> define sched vmware Daily_Incr Desc=’Daily Incremental Backup of Exchange and SQL VM’ Action=Backup Subaction=VM OPTions=’-vmfulltype=vstor -vmbackuptype=fullvm -asnodename=VC_DC1 -domain.vmfull=”vmfolder=Exchange,SQL” -MODE=INCRemental’ day=Weekday startt=20:00 tsm: TSM1> define sched vmware Weekly_Full Schedstyle=Enhanced Desc=’Weekly Full Backup of Exchange and SQL VM’ Action=Backup Subaction=VM OPTions=’-vmfulltype=vstor -vmbackuptype=fullvm -asnodename=VC_DC1 -domain.vmfull=”vmfolder=Exchange,SQL” -MODE=FULL’ DAYofweek=Saturday WEEKofmonth=’Second,Third,Fourth,Last’ startt=00:00:00 tsm: TSM1> define sched vmware Monthly_Archive Schedstyle=Enhanced Desc=’Host 01 02 Monthly Full Backup – Archive for 1 year’ Action=Backup Subaction=VM OPTions=’-vmfulltype=vstor -vmbackuptype=fullvm -asnodename=VC_DC1 -vmmc=Monthly -domain.vmfull=”VMHOST=host-01.sandbox.local,host-02.sandbox.local” -MODE=FULL’ DAYofweek=Saturday WEEKofmonth=First Startt=00:00:00 tsm: TSM1> define sched vmware Manual_Backup Desc=’Manual Backup all VM except VM1, VM2’ Action=Backup Subaction=VM OPTions=’-vmfulltype=vstor -vmbackuptype=fullvm -asnodename=VC_DC1 -domain.vmfull=”all-vm; -vm=VM1,VM2″ -MODE=FULL’ startd=today startt=now
Configure TDP Plugin
C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts\vmcliprofile
>>> VMCLI VMCLI_TRACE NO VE_TSM_SERVER_NAME arwen # -s #VE_TSM_SERVER_PORT 1500 # -p VE_TSMCLI_NODE_NAME VC_VCLI1 # -n VE_VCENTER_NODE_NAME VC # -v #VE_TRACE_FILE tsmcli.trace # -x tsmcli trace file #VE_TRACE_FLAGS api api_detail # -y trace flags VE_DATACENTER_NAME "My datacenter"::VC_DC1 VMCLI_TASK_EXPIRATION_TIME 864000 # in seconds, defaults to 864000s = 10 days VMCLI_RESTORE_TASK_EXPIRATION_TIME 2592000 # in seconds, defaults to 2592000s = 30 days VMCLI_GRACE_PERIOD 2592000 # in seconds, defaults to 2592000s = 30 days VMCLI_SCHEDULER_INTERVAL 60 # in seconds, defaults to 1s VMCLI_DB_HOST localhost VMCLI_DB_PORT 1527 VMCLI_CACHE_EXPIRATION_TIME 600 # in seconds, defaults to 600s = 10 min VMCLI_DB_NAME VMCLIDB VMCLI_RECON_INTERVAL_FCM 600 # backend specific recon interval setting in seconds default 600s = 10 min VMCLI_RECON_INTERVAL_TSM 1200 # backend specific recon interval setting in seconds default 1200s = 20 min VMCLI_DB_BACKUP AT 00:00 VMCLI_DB_BACKUP_VERSIONS 3 VMCLI_LOG_DIR logs DERBY_HOME C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\derby <<<
Now set the password for the TSM node VC_VCLI1 (on windows command line, don't leave a space between echo command and “>”):
C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts>echo vc_vcli1> pwd.txt C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts>vmcli -f set_password -I pwd.txt Tivoli Storage Manager Command Line Wrapper for Virtual Environments Version: 6. 3.0.00 Build Date: Mon Sep 26 16:14:08 2011 Tivoli Storage Manager API Version 63018 Tivoli Storage Manager Command Line Wrapper Compile Version 63000 #PARAM OPERATION_TYPE 6 #PHASE_COUNT 3 #PHASE PREPARE #PARAM BACKUP_TYPE=0 #PARAM TSM_SERVER_NAME=TSMSRV #PARAM TSMCLI_NODE_NAME=VC_VCLI1 #PARAM VCENTER_NODE_NAME= #PARAM DATACENTER_NODE_NAME= #PARAM OFFLOAD_HOST_NAME= #PARAM TSM_OPTFILE=C:\Users\ADMINI~1\AppData\Local\Temp\2\T4VC2F1.tmp #PARAM INPUT_FILE=C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlu gin\scripts\pwd.txt #PARAM TRACEFILE= #PARAM TRACEFLAGS= #PHASE INITIALIZE #PHASE CHANGE_PASSWORD Password updated. STATUS=success #END #INFO FMM16014I The return code is 0. #END
Validate configuration:
C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts> vmcli -f inquire_config #TASK 4 inquire_config 20121115171144573 #PARAM INSTALLED=TSM #RUN 4 20121115171144573 #LANG en_US #PARAM BACKEND=TSM #PARAM OPERATION_TYPE 4 #PHASE_COUNT 4 #PHASE PREPARE #PARAM BACKUP_TYPE=0 #PARAM TSM_SERVER_NAME=TSMSRV #PARAM TSMCLI_NODE_NAME=VC_VCLI1 #PARAM VCENTER_NODE_NAME=VC #PARAM DATACENTER_NODE_NAME= #PARAM OFFLOAD_HOST_NAME= #PARAM TSM_OPTFILE=C:\Users\ADMINI~1\AppData\Local\Temp\2\T4VC3BC.tmp #PARAM INPUT_FILE= #PARAM TRACEFILE= #PARAM TRACEFLAGS= #PHASE INITIALIZE #PHASE INQUIRE_DATACENTER_NODES #CHILD datacenternode:VC_DC1::VC_DC1 #PARENT vcenternode:VC #PHASE INQUIRE_PROXY_NODES #CHILD targetnode:VC_DC1 #PARENT peernode:VC_DC1_DM1 #CHILD hladdress:vmbackup #PARENT peernode:VC_DC1_DM1 #CHILD lladdress:49169 #PARENT peernode:VC_DC1_DM1 #PARAM STATUS=success #END RUN 4 20121115171145914 #END TASK 4 #INFO FMM16014I The return code is 0. #END
C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VMwarePlugin\scripts>vmcli -f inquire_detail -d VC_DC1 -s trsmsrv -p 1500 -n VC_VCLI1 #TASK 70 inquire_detail 20121116120528436 #RUN 70 20121116120528436 #LANG en_US #PARAM BACKEND=TSM #PARAM OPERATION_TYPE 5 #PHASE_COUNT 4 #PHASE PREPARE #PARAM BACKUP_TYPE=0 #PARAM TSM_SERVER_NAME=TSMSRV #PARAM TSMCLI_NODE_NAME=VC_VCLI1 #PARAM VCENTER_NODE_NAME= #PARAM DATACENTER_NODE_NAME=VC_DC1 #PARAM OFFLOAD_HOST_NAME= #PARAM TSM_OPTFILE=C:\Users\ADMINI~1\AppData\Local\Temp\2\T4V6607.tmp #PARAM INPUT_FILE= #PARAM TRACEFILE= #PARAM TRACEFLAGS= #CHILD backupid:304162919 #PARENT vmname:VM1 #CHILD baseid:0 #PARENT vmname:VM1 #CHILD backupdate:20121115115310 #PARENT vmname:VM1 #CHILD INACTIVE #PARENT vmname:VM1 #CHILD FULL #PARENT vmname:VM1 #CHILD vmdatacenter:"My datacenter" #PARENT vmname:VM1 #CHILD vmguestfullname:Microsoft Windows 7 (64-bit) #PARENT vmname:VM1 #CHILD proxynode:VC_DC1_DM1 #PARENT vmname:VM1 #CHILD vmdataformat:CDF #PARENT vmname:VM1 #CHILD vmproxyostype:Windows #PARENT vmname:VM1 #CHILD vmdisplayname:VM1 .... #CHILD backuporigin:TSM #PARENT vmname:VM1 #CHILD snapshotorigin:TSM_SOFT_ONLY ... #PARAM STATUS=success #END RUN 70 20121116120530121 #END TASK 70 #INFO FMM16014I The return code is 0. #END
Next Step
Works on a proxy node, no need to install additional client into VM
C:\Program Files\Tivoli\TSM\baclient>dsmc ba vm tsmreport -vmbackuptype=fullvm -mode=full C:\Program Files\Tivoli\TSM\baclient>dsmc ba vm tsmreport -vmbackuptype=fullvm -mode=incr
dsmc SET PASSWORD –type=VCB virtcenter.ibm.com Myuser Mypassword dsmc SET PASSWORD –type=VM virtcenter.ibm.com Myuser Mypassword
Works if client installed on the VM and grant proxy set on TSM sever:
dsmc backup vm -vmchost=test.ibm.com -vmcuser=myuser -vmcpw=bad-password C:\Program Files\Tivoli\TSM\baclient>dsmc ba vm tsmreport -vmbackuptype=file -mode=Incremental ANS4160E Proxy Rejected: Proxy authority has not been granted to Agent Node: 'LABTDPVE' for Target Node: 'WIN-L1VVAUKJIKA'. ANS4150E Incremental backup of Virtual Machine 'WIN-L1VVAUKJIKA' failed with RC5722
Read syntax diagramSkip visual syntax diagram
>>-Backup VM--+--------+--+---------+--------------------------->
'-VMNAME-' '-options-'
>--+-------------------------------------------------+---------->
| .- -mode=Full--------. |
'- -vmbackuptype=fullvm----+--------------------+-'
file '- -mode=Incremental-'
>--+--------------------+--------------------------------------><
'-VMMC--{class_name}-'
Syntax
Read syntax diagramSkip visual syntax diagram
.-;------------------------------------.
V .-vmname1,vmname2------------------. |
>>-DOMAIN.VMFUll----+----------------------------------+-+-----><
+-VM=vmname1,vmname2---------------+
+- -VM=vmname1,vmname2-------------+
+-ALL-VM---------------------------+
+-ALL-WINdows----------------------+
+-VMHost=srv1,srv2-----------------+
'-VMFolder=foldername1,foldername2-'
dsm.opt NODENAME LABTDPVE TCPSERVERADDRESS 192.168.10.111 TCPCLIENTADDRESS 192.168.10.107 SCHEDMODE PROMPTED PASSWORDACCESS GENERATE VMCHOST 192.168.10.100 VMCUSER root VMFULLTYPE VSTOR DOMAIN.VMFULL "ALL-VM" MANAGEDSERVICES WEBCLIENT SCHEDULE VMBACKDIR "c:\vmbackup" VMVSTORTRANSPORT "HOTADD" VMCPW **** DOMAIN "\\labtdpve\c$"
DOMAIN.VMFULL option need clarification Technote (FAQ) Question Is it possible to use VMHOST and VMFOLDER options to limit the scope of the DOMAIN.VMFULL to the machines belonging to one ( or multiple ) folders in one ( or multiple) ESX servers ? Cause All VM saved when using both options. Answer No, it is not possible. The VMHOST and VMFOLDER options are additive. That means all VM guests that will match one of the VMHOST or VMFOLDER option will be included in the DOMAIN. The solution to limit the scope of the domain is to use -VM keyword to exclude VM from backup by their name. At this time the -VM keyword does not support wildcards in the vmname and there is currently no -VMFOLDER (minus sign) support. So an alternate method to do this is to use VMCHOST option pointing to one ESX server and define only the VMFOLDER option to limit the scope of the VMFOLDER to only one ESX server , the one pointed by VMCHOST. Each ESX will need to be saved via a separate BACKUP VM command each one with it's own VMCHOST and DOMAIN.VMFULL VMFOLDER options values.
ISCSI config for individual file restore
Troubleshoot
- Change Block Tracking (CBT) not working
How to implement CBT, what do you need ?
VM version 7 at least, No snapshot on your VM Enable CBT, Finally the VM must go through a stun-unstun cycle (power on, resume after suspend, migrate, or snapshot create/delete/revert) before the reconfiguration takes effect.
http://www.lazywinadmin.com/2013/07/enabling-change-block-tracking-cbt-on.html
To do incremental backups on a VM, you have to enable change block tracking. Before VSphere 5.1 CBT was set to false, then update all your VM and template. On TDP for VE, in command line do a:
dsmc show VM ALL > c:\list_vm.txt
And check for Each VM if ctkEnabled is ON.
To update it using command line, you can't edit the file directly into the ESX, because .vmx files are cached. You must copy it on an other machine (scp), to modify and copy back. Just add the following line:
ctkEnabled = "true"
http://www.lazywinadmin.com/2013/01/enabling-change-block-tracking-cbt-on.html
ANS1111W: Incremental backup unsuccessful for virtual machine 'TESTVM'. Performing Full backup instead. ANS1111W: Unable to get VMware Changed Block Tracking(CBT) data for VM TESTVM; Full VM backup will continue, and include both used and unused areas of the disk.
Resolving the problem
If the ctkEnabled option is not present or set to false then it means that the ESX or ESXi host will not allow a backup to utilize the Changed Block Tracking technology.
* Too many virtual disk per virtual SCSI on a VM
ANS9365E vStorage API error during FULLVM backup
Resolving the problem
Spread disks on more virtual SCSI http://www-01.ibm.com/support/docview.wss?uid=swg21459852
References
