Professional Documents
Culture Documents
AIX?
A: Linux #/etc/shadow and Solaris # /etc/shadow
Q: Which command would you use to list partitions on Linux?
A: Linux # mount l or # df -kh
Q: Which command/commands would you use to manage installed packages
on RedHat Linux?
A: Linux # rpm
Q: What is the default port for SSH server?
A: 22
Q: Which command/commands would you use to manage installed packages
on Solaris?
A: #pkginfo #pkgrm # pkgadd #pkgchk
Q: What command would you use to install an OS patch on Solaris?
A: #showrev p and #patchadd -p
Q: Which Veritas command would you use to display a list of Veritas
volumes?
A: # vxprint
Q: Which Veritas command would you use to display a list of disks on a
system?
A: # vxdisk list
/dev/rdsk/devicename
6) Once the VTOC is in place now use metareplace command to
replace the faulty meta devices
#metareplace -e d11 devicename
( Where d11 is the
meta device )
...........................................................
4. What is the significance of 51% state database replicas
in SVM?
Ans: A state database is the collections of multiple,
replicated database copies and each copy is considered as a
state database replica.
If the solaris box loses a state database replica, SVM
should figure out which state database replicas still
contain valid data and boot using the valid ones and this
is achieved by using majority consensus algorithm and
according to this algorithm we need half+1 number of state
database replicas before it finds for a valid data.
And for the above reason we need to recreate at least three
state database replicas when we setup a disk configuration.
If all the three database replicas are corrupted meaning we
lose all data stored on svm volumes.
Hence its good to create as many replicas on separate
drives across controllers.
..........................................................
5. What are the common errors you find in Solaris Volume
manager?
Ans: 1) Disks failures (Boot device failures) 2)
Insufficient state database replicas issues 3) Wrong
entries in /etc/vfstab file
..........................................................
6. You have a boot disk under svm, the machine fails to
boot and remains in ok prompt? what could be the possible
reason?
Ans: 1) May be issue with /etc/system file 2) Root file
VCS Troubleshooting
Cluster Not Up -- HELP
The normal debugging of steps includes: checking on status, restarting
if no faults, checking licenses, clearing faults if needed, and checking
logs.
To find out Current Status:
/opt/VRTSvcs/bin/hastatus -summary
This will give the general status of each machine and processes
/opt/VRTSvcs/bin/hares -display
This gives much more details - down to the resource level.
If hastatus fails on both machines (it returns that the cluster is not up
or returns nothing), try to start the cluster
/opt/VRTSvcs/bin/hastart
/opt/VRTSvcs/bin/hastatus -summary
will tell you if processes started properly. It will NOT start processes on
a FAULTED system.
Starting Single System NOT Faulted
If the system is NOT FAULTED and only one system is up, the cluster
probably needs to have gabconfig manually started. Do this by
running:
/sbin/gabconfig -c -x
/opt/VRTSvcs/bin/hastart
/opt/VRTSvcs/bin/hastatus -summary
If the system is faulted, check licenses and clear the faults as
described next.
To check licenses:
vxlicense -p
Make sure all licenses are current - and NOT expired! If they are
expired, that is your problem. Call VERITAS to get temporary licenses.
There is a BUG with veritas licences. Veritas will not run if there are
ANY expired licenses -- even if you have the valid ones you need. To
get veritas to run, you will need to MOVE the expired licenses. [Note:
you will minimally need VXFS, VxVM and RAID licenses to NOT be
expired from what I understand.]
vxlicense -p
Note the NUMBER after the license (ie: Feature name:
DATABASE_EDITION [100])
cd /etc/vx/elm
mkdir old
mv lic.number old [do this for all expired licenses]
vxlicense -p [Make sure there are no expired licenses AND your good
licenses are there]
hastart
If still fails, call veritas for temp licenses. Otherwise, be certain to do
the same on your second machine.
To clear FAULTS:
hares -display
For each resource that is faulted run:
hares -clear resource-name -sys faulted-system
If all of these clear, then run hastatus -summary and make sure that
these are clear. If some don't clear you MAY be able to clear them on
the group level. Only do this as last resort:
hagrp -disableresources groupname
hagrp -flush group -sys sysname
hagrp -enableresources groupname
To get a group to go online:
hagrp -online group -sys desired-system
BEFORE working on DB
Run hastop -all -force
AFTER working on Dbs:
You MUST bring up oracle on same machine
Once Oracle is up, run:
hastart on the same machine as you started the work on (the first on
system with oracle running)
wait 3-5 minutes then run hastart on the other system
If you need the instance to run on the other system, you can run:
hagrp -switch oragrp -to othersystem
Shutting down db machines:
If you shutdown the machine that is running veritas cluster, it will NOT
start on the other machine. It only fails over if the machine crashes.
You need to manually switch the services if you shutdown the
machine. To switch processes:
Find out groups to transfer over
hagrp -display
Switch over each group
hagrp -switch group-to-move -to new-system
Then shutdown machine as desired. When rebooted will start cluster
daemon automatically.
Doing Maintenance on Admin Network:
If the admin network is brought down (that the veritas cluster uses),
veritas WILL fault both machines AND bring down oracle (nicely). You
will need to do the following to recover:
hastop -all
On ONE machine: hastart
wait 5 minutes
On other machine: hastart
Manual start/stop WITHOUT veritas cluster:
THIS IS ONLY USED WHEN THERE ARE DB FAILURES
If possible, use the section on DB Maintenance. Only use this if system
Things to Remember
Service Group: Collection of dependent Resources
Resource: Anything that the end user requires
Resource Type: Collection of the resources with same type
Agents: To manage the Resource Types (Start,Stop and Monitor)
Service Group Online: Child Resource to Parent Resource
Service Group Offline: Parent Resource to Child Resource
LLT Files
/etc/llthosts
/etc/llttab
GAB Files:
/etc/gabtab
Manipulating Service Groups:
1. hagrp -offline AppSG -sys S1 -localclus --> Offline the AppSG only
in S1 system (node)
2. hagrp -offline OracleSG -any --> Offline the OracleSG in all the
systems
3. hagrp -online AppSG -sys S2 -localclus --> Online the AppSG in
node S2
4. hagrp -switch AppSG -to S1 --> AppSG will be moved to node S1
Manipulating Resources:
Useful Commands
SERVICE GROUPS AND RESOURCE OPERATIONS:
Configuring service groups
hagrp add|-delete|-online|-offline group_name
Modifying resources
hares add|-delete res_name type group
hares online|-offline res_name sys system_name
Modifying agents
haagent start|-stop agent_name sys system_name
BASIC CONFIGURATION OPERATIONS:
Service Goups
hagrp -modify group_name attribute_name value
hagrp list group_name
hagrp value attribute_name
Resources
hares -modify res_name attribute_name value
hares -link res_name res_name
Agents
haagent -display agent_name sys system_name
hatype modify
VCS/VxVm vs HACMP/AIX/LVM
VCS Concepts
Concepts
VCS is built on three components: LLT, GAB, and HAD.
LLT (Low-Latency Transport)
veritas uses a high-performance, low-latency protocol for cluster
communications. LLT runs directly on top of the data link provider
interface (DLPI) layer ver ethernet and has several major junctions:
sending and receiving heartbeats
monitoring and transporting network traffic over multiple network
links to every active system within the cluster
load-balancing traffic over multiple links
maintaining the state of communication
providing a nonroutable transport mechanism for cluster
communications.
Group membership services/Atomic Broadcast (GAB)
GAB provides the following:
Group Membership Services - GAB manitains the overall cluster
Phantom Phantom_NIC (
)
Configuration
VCS configuration is fairly simple. The three configurations to worry
about are LLT, GAB, and VCS resources.
LLT
LLT configuration requires two files: /etc/llttab and /etc/llthosts. llttab
contains information on node-id, cluster membership, and heartbeat
links. It should look like this:
# llttab -- low-latency transport configuration file
# this sets our node ID, must be unique in cluster
set-node 0
# set the heartbeat links
link hme1 /dev/hme:1 - ether - # link-lowpri is for public networks
link-lowpri hme0 /dev/hme:0 - ether - # set cluster number, must be unique
set-cluster 0
start
The "link" directive should only be used for private links. "link-lowpri"
is better suited to public networks used for heartbeats, as it uses less
bandwidth. VCS requires at least two heartbeat signals (although one
of these can be a communication disk) to function without complaints.
The "set-cluster" directive tells LLT which cluster to listen to. The llttab
needs to end in "start" to tell LLT to actually run.
The second file is /etc/llthosts. This file is just like /etc/hosts, except
instead of IP->hostnames, it does llt node numbers (as set in setnode). You need this file for VCS to start. It should look like this:
0 daldev05
1 daldev06
GAB
GAB requires only one configuration file, /etc/gabtab. This file lists the
number of nodes in the cluster and also, if there are any
communication disks in the system, configuration for them. Ex:
/sbin/gabconfig -c -n2
tells GAB to start GAB with 2 hosts in the cluster. To specify VCS
communication disks:
/sbin/gabdisk -a /dev/dsk/cXtXdXs2 -s 16 -p a
/sbin/gabdisk -a /dev/dsk/cXtXdXs2 -s 144 -p h
/sbin/gabdisk -a /dev/dsk/cYtYdYs2 -s 16 -p a
/sbin/gabdisk -a /dev/dsk/cYtYdYs2 -s 144 -p h
-a specifies the disk, -s specifies the start block for each
communication region, and -p specifies the port to use, "a" being the
GAB seed port and "h" the VCS port. The ports are the same as the
network ports used by LLT and GAB, but are simulated on a disk.
VCS
The VCS configuration file(s) are in /etc/VRTSvcs/conf/config. The
two most important files are main.cf and types.cf. I like to set
$VCSCONF to that directory to make my life easier. main.cf contains
the actual VCS configuration for Clusters, Groups, and Resources,
while types.cf contains C-like prototypes for each possible Resource.
The VCS configurationis very similar to the C language, but all you are
doing is defining variables. Comments are "//" (if you try to use #'s,
you'll be unhappy with the result), and you can use "include"
statements if you want to break up your configuration to make it more
readable. One file you must include is types.cf.
In main.cf, you need to specify a Cluster definition:
cluster iMS ( )
You can specify variables within this cluster definition, but for the most
part, the defaults are acceptible. Cluster variables include maximum
number of groups per cluster, link monitoring, log size, maximum
number of resources, maximum number of types, and a list of user
names for the GUI that you will never use and shouldn't install.
You then need to specify the systems in the cluster:
system daldev05 ( )
system daldev06 ( )
These systems must be in /etc/llthosts for VCS to start.
You can also specify SNMP settings for VCS:
snmp vcs (
Enabled = 1
IPAddr = 0.0.0.0
TrapList = { 1 = "A new system has joined the VCS Cluster",
2 = "An existing system has changed its state",
3 = "A service group has changed its state",
4 = "One or more heartbeat links has gone down",
5 = "An HA service has done a manual restart",
6 = "An HA service has been manually idled",
7 = "An HA service has been successfully started" }
)
IPAddr is the IP address of the trap listener. Enabled defaults to 0, so
you need to include this if you want VCS to send traps. You can also
specify a list of numerical traps; listed above are the VCS default
traps.
Each cluster can have multiple Service Group definitions. The most
basic Service Group looks like this:
group iMS5a (
SystemList = { daldev05, daldev06 }
AutoStartList = { daldev05 }
)
You can also set the following variables (not a complete list):
FailOverPolicy - you can set which policy is used to determine which
system to fail over to, choose from Priority (numerically based on
node-id), Load (system with the lowest system load gets failover), or
RoundRobin (system with the least number of active services is
chosen).
ManualOps - whether VCS allows manual (CLI) operation on this
Group
Parallel - indicats if the service group is parallel or failover
Inside each Service Group you need to define Resources. These are
the nuts and bolts of VCS. A full description of the bundled Resources
can be found in the Install Guide and a full description of the
configuration language can be found in the User's Guide.
Here are a couple of Resource examples:
NIC networka (
Device = hme0
NetworkType = ether
)
IP logical_IPa (
Device = hme0
Address = "10.10.30.156"
)
The first line begins with a Resource type (e.g. NIC or IP) and then a
globally unique name for that particular resource. Inside the paren
block, you can set the variables for each resource.
Once you have set up resources, you need to build a resource
dependancy tree for the group. The syntax is "child_resource requires
parent_resource." A dependancy tree for the above resources would
look like this:
logical_IPa requires networka
The dependancy tree tells VCS which resources need to be started
before other resources can be activated. In this case, VCS knows that
the NIC hme0 has to be working before resource logical_IPa can be
started. This works well with things like volumes and volumegroups;
groups from host to host. This is one of the most useful VCS tools.
hares controls Resources. This is the finest granular tool for VCS, as
it can add, remove, or modify individual resources and resource
attributes.
Here are some useful things you can do with VCS:
Activate VCS: run "hastart" on one system. All members of the
cluster will use the seeded configuration. All the resources come up.
Swing a whole Group administratively:
Assuming the system you're running GroupA on is sysa, and you want
to swing it to sysb
hagrp -switch GroupA -to sysb
Turn off a particular resource (say, ResourceA on sysa):
hares -offline ResourceA -sys sysa
In a failover Group, you can only online the resource on system on
which the group is online, so if ResourceA is a member of GroupA, you
can only bring ResourceA online on the system that is running GroupA.
To online a resource:
hares -online ResourceA -sys sysa
If you get a fault on any resource or group, you need to clear the Fault
on a system before you can bring that resource/group up on it. To
clear faults:
hagrp -clear GroupA
hares -clear ResourceA
Caveats
Here are some tricks for VCS:
VCS likes to have complete control of all its resources. It brings up all
its own virtual interfaces, so don't bother to do that in your init scripts.
VCS also likes to have complete control of all the Veritas volumes and
groups, so you shouldn't mount them at boot. VCS will fail to mount a
volume unless it is responsible for importing the Volume Group; if you
import the VG and then start VCS, it will fail after about 5 minutes and
drop the volume without cleaning the FS. So make sure all VCScontrolled VG's are exported before starting VCS.
Resource and Group names have no scope in VCS, so each must be a
unique identifier or VCS will fail to load your new configuration. There
is no equivalent to perl's my or local. VCS is also very case sensitive,
so all Types, Groups, Resources, and Systems must be the same every
time. To make matters worse, most of the VCS bundled types use
random capitalization to try to fool you. Copy and paste is your friend.
Make sure to create your Resource Dependancy Tree before your start
VCS or It will creates a problem to your whole cluster.
The default time-out for LLT/GAB communication is 15 seconds. If VCS
detects a system is down on all communcations channels for 15
VCS Intro
Veritas Cluster Server:
Veritas Cluster Server is the industry's leading cross-platform
clustering solution for minimizing application downtime. Through
central management tools, automated failover, features to test
disaster recovery plans without disruption, and advanced failover
management based on server capacity, Cluster Server allows IT
managers to maximize resources by moving beyond reactive recovery
to proactive management of application availability.
System Requirements:
Solaris
Solaris 9 & 10 on SPARC
Solaris 10 on x64
AIX
AIX 5.3
AIX 6.1
LINUX
Red Hat Enterprise Linux (RHEL) 5 on x86 and IBM system p
Novell SUSE Linux Enterprise Server (SLES) 10 & 11 on x86 & IBM
system p
Oracle Enterprise Linux (OEL) 5 on x86
HP-UX
HP-UX 11i version 1/2/3
Windows (Requires Veritas Storage Foundation)
Windows Server 2003 SP2 (x86): Web Edition
Windows Server 2003 SP2 (x86, x64, IA64): Standard, Enterprise,
Datacenter Editions
Windows Server 2003 R2 SP2 (x86, x64): Standard, Enterprise,
Datacenter Editions
Windows Server 2008 SP1 or SP2 (x86, x64): Web, Standard (without
Hyper-V or in guest), Enterprise (without Hyper-V or in guest),
Datacenter (without Hyper-V or in guest) Editions
Windows Server 2008 for Itanium-based Systems SP1 or SP2 (IA64)
Windows Server 2008 R2 (x64): Web, Standard (without Hyper-V or in
guest), Enterprise (without Hyper-V or in guest), Datacenter (without
Hyper-V or in guest) Editions
Windows Server 2008 R2 for Itanium-based Systems (IA64)
Windows 7 (x86, x64): (use with SFWHA client components)
Windows Vista SP1 or SP2 (x86, x64): Ultimate, Enterprise, Business
Editions (use with SFWHA client components)
Windows XP SP2 or SP3 (x86, x64): (use with SFWHA client
components)
Vertias Cluster Server for Windows (Standalone)
Windows Server 2003 SP2 (x86, x64, IA64): Standard, Enterprise,
Datacenter Editions
Windows Server 2003 R2 SP2 (x86, x64): Standard, Enterprise,
Datacenter Editions
Windows 7 (x86, x64): (use with SFWHA client components)
Windows Vista SP1 or SP2 (x86, x64): Ultimate, Enterprise, Business
Editions (use with SFWHA client components)
Windows XP SP2 or SP3 (x86, x64): (use with SFWHA client
components)
Concepts
VCS is built on three components: LLT, GAB, and VCS itself. LLT handles
kernel-to-kernel communication over the LAN heartbeat links, GAB handles
shared disk communication and messaging between cluster members, and VCS
handles the management of services.
Once cluster members can communicate via LLT and GAB, VCS is started. In
the VCS configuration, each Cluster contains systems, Service Groups, and
Resources. Service Groups contain a list of systems belonging to that group, a
list of systems on which the Group should be started, and Resources. A
Resource is something controlled or monitored by VCS, like network
interfaces, logical IP's, mount point, physical/logical disks, processes, files, etc.
Each resource corresponds to a VCS agent which actually handles VCS control
over the resource.
VCS configuration can be set either statically through a configuration file,
dynamically through the CLI, or both. LLT and GAB configurations are
primarily set through configuration files.
Configuration
VCS configuration is fairly simple. The three configurations to worry about are
LLT, GAB, and VCS resources.
LLT
LLT configuration requires two files: /etc/llttab and /etc/llthosts. llttab contains
information on node-id, cluster membership, and heartbeat links. It should look
like this:
# llttab -- low-latency transport configuration file # this sets our
node ID, must be unique in cluster set-node 0 # set the heartbeat
links link hme1 /dev/hme:1 - ether - - # link-lowpri is for public
networks link-lowpri hme0 /dev/hme:0 - ether - - # set cluster number,
must be unique set-cluster 0 start
The "link" directive should only be used for private links. "link-lowpri" is
better suited to public networks used for heartbeats, as it uses less bandwidth.
VCS requires at least two heartbeat signals (although one of these can be a
communication disk) to function without complaints.
The "set-cluster" directive tells LLT which cluster to listen to. The llttab needs
to end in "start" to tell LLT to actually run.
The second file is /etc/llthosts. This file is just like /etc/hosts, except instead of
IP->hostnames, it does llt node numbers (as set in set-node). You need this file
for VCS to start. It should look like this:
0
daldev05 1
daldev06
GAB
GAB requires only one configuration file, /etc/gabtab. This file lists the number
of nodes in the cluster and also, if there are any communication disks in the
system, configuration for them. Ex:
/sbin/gabconfig -c -n2
tells GAB to start GAB with 2 hosts in the cluster. To specify VCS
communication disks:
/sbin/gabdisk -a /dev/dsk/cXtXdXs2 -s 16 -p a /sbin/gabdisk -a
/dev/dsk/cXtXdXs2 -s 144 -p h /sbin/gabdisk -a /dev/dsk/cYtYdYs2 -s 16
-p a /sbin/gabdisk -a /dev/dsk/cYtYdYs2 -s 144 -p h
-a specifies the disk, -s specifies the start block for each communication region,
and -p specifies the port to use, "a" being the GAB seed port and "h" the VCS
port. The ports are the same as the network ports used by LLT and GAB, but
are simulated on a disk.
VCS
The VCS configuration file(s) are in /etc/VRTSvcs/conf/config. The two most
important files are main.cf and types.cf. I like to set $VCSCONF to that
directory to make my life easier. main.cf contains the actual VCS configuration
for Clusters, Groups, and Resources, while types.cf contains C-like prototypes
for each possible Resource.
The VCS configurationis very similar to the C language, but all you are doing
is defining variables. Comments are "//" (if you try to use #'s, you'll be unhappy
with the result), and you can use "include" statements if you want to break up
your configuration to make it more readable. One file you must include is
types.cf.
In main.cf, you need to specify a Cluster definition:
cluster iMS ( )
You can specify variables within this cluster definition, but for the most part,
the defaults are acceptible. Cluster variables include maximum number of
groups per cluster, link monitoring, log size, maximum number of resources,
maximum number of types, and a list of user names for the GUI that you will
never use and shouldn't install.
You then need to specify the systems in the cluster:
system daldev05 ( ) system daldev06 ( )
snmp vcs (
Enabled = 1
IPAddr = 0.0.0.0
TrapList = { 1 = "A new
system has joined the VCS Cluster",
2 = "An existing system has
changed its state",
3 = "A service group has changed its state",
4 = "One or more heartbeat links has gone down",
5 = "An HA service
has done a manual restart",
6 = "An HA service has been manually
idled",
7 = "An HA service has been successfully started" }
)
IPAddr is the IP address of the trap listener. Enabled defaults to 0, so you need
to include this if you want VCS to send traps. You can also specify a list of
numerical traps; listed above are the VCS default traps.
Each cluster can have multiple Service Group definitions. The most basic
Service Group looks like this:
group iMS5a (
SystemList = { daldev05, daldev06 }
AutoStartList = { daldev05 }
)
You can also set the following variables (not a complete list):
Inside each Service Group you need to define Resources. These are the nuts
and bolts of VCS. A full description of the bundled Resources can be found in
the Install Guide and a full description of the configuration language can be
found in the User's Guide.
Here are a couple of Resource examples:
NIC networka (
Device = hme0
NetworkType = ether
)
logical_IPa (
Device = hme0
Address = "10.10.30.156"
IP
The first line begins with a Resource type (e.g. NIC or IP) and then a globally
unique name for that particular resource. Inside the paren block, you can set the
variables for each resource.
Once you have set up resources, you need to build a resource dependancy tree
for the group. The syntax is "child_resource requires parent_resource." A
dependancy tree for the above resources would look like this:
logical_IPa requires networka
The dependancy tree tells VCS which resources need to be started before other
resources can be activated. In this case, VCS knows that the NIC hme0 has to
be working before resource logical_IPa can be started. This works well with
things like volumes and volumegroups; without a dependancy tree, VCS could
try to mount a volume before importing the volume group. VCS deactivates all
VCS controlled resources when it shuts down, so all virtual interfaces (resource
type IP) are unplumbed and volumes are unmounted/exported at VCS
shutdown.
Once the configuration is buld, you can verify it by running
/opt/VRTSvcs/bin/hacf -verify and then you can start VCS by running
/opt/VRTSvcs/bin/hastart.
The last digits in each line are the node IDs of the cluster members. Any
mention of "jeopardy" ports means there's a problem with that node in
the cluster.
haclus displays information about the VCS cluster. It's not particularly
useful because there are other, more detailed tools you can use:
hasys controls information about VCS systems. hasys -display shows
each host in the cluster and it's current status. You can also set this to
add, delete, or modify existing systems in the cluster.
hagrp controls Service Groups. It can offline, online (or swing) groups
from host to host. This is one of the most useful VCS tools.
hares controls Resources. This is the finest granular tool for VCS, as it
can add, remove, or modify individual resources and resource attributes.
In a failover Group, you can only online the resource on system on which the
group is online, so if ResourceA is a member of GroupA, you can only bring
ResourceA online on the system that is running GroupA. To online a resource:
hares -online ResourceA -sys sysa
If you get a fault on any resource or group, you need to clear the Fault on a
system before you can bring that resource/group up on it. To clear faults:
hagrp -clear GroupA hares -clear ResourceA
Caveats
Here are some tricks for VCS:
VCS likes to have complete control of all its resources. It brings up all its own
virtual interfaces, so don't bother to do that in your init scripts. VCS also likes
to have complete control of all the Veritas volumes and groups, so you
shouldn't mount them at boot. VCS will fail to mount a volume unless it is
responsible for importing the Volume Group; if you import the VG and then
start VCS, it will fail after about 5 minutes and drop the volume without
cleaning the FS. So make sure all VCS-controlled VG's are exported before
starting VCS.
Resource and Group names have no scope in VCS, so each must be a unique
identifier or VCS will fail to load your new configuration. There is no
equivalent to perl's my or local. VCS is also very case sensitive, so all Types,
Groups, Resources, and Systems must be the same every time. To make matters
worse, most of the VCS bundled types use random capitalization to try to fool
you. Copy and paste is your friend.
Make sure to create your Resource Dependancy Tree before your start VCS or
you could fuck up your whole cluster.
The default time-out for LLT/GAB communication is 15 seconds. If VCS
detects a system is down on all communcations channels for 15 seconds, it fails
all of that system's resource groups over to a new system.
If you use Veritas VM, VCS can't manage volumes in rootdg, so what I do is
encapsulate the root disk into rootdg and create new volume in their own VCS
managed VG. Don't put VCS and non-VCS volumes in the same VG.
Don't let VCS manage non-virtual interfaces. I did this in testing, and if you fail
a real interface, VCS will unplumb it, fail it over to a virtual on the fail-over
system. Then when you try to swing it back, it will fail.
Notes on how the configuration is loaded
Because VCS doesn't have any determination of primary/slave for the cluster,
VCS needs to determine who has the valid configuration for the cluster. As far
as I can tell (because of course it's not documented), this is how it works: When
VCS starts, GAB waits a predetermined timeout for the number of systems in
/etc/gabtab to join the cluster. At this point, all the systems in the cluster
compare local configurations, and the system with the newest config tries to
load it. If it's invalid, it pulls down the second newest valid config. If it is valid,
all the systems in VCS load that config.