AIM : Make an nfs type service group which will provide redundancy to a nfs share.
Description : We need to have an nfs share point on one of the nodes of a cluster. Whenever this server goes down, this share point will failover to another node and nsf share will be available to the client continuously. That's what the aim of VCS , right ? I mean , if any thing goes wrong at one node, it should not affect client.
So here we go :
The 2 most important things to reach our objective :
1. NFS configuration at operating system level at all the nodes.
2. Hierarchy of resources i.e. dependency of resources.
1. NFS needs a particular set of services NOT to run at OS level to work perfectly under VCS. We need to disable them NOT through SMF ( svcadm ) but at configuration files so that it may not get enabled when system reboots. So execute following commands to make changes at all the nodes :-
Now we are all set to play with VCS. I mean it's time to decide what all resources are required for this service group. In technical language - "Hierarchy of resources"
5. IP
We will take a bottom-up approach , it's not a rocket science, trust me. See, my approach is , if anything feels too tough or lengthy , just break it into peaces , and then assemble it at the end.
First thing that we can do is to make a service group with a familiar name , just in case we need to recall "nfssg".
#hagrp -add nfssg
#hagrp -modify nfssg SystemList sys1 0 sys2 1
#hagrp -modify nfssg AutoStartList sys1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we have to make a mount resource which will mount one of the volume from this disk group to a mount point /test :
Now, when the mount point is available, our task is to make a share resource which will nfs share this mount point , /test.
You know what , we are done with resources and service group. Nothing mechanical stuff needed now. Only bit of logic and now you will use only 1000th part of your brain to how exactly we should link these resources.
Now resources : (BOTTOM - UP)
Start with the easiest one, i.e. NIC and IP resources :
Now we need a mount point to be shared. This mount point will come from a disk group , as we are using VxVM. So make a DiskGroup resource and name it nfsdg. (Don't confuse with name nfssg)
#hares -add nfsdg DiskGroup nfssg
#hares -modify nfsdg Critical 0
#hares -modify nfsdg DiskGroup dg1
#hares -modify nfsdg Enabled 1
#hares -add nfsmount Mount nfssg
#hares -modify nfsmount Critical 0
#hares -modify nfsmount BlockDevice /dev/vx/dg1/dsk/vol1
#hares -modify nfsmount MountPoint /test
#hares -modify nfsmount MountOpt
#hares -modify nfsmount Enabled 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#hares -add nfsshare Share nfssg
#hares -modify nfsshare Critical 0
#hares -modify nfsshare PathName /test
#hares -modify nfsshare Options %-y
#hares -modify nfsshare Enabled 1
The most important resource is NFRestart resource, it will restart nfs services whenever it is called by VCS. Usually whenever service group is brought online or offline, this resource is triggered. As it is most important, we will give highest priority to this resource and it will be our top resource in dependency hierarchy. First add it :
#hares -add nfsrestart NFSRestart nfssg
#hares -modify nfsrestart Critical 0
#hares -modify nfsrestart Enabled 1
As we know NFSRestart is most important , so make it grandfather, I mean keep it at the top of dependency tree : NFSRestart ->Share->Mount->DiskGroup and another one is IP->NIC , thats it. DONE. We will make 2 dependency tree not 1 because making 1 dependency tree will violate the rule of max 5 dependency in a tree.
#hares -link nfsrestart nfsshare
#hares -link nfsshare nfsmount
#hares -link nfsmount nfsdg
#hares -link ipmnicb mnicb
~~~~~~~~~~~~~DONE ...!!!~~~~~~~~~~~
#haconf -dump -makero
BRING THE SERVICE GROUP ONLINE :-
#hagrp -online nfssg -sys sys1
Clarity of facts :-
1. Here , we are working on NFS server and not on client. We are providing high availability to "nfs share".
2. On client, simply mount it by "mount" command. If you want to provide HA on this mount point as well, simple "Mount" type resource will work , with block device modified as "192.168.1.100:/test" .
Description : We need to have an nfs share point on one of the nodes of a cluster. Whenever this server goes down, this share point will failover to another node and nsf share will be available to the client continuously. That's what the aim of VCS , right ? I mean , if any thing goes wrong at one node, it should not affect client.
So here we go :
The 2 most important things to reach our objective :
1. NFS configuration at operating system level at all the nodes.
2. Hierarchy of resources i.e. dependency of resources.
1. NFS needs a particular set of services NOT to run at OS level to work perfectly under VCS. We need to disable them NOT through SMF ( svcadm ) but at configuration files so that it may not get enabled when system reboots. So execute following commands to make changes at all the nodes :-
svccfg -s nfs/server setprop "application/auto_enable=false"
svccfg -s nfs/mapid setprop "application/auto_enable=false"
svccfg -s nfs/nlockmgr setprop "application/auto_enable=false"
svccfg -s nfs/status setprop "application/auto_enable=false"
Now we are all set to play with VCS. I mean it's time to decide what all resources are required for this service group. In technical language - "Hierarchy of resources"
Keep NFSRestart at top, all other should be child of this resource.
it needs following 6 resources :-
1. NFSRestart
2. Share
3. DiskGroup
4. Mount
5. IP
6. NIC
Don't get scared how to handle these many resources, we already have divided in 3 sections.
We will take a bottom-up approach , it's not a rocket science, trust me. See, my approach is , if anything feels too tough or lengthy , just break it into peaces , and then assemble it at the end.
First thing that we can do is to make a service group with a familiar name , just in case we need to recall "nfssg".
#hagrp -add nfssg
#hagrp -modify nfssg SystemList sys1 0 sys2 1
#hagrp -modify nfssg AutoStartList sys1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we have to make a mount resource which will mount one of the volume from this disk group to a mount point /test :
Now, when the mount point is available, our task is to make a share resource which will nfs share this mount point , /test.
You know what , we are done with resources and service group. Nothing mechanical stuff needed now. Only bit of logic and now you will use only 1000th part of your brain to how exactly we should link these resources.
Now resources : (BOTTOM - UP)
Start with the easiest one, i.e. NIC and IP resources :
Why this IP ? Because it is used to access NFS share from client. ( remember what we do at client, mount ip:/nfsshare /mnt , so this ip address will be used by client to access our share point)
#hares -add mnicb MultiNICB nfssg
#hares -modify mnicb Critical 0
#hares -modify mnicb Device e1000g0
#hares -modify mnicb Enabled 1
#hares add ipmnicb IPMultiNICB nfssg
#hares -modify ipmnicb Critical 0
#hares -modify ipmnicb Address 192.168.1.100
#hares -modify ipmnicb BaseResName mnicb
#hares -modify ipmnicb NetMask 255.255.255.0
#hares -modify ipmnicb Enabled 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we need a mount point to be shared. This mount point will come from a disk group , as we are using VxVM. So make a DiskGroup resource and name it nfsdg. (Don't confuse with name nfssg)
#hares -modify nfsdg Critical 0
#hares -modify nfsdg DiskGroup dg1
#hares -modify nfsdg Enabled 1
#hares -add nfsmount Mount nfssg
#hares -modify nfsmount Critical 0
#hares -modify nfsmount BlockDevice /dev/vx/dg1/dsk/vol1
#hares -modify nfsmount MountPoint /test
#hares -modify nfsmount MountOpt
#hares -modify nfsmount Enabled 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#hares -add nfsshare Share nfssg
#hares -modify nfsshare Critical 0
#hares -modify nfsshare PathName /test
#hares -modify nfsshare Options %-y
#hares -modify nfsshare Enabled 1
#hares -add nfsrestart NFSRestart nfssg
#hares -modify nfsrestart Critical 0
#hares -modify nfsrestart Enabled 1
As we know NFSRestart is most important , so make it grandfather, I mean keep it at the top of dependency tree : NFSRestart ->Share->Mount->DiskGroup and another one is IP->NIC , thats it. DONE. We will make 2 dependency tree not 1 because making 1 dependency tree will violate the rule of max 5 dependency in a tree.
#hares -link nfsrestart nfsshare
#hares -link nfsshare nfsmount
#hares -link nfsmount nfsdg
#hares -link ipmnicb mnicb
~~~~~~~~~~~~~DONE ...!!!~~~~~~~~~~~
#haconf -dump -makero
BRING THE SERVICE GROUP ONLINE :-
#hagrp -online nfssg -sys sys1
Clarity of facts :-
1. Here , we are working on NFS server and not on client. We are providing high availability to "nfs share".
2. On client, simply mount it by "mount" command. If you want to provide HA on this mount point as well, simple "Mount" type resource will work , with block device modified as "192.168.1.100:/test" .
Abhishek,
ReplyDeletei am having issue with failover NFS in VCS.
I did all the config as per you, i used NIC insted of MultiNICB.
bash-3.00# hares -add nica NIC nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nica Critical 0
bash-3.00# hares -modify nica Device e1000g0
bash-3.00# hares -modify nica NetworkType ether
bash-3.00# hares -modify nica Enabled 1
bash-3.00# hares -add nfsip IP nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nfsip Device e1000g0
bash-3.00# hares -modify nfsip Address "192.168.10.100"
bash-3.00# hares -modify nfsip Enabled 1
Nut when i try to online service group it give me below error.
bash-3.00# hagrp -online nfssg -sys node1
VCS WARNING V-16-1-10162 Group nfssg has not been fully probed on system node1
Can you please let me know where did i mistake.
Thanks in advance,
tushar.vadhawana@gmail.com
My whole config as below. i think did mistake to create a link (dependcy)
ReplyDeletebash-3.00# haconf -makerw
bash-3.00# hagrp -add nfssg
VCS NOTICE V-16-1-10136 Group added; populating SystemList and setting the Parallel attribute recommended before adding resources
bash-3.00# hagrp -modify nfssg SystemList node1 0 node2 1
bash-3.00# hagrp -modify nfssg AutoStartList node1
Step-5 Create Mount Resource.
bash-3.00# hares -add nfsrs DiskGroup nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nfsrs DiskGroup nfsdg
bash-3.00# hares -modify nfsrs Critical 0
bash-3.00# hares -modify nfsrs Enabled 1
bash-3.00# hares -add nfsmount Mount nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nfsmount BlockDevice /dev/vx/dsk/nfsdg/nfsvol
bash-3.00# hares -modify nfsmount MountPoint /nfs_export/
bash-3.00# hares -modify Critical 0
bash-3.00# hares -modify nfsmount Critical 0
bash-3.00# hares -modify nfsmount Enabled 1
bash-3.00# hares -modify nfsmount FSType vxvm
bash-3.00# hares -modify nfsmount MountOpt rw
bash-3.00# hares -modify nfsmount FsckOpt %-y
Step-6 Add NFS Share resouces.
bash-3.00# hares -add nfsshare Share nfssg
bash-3.00# hares -modify nfsshare PathName /nfs_export/
bash-3.00# hares modify nfsshare Critical 0
bash-3.00# hares -modify nfsshare Enabled 1
bash-3.00# hares -modify nfsshare Options %-y
Step-7 Create NIC Resource
bash-3.00# hares -add nica NIC nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nica Critical 0
bash-3.00# hares -modify nica Device e1000g0
bash-3.00# hares -modify nica NetworkType ether
bash-3.00# hares -modify nica Enabled 1
Step-8 Create IP Resource
bash-3.00# hares -add nfsip IP nfssg
VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
bash-3.00# hares -modify nfsip Device e1000g0
bash-3.00# hares -modify nfsip Address "192.168.10.100"
bash-3.00# hares -modify nfsip Enabled 1
Step-9Create link.
bash-3.00# hares -link nfsip nfsshare
bash-3.00# hares -link nfsip nica
bash-3.00# hares -link nfsmount nfsrs
bash-3.00# hares -link nfsshare nfs2
bash-3.00# hares -link nfsshare nfsmount
Thanks
tushar.vadhawana@gmail.com
Can you please tell me the command to configure VCS SG/resource to mount NFS on the client.
ReplyDelete