Wednesday, August 1, 2012

NFS service group in VCS

AIM : Make an nfs type service group which will provide redundancy to a nfs share.

Description : We need to have an nfs share point on one of the nodes of a cluster. Whenever this server goes down, this share point will  failover to another node and nsf share will be available to the client continuously. That's what the aim of VCS , right ? I mean , if any thing goes wrong at one node, it should not affect client.

So here we go :

The 2 most important things to reach our objective :

1. NFS configuration at operating system level at all the nodes.
2. Hierarchy of resources i.e. dependency of resources.

1. NFS needs a particular set of services NOT to run at OS level to work perfectly under VCS. We need to disable them NOT through SMF ( svcadm ) but at configuration files so that it may not get enabled when system reboots. So execute following commands to make changes at all the nodes :-


svccfg -s nfs/server setprop "application/auto_enable=false"
svccfg -s nfs/mapid setprop "application/auto_enable=false" 
svccfg -s nfs/nlockmgr setprop "application/auto_enable=false"
svccfg -s nfs/status setprop "application/auto_enable=false"

Now we are all set to play with VCS. I mean it's time to decide what all resources are required for this service group. In technical language - "Hierarchy of resources" 

Keep NFSRestart at top, all other should be child of this resource. 

it needs following 6 resources :-

1. NFSRestart
2. Share

3. DiskGroup
4. Mount

5. IP
6. NIC

Don't get scared how to handle these many resources, we already have divided in 3 sections.

We will take a bottom-up approach , it's not a rocket science, trust me. See, my approach is , if anything feels too tough or lengthy , just break it into peaces , and then assemble it at the end.

First thing that we can do is to make a service group with a familiar name , just in case we need to recall "nfssg".


#hagrp -add nfssg
#hagrp -modify nfssg SystemList sys1 0 sys2 1
#hagrp  -modify nfssg AutoStartList sys1


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we have to make a mount resource which will mount one of the volume from this disk group to a mount point /test :
Now, when the mount point is available, our task is to make a share resource which will nfs share this mount point , /test.
You know what , we are done with resources and service group. Nothing mechanical stuff needed now. Only bit of logic and now you will use only 1000th part of your brain to how exactly we should link these resources. 



Now resources : (BOTTOM - UP)


Start with the easiest one, i.e. NIC and IP resources :

Why this IP ? Because it is used to access NFS share from client. ( remember what we do at client, mount ip:/nfsshare  /mnt , so this ip address will be used by client to access our share point)


#hares -add mnicb MultiNICB nfssg
#hares -modify mnicb Critical 0
#hares -modify mnicb Device e1000g0
#hares -modify mnicb Enabled 1

#hares add ipmnicb IPMultiNICB nfssg
#hares -modify ipmnicb Critical 0
#hares -modify ipmnicb Address 192.168.1.100
#hares -modify ipmnicb BaseResName mnicb
#hares -modify ipmnicb NetMask 255.255.255.0
#hares -modify ipmnicb Enabled 1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Now we need a mount point to be shared. This mount point will come from a disk group , as we are using VxVM. So make a DiskGroup resource and name it nfsdg. (Don't confuse with name nfssg)

#hares -add nfsdg DiskGroup nfssg
#hares -modify  nfsdg  Critical 0
#hares -modify nfsdg DiskGroup dg1
#hares -modify  nfsdg  Enabled 1



#hares -add nfsmount Mount nfssg
#hares -modify   nfsmount  Critical 0
#hares -modify  nfsmount BlockDevice /dev/vx/dg1/dsk/vol1
#hares -modify nfsmount MountPoint /test
#hares -modify nfsmount MountOpt
#hares -modify nfsmount Enabled 1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


#hares -add nfsshare Share nfssg
#hares -modify nfsshare Critical 0
#hares -modify nfsshare PathName /test
#hares -modify nfsshare Options %-y
#hares -modify nfsshare Enabled 1



The most important resource is NFRestart resource, it will restart nfs services whenever it is called by VCS. Usually whenever service group is brought online or offline, this resource is triggered. As it is most important, we will give highest priority to this resource and it will be our top resource in dependency hierarchy. First add it : 

#hares -add nfsrestart NFSRestart nfssg
#hares -modify nfsrestart Critical 0
#hares -modify nfsrestart Enabled 1



As we know NFSRestart is most important , so make it grandfather, I mean keep it at the top of dependency tree :   NFSRestart  ->Share->Mount->DiskGroup   and another one is  IP->NIC  , thats it. DONE.  We will make 2 dependency tree not 1 because making 1 dependency tree will violate the rule of max 5 dependency in a tree.

#hares -link nfsrestart nfsshare
#hares -link nfsshare nfsmount
#hares -link nfsmount nfsdg

#hares -link ipmnicb mnicb

~~~~~~~~~~~~~DONE ...!!!~~~~~~~~~~~

#haconf -dump -makero

BRING THE SERVICE GROUP ONLINE :-

#hagrp -online nfssg -sys sys1

Clarity of facts :-
1. Here , we are working on NFS server and not on client. We are providing high availability to "nfs share".
2. On client, simply mount it by "mount" command. If you want to provide HA on this mount point as well, simple "Mount" type resource will work , with block device modified as "192.168.1.100:/test" .




3 comments:

  1. Abhishek,

    i am having issue with failover NFS in VCS.
    I did all the config as per you, i used NIC insted of MultiNICB.


    bash-3.00# hares -add nica NIC nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nica Critical 0
    bash-3.00# hares -modify nica Device e1000g0
    bash-3.00# hares -modify nica NetworkType ether
    bash-3.00# hares -modify nica Enabled 1


    bash-3.00# hares -add nfsip IP nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nfsip Device e1000g0
    bash-3.00# hares -modify nfsip Address "192.168.10.100"
    bash-3.00# hares -modify nfsip Enabled 1

    Nut when i try to online service group it give me below error.

    bash-3.00# hagrp -online nfssg -sys node1
    VCS WARNING V-16-1-10162 Group nfssg has not been fully probed on system node1

    Can you please let me know where did i mistake.

    Thanks in advance,
    tushar.vadhawana@gmail.com

    ReplyDelete
  2. My whole config as below. i think did mistake to create a link (dependcy)

    bash-3.00# haconf -makerw
    bash-3.00# hagrp -add nfssg
    VCS NOTICE V-16-1-10136 Group added; populating SystemList and setting the Parallel attribute recommended before adding resources
    bash-3.00# hagrp -modify nfssg SystemList node1 0 node2 1
    bash-3.00# hagrp -modify nfssg AutoStartList node1

    Step-5 Create Mount Resource.
    bash-3.00# hares -add nfsrs DiskGroup nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nfsrs DiskGroup nfsdg
    bash-3.00# hares -modify nfsrs Critical 0
    bash-3.00# hares -modify nfsrs Enabled 1

    bash-3.00# hares -add nfsmount Mount nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nfsmount BlockDevice /dev/vx/dsk/nfsdg/nfsvol
    bash-3.00# hares -modify nfsmount MountPoint /nfs_export/
    bash-3.00# hares -modify Critical 0
    bash-3.00# hares -modify nfsmount Critical 0
    bash-3.00# hares -modify nfsmount Enabled 1
    bash-3.00# hares -modify nfsmount FSType vxvm
    bash-3.00# hares -modify nfsmount MountOpt rw
    bash-3.00# hares -modify nfsmount FsckOpt %-y

    Step-6 Add NFS Share resouces.
    bash-3.00# hares -add nfsshare Share nfssg
    bash-3.00# hares -modify nfsshare PathName /nfs_export/
    bash-3.00# hares modify nfsshare Critical 0
    bash-3.00# hares -modify nfsshare Enabled 1
    bash-3.00# hares -modify nfsshare Options %-y

    Step-7 Create NIC Resource
    bash-3.00# hares -add nica NIC nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nica Critical 0
    bash-3.00# hares -modify nica Device e1000g0
    bash-3.00# hares -modify nica NetworkType ether
    bash-3.00# hares -modify nica Enabled 1

    Step-8 Create IP Resource
    bash-3.00# hares -add nfsip IP nfssg
    VCS NOTICE V-16-1-10242 Resource added. Enabled attribute must be set before agent monitors
    bash-3.00# hares -modify nfsip Device e1000g0
    bash-3.00# hares -modify nfsip Address "192.168.10.100"
    bash-3.00# hares -modify nfsip Enabled 1

    Step-9Create link.
    bash-3.00# hares -link nfsip nfsshare
    bash-3.00# hares -link nfsip nica
    bash-3.00# hares -link nfsmount nfsrs
    bash-3.00# hares -link nfsshare nfs2
    bash-3.00# hares -link nfsshare nfsmount

    Thanks
    tushar.vadhawana@gmail.com

    ReplyDelete
  3. Can you please tell me the command to configure VCS SG/resource to mount NFS on the client.

    ReplyDelete