Share This:

Helpful information for running a CentOS Cluster continued

 

As noted at the end of last post, todays post is more tasty documentation, straight off our internal R&D Support Wiki and as written by the Czar of NAS, Dan Goetzman, for running a CentOS cluster. Admittedly, this is the kind of post that is more useful from Google than as exciting reading. When you need to know this stuff, you need to know it though, and so i am posting it to help out whomever might come down the same path we have.

 

Next post I'll be talking about a new BMC Open Source initiative that I am intimately involved with: that one will be one of the most fun posts I have ever had the pleasure to write. In fact, I am ready to write it now, so lets get into the meat of this HOW-TO.

 

Take it away Dan:

 


HOWTO Shutdown or reboot a single node

The cluster software is started and stopped using the standard system startup scripts. So all that is required is to use the normal Linux system reboot or shutdown commands.

  • shutdown -h now - To shutdown a single node
Note: 2 out of 3 nodes must remain running to keep the cluster "quorate", or running.
  • reboot - To reboot a single node

HOWTO Remove a node from the cluster

  • Stop applications using cluster resources
  1. uvscan - Currently running only on rnd-fs03
  2. nfs - Actually runs on all cluster nodes but not controlled by rgmanager
  3. Samba - Controlled by rgmanager
  4. bbd - Controlled by rgmanager
  5. vsftpd - Actually runs on all cluster nodes but not controlled by rgmanager

Notes: Services controlled by rgmanager will be stopped when rgmanager is stopped. Non cluster applications, like DNS and NIS slaves, do not need to be stopped.

  • Stop cluster services in this order
  1. service rgmanager stop
  2. service gfs stop
  3. service clvmd stop
  4. service cman stop

 

  • Optional, disable services on reboot
  1. chkconfig uvscan off
  2. chkconfig nfs off
  3. chkconfig vsftpd off
  4. chkconfig rgmanager off
  5. chkconfig gfs off
  6. chkconfig clvmd off
  7. chkconfig cman off

Note: To add the node back into the cluster, run this procedure in reverse order.

 

HOWTO Troubleshoot NFS serving

Local tests on the server

  • rpcinfo -p - Verify portmapper is responding
  • showmount -e - Verify mountd is responding
  • rpcinfo -u rnd-clunfs nfs - Verify NFS daemon is reponding to UDP requests
  • rpcinfo -t rnd-clunfs nfs - Verify NFS daemon is responding to TCP request

Test from a NFS client

  • rpcinfo -p rnd-clunfs - Verify portmapper is responding
  • showmount -e rnd-clunfs - Verify mountd is responding
  • rpcinfo -u rnd-clunfs nfs - Verify NFS daemon is reponding to UDP requests
  • rpcinfo -t rnd-clunfs nfs - Verify NFS daemon is responding to TCP request

HOWTO Troubleshoot Samba CIFS serving

The following procedure is what I do to verify that Samba is available. It starts from the server, and works it's way back to testing on the client.

Local tests on the server

  • smbstatus - Verify normal Samba status.
  • cd /var/log/samba - Check for errors in the Samba event logs.

Remote tests from a client

  • ping rnd-fs - Verify TCP/IP connectivity to Samba
  • nbtstat -s rnd-fs - Verify if a simple NetBIOS operation will respond
  • net view \\rnd-fs - Verify if the shares can be browsed
  • net use * \\rnd-fs\${SHARE} * /user:adprod.bmc.com\username - Verify that a share can be mapped
Note: Samba is a cluster service and only runs on a single node at a time.

HOWTO start/stop/admin Samba

Samba is a layered software application that emulates a CIFS server, and on the cluster it is configured as a cluster service.

Note: You must use the cluster commands to start and stop the Samba service,
      NOT the normal scripts in /etc/init.d.

Query the cluster services

  • clustat - To verify if the Samba service is running and on what node.

Start and Stop Samba

  • clusvcadm -d Samba - Stop Samba
  • clusvcadm -e Samba - Start Samba

Query commands for Samba

  • smbstatus - Show status

Adding shares and permissions

   
  • vi /etc/samba/smb.com - Add/Modify/Remove a share

That is it for today. Next time, as noted above, Open Source at BMC.