FSID and Linux NFS servers

Version 1
    Share:|

    A light goes on during a hallway  discussion


    From the Fedora Core 4 man page for  "exports":

    fsid=num


                  This option forces the filesystem  identification portion of the


                  file handle and file attributes  used on  the  wire  to  be  num


                  instead  of a number derived  from the major and minor number of


                  the block device on which  the filesystem is  mounted.   Any  32


                  bit  number  can be used,  but it must be unique amongst all the


                  exported  filesystems.

     


     

    I have been working with Linux for a fair number of years, and the funny  thing about it is, every year I really learn more about how little I really  know. Every now and then, there is one of those bang your head on the table and  say 'duh' moments, like the one today.

     

    I was talking to one of my senior technical people about how the 64 bit file  server test was going, and all the various permutations of issues we could have  with how file systems are created and laid out on disk that could affect  upstream kernel behavior and speed. As an aside, ext3 under the 2.6 kernel does  not appear to perform much better than it did under the 2.4 kernel with the  documented here big kernel lock problem. Initial numbers with the 2.6.14 kernel on our new test hardware are  that we are seeing about 5 MB/Sec writes versus 40-50 MB/Sec writes for XFS.  Beating on it with our Apple Xserves and some fast Suns, and give or take an  elephant, XFS is about 25 times faster then Ext3. For file servers we tend to  just test writes, since it is assumed reads will be faster. But I  digress....

     

    He mentioned to me in passing that he has started using the FSID parameter in  /etc/fstab for the test file systems because he got tired of all the stale NFS  mounts. Being a manager, and very therefore very quick on my feet I said  "Huh?".

     

    Stale NFS mounts are the bane of existence for anyone who has ever run an NFS  file server. The file server goes down, you fix things, and when you come back  in some cases (Example: You had to move the file system(s) to a new device to  get things going) none of the NFS clients recognize that the file system is  back. This is because the NFS handle is based on the file handle it generated  out of the major and minor device nodes. We have even seen cases where even when  things did not move, the NFS server just re-did the file handles on what  appeared to be a whim.

     

    For older NFS clients more than newer ones, they might never time out: they  will have to be re-booted to recover. A royal pain and not a real productivity  enhancer. By using the FSID in /etc/fstab in the options sections (or on the  exportfs command of course), he makes sure *we* control the handle, and that it  is always the same no matter what the underlying devices major/minor number is.  Well duh.

     

    We theorize this was added as part of cluster support at some time, so that 2  different computers could mount a shared disk with the same file number. But  maybe it has been there since Linux 1.0 and we just never noticed it. Linux is a  big place.

     

    So far, in our limited testing it is having the effect of making it such that  when we tear down a test file system and rebuild it as something else (different  FS, different size, different RAID level... whatever), the clients just whine  about stale NFS handles until we bring it back and then they recover. Not tested  with every client yet: we are really just sanity checking our current server's  hardware and software configurations. But once in production, this is  potentially a very cool option to make system recovery much smoother for our  customers. It is going to speed testing to be sure.

    Humm: Yumex says 2.6.15-1.1831 is available for FC4 on my T41... better get  that downloading...