Pseudofilesystem improvements

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
m
 
(10 intermediate revisions not shown)
Line 1: Line 1:
-
[http://people.msoe.edu/~millerni/forums.php?show=topic&id=96&forum=13 diazepam online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=148&forum=13 order tramadol] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=156&forum=13 cheap wellbutrin] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=107&forum=13 free jazz ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a00 cheap rivotril] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=119&forum=13 mtv ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=110&forum=13 cheap lipitor] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=89&forum=13 celexa online] [http://wc1.worldcrossing.com/WebX/.1de609e0 ativan online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=123&forum=13 free nokia ringtones] [http://wc1.worldcrossing.com/WebX/.1de609f9 cheap norco] [http://wc1.worldcrossing.com/WebX/.1de60a18 motorola ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a2c cheap ultram] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=150&forum=13 ultram online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=124&forum=13 norco] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=116&forum=13 free mono ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a0e celexa online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=97&forum=13 buy didrex] [http://wc1.worldcrossing.com/WebX/.1de609fa online pharmacy] [http://wc1.worldcrossing.com/WebX/.1de60a24 cheap propecia] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=128&forum=13 cheap phentermine] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=153&forum=13 buy viagra] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=154&forum=13 vicodin online] [http://wc1.worldcrossing.com/WebX/.1de60a27 but sildenafil] [http://wc1.worldcrossing.com/WebX/.1de60a1f lisinopril online] [http://wc1.worldcrossing.com/WebX/.1de60a26 free sharp ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a16 levitra] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=103&forum=13 funny ringtones] [http://wc1.worldcrossing.com/WebX/.1de609ee hydrocodone] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=151&forum=13 valium online] [http://wc1.worldcrossing.com/WebX/.1de609e5 free cool ringtones] [http://wc1.worldcrossing.com/WebX/.1de609e4 cheap clonazepam] [http://wc1.worldcrossing.com/WebX/.1de609e6 cyclobenzaprine online] [http://wc1.worldcrossing.com/WebX/.1de60a35 verizon ringtones] [http://wc1.worldcrossing.com/WebX/.1de609ec buy hgh] [http://wc1.worldcrossing.com/WebX/.1de609df ambien online] [http://wc1.worldcrossing.com/WebX/.1de60a03 sony ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a0f cheap cialis] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=91&forum=13 cingular ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=90&forum=13 cheap cialis] [http://wc1.worldcrossing.com/WebX/.1de60a33 cheap prozac] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=133&forum=13 free qwest ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a10 cheap didrex] [http://wc1.worldcrossing.com/WebX/.1de609fd free punk ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a14 free jazz ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=102&forum=13 free free ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a2b but ultracet] [http://wc1.worldcrossing.com/WebX/.1de60a1e flexeril online] [http://wc1.worldcrossing.com/WebX/.1de609f8 nokia ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=87&forum=13 ativan online] [http://wc1.worldcrossing.com/WebX/.1de60a09 cheap viagra] [http://wc1.worldcrossing.com/WebX/.1de609f3 midi ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=142&forum=13 sony ericsson ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=135&forum=13 free real ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=82&forum=13 cheap adipex] [http://wc1.worldcrossing.com/WebX/.1de609eb funny ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=132&forum=13 punk ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=146&forum=13 tenuate online] [http://wc1.worldcrossing.com/WebX/.1de609f2 online meridia] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=108&forum=13 kyocera ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a05 sprint ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=114&forum=13 meridia] [http://wc1.worldcrossing.com/WebX/.1de60a06 tenuate online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=92&forum=13 clomid online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=86&forum=13 ambien online] [http://wc1.worldcrossing.com/WebX/.1de60a04 sonyericsson ringtones] [http://wc1.worldcrossing.com/WebX/.1de609e3 clomid online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=120&forum=13 free music ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=99&forum=13 ericsson ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=109&forum=13 cheap levitra] [http://wc1.worldcrossing.com/WebX/.1de60a0d wwe ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a15 free kyocera ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a20 nexium online] [http://wc1.worldcrossing.com/WebX/.1de60a19 free mp3 ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=104&forum=13 cheap hgh] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=158&forum=13 cheap xanax] [http://wc1.worldcrossing.com/WebX/.1de60a01 free sagem ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a30 cheap xenical] [http://wc1.worldcrossing.com/WebX/.1de609de cheap alprazolam] [http://wc1.worldcrossing.com/WebX/.1de60a1b music ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a31 buy zanaflex] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=152&forum=13 verizon ringtones] [http://wc1.worldcrossing.com/WebX/.1de609f7 nextel ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=147&forum=13 free tracfone ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a11 diethylpropion online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=83&forum=13 cheap albuterol] [http://wc1.worldcrossing.com/WebX/.1de60a29 buy tramadol] [http://wc1.worldcrossing.com/WebX/.1de60a25 free samsung ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=118&forum=13 free mp3 ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=157&forum=13 wwe ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a28 cheap soma] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=139&forum=13 free sharp ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=111&forum=13 order lisinopril] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=126&forum=13 paxil online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=117&forum=13 free motorola ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=138&forum=13 free samsung ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=155&forum=13 vigrx] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=140&forum=13 sildenafil online] [http://wc1.worldcrossing.com/WebX/.1de60a21 buy paxil] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=160&forum=13 cheap zanaflex] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=136&forum=13 cheap rivotril] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=98&forum=13 cheap diethylpropion] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=145&forum=13 sprint ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=129&forum=13 free polyphonic ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=137&forum=13 sagem ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=121&forum=13 cheap nexium] [http://wc1.worldcrossing.com/WebX/.1de60a1d free cingular ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=95&forum=13 cheap cyclobenzaprine] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=127&forum=13 pharmacy online online] [http://wc1.worldcrossing.com/WebX/.1de609dd alltel ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=144&forum=13 free sonyericsson ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a34 sony ericsson ringtones] [http://wc1.worldcrossing.com/WebX/.1de609e7 cheap diazepam] [http://wc1.worldcrossing.com/WebX/.1de60a0a vicodin online] [http://wc1.worldcrossing.com/WebX/.1de60a17 buy lipitor] [http://wc1.worldcrossing.com/WebX/.1de60a0b vigrx online] [http://wc1.worldcrossing.com/WebX/.1de60a1a mtv ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=141&forum=13 cheap soma] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=112&forum=13 buy lorazepam] [http://wc1.worldcrossing.com/WebX/.1de609ff free real ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=162&forum=13 cheap zyban] [http://wc1.worldcrossing.com/WebX/.1de609dc albuterol online] [http://wc1.worldcrossing.com/WebX/.1de60a12 free ericsson ringtones] [http://wc1.worldcrossing.com/WebX/.1de609f0 lorazepam online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=161&forum=13 but zoloft] [http://wc1.worldcrossing.com/WebX/.1de609db cheap adipex] [http://wc1.worldcrossing.com/WebX/.1de60a0c buy wellbutrin] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=84&forum=13 free alltel ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=94&forum=13 cool ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=122&forum=13 nextel ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a32 zoloft] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=93&forum=13 clonazepam online] [http://wc1.worldcrossing.com/WebX/.1de60a2d cheap valium] [http://wc1.worldcrossing.com/WebX/.1de60a13 fioricet online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=115&forum=13 midi ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a2e cheap xanax] [http://wc1.worldcrossing.com/WebX/.1de60a23 free polyphonic ringtones] [http://wc1.worldcrossing.com/WebX/.1de609ed hoodia online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=113&forum=13 but lortab] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=85&forum=13 order alprazolam] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=106&forum=13 cheap hydrocodone] [http://wc1.worldcrossing.com/WebX/.1de609f1 cheap lortab] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=149&forum=13 cheap ultracet] [http://wc1.worldcrossing.com/WebX/.1de609ea free ringtones] [http://wc1.worldcrossing.com/WebX/.1de609f4 mono ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=88&forum=13 cheap carisoprodol] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=143&forum=13 free sony ringtones] [http://wc1.worldcrossing.com/WebX/.1de609fb ortho online] [http://wc1.worldcrossing.com/WebX/.1de60a07 tracfone ringtones] [http://wc1.worldcrossing.com/WebX/.1de60a22 but phentermine] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=125&forum=13 cheap ortho] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=159&forum=13 cheap xenical] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=131&forum=13 prozac online] [http://wc1.worldcrossing.com/WebX/.1de609e1 carisoprodol online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=130&forum=13 cheap propecia] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=101&forum=13 cheap flexeril] [http://wc1.worldcrossing.com/WebX/.1de609fe free qwest ringtones] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=100&forum=13 fioricet online] [http://people.msoe.edu/~millerni/forums.php?show=topic&id=105&forum=13 cheap hoodia] [http://wc1.worldcrossing.com/WebX/.1de60a36 zyban online] = The Problem =
+
= The Problem =
-
See also [http://bugzilla.linux-nfs.org/show_bug.cgi?id=75 This bugzilla bug report].
+
NOTE: all of this has since been mostly fixed, so this page is out of date.
 +
 
 +
See also [http://bugzilla.linux-nfs.org/show_bug.cgi?id=75 This bugzilla bug report], or [http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=237108 This redhat bugzilla report], or [http://marc.info?l=linux-nfs&m=117408234516807&w=2 possibly relevant mail thread].
While NFSv2 and NFSv3 use a separate mount protocol to discover a server's exported filesystems, NFSv4 uses the same standard filesystem protocol (lookup, readdir, etc.) that is used to traverse within filesystems.
While NFSv2 and NFSv3 use a separate mount protocol to discover a server's exported filesystems, NFSv4 uses the same standard filesystem protocol (lookup, readdir, etc.) that is used to traverse within filesystems.
Line 13: Line 15:
For example, to quote Trond:
For example, to quote Trond:
 +
<pre>
  the current system means that if your export file
  the current system means that if your export file
  looks like this:
  looks like this:
 +
   /export/home myclient(rw,sync,no_subtree_check,fsid=0)
   /export/home myclient(rw,sync,no_subtree_check,fsid=0)
 +
  then that means that an NFSv4 fstab entry on 'myclient' will look like
  then that means that an NFSv4 fstab entry on 'myclient' will look like
 +
   myserver:/              /mnt    nfs4    rw,hard,intr    0 0
   myserver:/              /mnt    nfs4    rw,hard,intr    0 0
 +
  whereas an NFSv3 entry would look like
  whereas an NFSv3 entry would look like
 +
   myserver:/export/home  /mnt    nfs    rw,hard,intr 0 0
   myserver:/export/home  /mnt    nfs    rw,hard,intr 0 0
 +
  This difference in path semantics means that there is no way we could
  This difference in path semantics means that there is no way we could
  have 'mount' try NFSv4 first, then automatically fall back to NFSv3 if
  have 'mount' try NFSv4 first, then automatically fall back to NFSv3 if
  the server doesn't support NFSv4.
  the server doesn't support NFSv4.
 +
  What we ought to do (what Solaris, Netapp,... all do) is for the NFSv4
  What we ought to do (what Solaris, Netapp,... all do) is for the NFSv4
  server to have a pseudo-fs that contains the entries '/', '/export', and
  server to have a pseudo-fs that contains the entries '/', '/export', and
  '/export/home' so that the NFSv4 client can mount the
  '/export/home' so that the NFSv4 client can mount the
  directory /export/home instead of '/'."
  directory /export/home instead of '/'."
 +
</pre>
You could try to solve this problem in this example by creating a separate pseudofilesystem at
You could try to solve this problem in this example by creating a separate pseudofilesystem at
Line 57: Line 68:
Note some care has to be taken when reexporting, modifying the export list, etc., not to modify paths in the pseudofilesystem if not necessary; we'd rather not give clients unnecessary STALE errors.  Also we should probably save the pseudofilesystem across reboots to prevent filehandles from changing after a reboot.
Note some care has to be taken when reexporting, modifying the export list, etc., not to modify paths in the pseudofilesystem if not necessary; we'd rather not give clients unnecessary STALE errors.  Also we should probably save the pseudofilesystem across reboots to prevent filehandles from changing after a reboot.
-
Also, we should probably hide all the automatically created mountpoints under /var/lib/nfs/v4root/ from other processes on the server; this means mountd should be run in its own namespace (see CLONE_NEWNS in "man 2 clone").  However, nfsd needs to be able to do lookups in the namespace used by mountd, to find the exported filesystems.  I'm not sure how to do that; perhaps we should modify the exports cache downcall to allow passing file descriptors?
+
Also, we should probably hide all the automatically created mountpoints under /var/lib/nfs/v4root/ from other processes on the server; this means mountd should be run in its own namespace (see CLONE_NEWNS in "man 2 clone").  I believe all the lookups done by nfsd are actually done in downcalls that are performed in the context of the downcaller (mountd), so mountd's namespace should be the one it ends up seeing.
-
For now we should probably shouldn't be performing all the above steps by default; we could give mountd an extra commandline option or something.
+
For now we probably shouldn't be performing all the above steps by default; we could give mountd an extra commandline option or something.
= Other solutions =
= Other solutions =

Latest revision as of 22:37, 23 August 2010

The Problem

NOTE: all of this has since been mostly fixed, so this page is out of date.

See also This bugzilla bug report, or This redhat bugzilla report, or possibly relevant mail thread.

While NFSv2 and NFSv3 use a separate mount protocol to discover a server's exported filesystems, NFSv4 uses the same standard filesystem protocol (lookup, readdir, etc.) that is used to traverse within filesystems.

This gives the impression that these filesystems are all mounted on top of a top-level "pseudofilesystem".

Rather than constructing the pseudofilesystem from the list of exports in the /etc/exports file, the nfsd server just uses a real filesystem as the pseudofilesystem, and allows the administrator to mount and export filesystems underneath it. So that the server knows which exported filesystem to use as the pseudofilesystem (the filesystem that NFSv4 clients will see as "/"), that filesystem is marked with the export option "fsid=0".

This system was relatively simple to implement, but has lead to severe problems for automount users, or for anyone attempting to migrate from NFSv2/v3 to v4, because v4 clients see different paths than mountd clients.

For example, to quote Trond:

 the current system means that if your export file
 looks like this:

   /export/home myclient(rw,sync,no_subtree_check,fsid=0)

 then that means that an NFSv4 fstab entry on 'myclient' will look like

   myserver:/              /mnt    nfs4    rw,hard,intr    0 0

 whereas an NFSv3 entry would look like

   myserver:/export/home   /mnt    nfs     rw,hard,intr 0 0

 This difference in path semantics means that there is no way we could
 have 'mount' try NFSv4 first, then automatically fall back to NFSv3 if
 the server doesn't support NFSv4.

 What we ought to do (what Solaris, Netapp,... all do) is for the NFSv4
 server to have a pseudo-fs that contains the entries '/', '/export', and
 '/export/home' so that the NFSv4 client can mount the
 directory /export/home instead of '/'."

You could try to solve this problem in this example by creating a separate pseudofilesystem at /var/lib/nfs/v4root, mount --bind'ing /export/home on /var/lib/nfs/v4root/export/home, and creating another export for /var/lib/nfs/v4root/export/home. Then it will be possible to mount myserver:/export/home using either v3 or v4. Unfortunately, anyone using showmount or an automounter will now see a list of exports that looks like

/export/home
/var/lib/nfs/v4root
/var/lib/nfs/v4root/export/home

Also, setting up the pseudofilesystem and creating these extra exports is tedious work for the administrator.

But one solution is to modify mountd so that it creates those new exports itself, and hides the resulting new exports from the MOUNT protocol:

Solving the problem in mountd

One possible solution can be implemented entirely in mountd, without changing the kernel or any interfaces:

First, if we find "fsid=0" in the /etc/exports file anywhere, then we fall back on the current behavior, to preserve backwards compatibility.

If the file lacks any "fsid=0", then we automatically construct a pseudofilesystem in mountd:

  • As above, create a new filesystem at /var/lib/nfs/v4root/ to use as a pseudofilesystem; you'll probably need to loopback-mount a file so the user doesn't have to set aside a separate partition for this.
  • For each export in the export file, create a corresponding path under the pseudofilesystem.
  • Create a new fsid=0,ro export for the pseudofilesystem.
  • For each export in the export file, create a corresponding export for the path under the pseudofilesystem, with the same client and the same options.
  • Mark all of these automatically created exports specially so that mountd knows to use them only for answer upcalls from the kernel, and not for responding to mountd requests.

The end result is an automatically-built filesystem and a set of "shadow" exports that are visible to NFSv4 but not to anyone (NFSv2/3 clients or automounters) using the MOUNT protocol, which have the effect of allowing everyone to see the same export paths.

Note some care has to be taken when reexporting, modifying the export list, etc., not to modify paths in the pseudofilesystem if not necessary; we'd rather not give clients unnecessary STALE errors. Also we should probably save the pseudofilesystem across reboots to prevent filehandles from changing after a reboot.

Also, we should probably hide all the automatically created mountpoints under /var/lib/nfs/v4root/ from other processes on the server; this means mountd should be run in its own namespace (see CLONE_NEWNS in "man 2 clone"). I believe all the lookups done by nfsd are actually done in downcalls that are performed in the context of the downcaller (mountd), so mountd's namespace should be the one it ends up seeing.

For now we probably shouldn't be performing all the above steps by default; we could give mountd an extra commandline option or something.

Other solutions

The purely-mountd solution does seem a little complicated. We could build the pseudofilesystem entirely in the kernel, but I think that would require new kernel code and kernel interfaces. Also it might not fit well with the current export table architecture where only mountd every knows the complete list of exports, and kernel just requests information about particular exports as needed.

Other ideas?

Personal tools