Nfsv4 configuration

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
m (→A Real Life Example)
 
(24 intermediate revisions not shown)
Line 1: Line 1:
from [http://www.citi.umich.edu/projects/nfsv4/linux/using-nfsv4.html CITI] documentation.
from [http://www.citi.umich.edu/projects/nfsv4/linux/using-nfsv4.html CITI] documentation.
-
= NFSv4 Configuration =
+
= NFSv4 configuration =
== Base configuration ==
== Base configuration ==
-
This section maybe skiped on most recent distribution. If so, go [[#Exporting_directories|here]]
+
This section may be skipped on most recent distribution. If so, go [[#Exporting_directories|here]]
You should have already built and installed the kernel and user utilities and setup krb5.
You should have already built and installed the kernel and user utilities and setup krb5.
-
The following steps need only be done once. It maybe skiped most recent distribution.
+
The following steps need only be done once. It maybe skipped most recent distribution.
-
#Add the following lines to /etc/fstab to mount the rpc_pipefs and nfsd filesystems automatically:
+
Add the following lines to /etc/fstab to mount the rpc_pipefs and nfsd filesystems automatically:
  rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs defaults 0 0
  rpc_pipefs /var/lib/nfs/rpc_pipefs rpc_pipefs defaults 0 0
  nfsd /proc/fs/nfsd nfsd defaults 0 0
  nfsd /proc/fs/nfsd nfsd defaults 0 0
Line 20: Line 20:
*mount nfsd
*mount nfsd
-
#Modify the fstab line for filesystem you plan to export on the server to add the "acl" option to the mount options.
+
Modify the fstab line for filesystem you plan to export on the server to add the "acl" option to the mount options.
-
#Create a directory to export on the server:
+
Create a directory to export on the server:
*mkdir /export
*mkdir /export
*chmod a+rwxt /export
*chmod a+rwxt /export
Line 28: Line 28:
The following steps need to be performed on every boot; this should really be done by init scripts. The necessary init scripts already exist in recent Fedora distributions. See also debian/nfs-common.init and debian/nfs-kernel-server.init in the patched nfs-utils tarball for example init scripts.
The following steps need to be performed on every boot; this should really be done by init scripts. The necessary init scripts already exist in recent Fedora distributions. See also debian/nfs-common.init and debian/nfs-kernel-server.init in the patched nfs-utils tarball for example init scripts.
-
#On your server, run rpc.mountd, rpc.svcgssd, rpc.idmapd, and rpc.nfsd:
+
On your server, run rpc.mountd, rpc.svcgssd, rpc.idmapd, and rpc.nfsd:
  rpc.mountd
  rpc.mountd
  rpc.idmapd
  rpc.idmapd
-
  rpc.svcgssd
+
  rpc.svcgssd   -> this is obsolete as of nfs-utils version 1.0.8-1
  rpc.nfsd 8
  rpc.nfsd 8
 +
 +
### Feb 2012: I've added some updates on this in the below
On the client, run rpc.gssd and rpc.idmapd:
On the client, run rpc.gssd and rpc.idmapd:
  rpc.gssd -m
  rpc.gssd -m
  rpc.idmapd
  rpc.idmapd
-
 
== Exporting directories ==
== Exporting directories ==
Line 44: Line 45:
Export your server directory (in this case, to anyone, using any authentication flavor), by adding the following lines to /etc/exports
Export your server directory (in this case, to anyone, using any authentication flavor), by adding the following lines to /etc/exports
-
/export  *(rw,fsid=0,insecure,no_subtree_check)
 
  /export  gss/krb5(rw,fsid=0,insecure,no_subtree_check)
  /export  gss/krb5(rw,fsid=0,insecure,no_subtree_check)
  /export  gss/krb5i(rw,fsid=0,insecure,no_subtree_check)
  /export  gss/krb5i(rw,fsid=0,insecure,no_subtree_check)
Line 63: Line 63:
and mount the server:
and mount the server:
-
  mount -t nfs4 yourserver:/ /mnt/nfs4/
+
  mount -t nfs4 yourserver:/export /mnt/nfs4/
or, to mount with kerberos,
or, to mount with kerberos,
-
  mount -t nfs4 -o sec=krb5 yourserver:/ /mnt/nfs4/
+
  mount -t nfs4 -o sec=krb5 yourserver:/export /mnt/nfs4/
Play around with the mounted filesystem:
Play around with the mounted filesystem:
Line 76: Line 76:
  getfacl FOO
  getfacl FOO
-
etc.  
+
etc.
-
Exporting and mounting can be automated using /etc/fstab and /etc/exports; see the man pages fstab(5) and exports(5). Note that exports behave quite differently under v4 and v3, so some additional explanation may be called for:
+
To mount a filesystem using krb5, provide the "-osec=krb5" option to mount. To export a filesystem using krb5, export it to the special client named "gss/krb5". Replace "krb5" by "krb5i" or "krb5p" for integrity or for privacy, respectively. The -osec= options should also work for NFSv3 exports and mounts.
-
NFSv4 exports on linux
+
-
NFSv4 no longer has a separate "mount" protocol. Instead of exporting a number of distinct exports, an NFSv4 client sees the NFSv4 server's exports as existing inside a single filesystem, called the nfsv4 "pseudofilesystem".
+
Exporting and mounting can be automated using /etc/fstab and /etc/exports; see the man pages fstab(5) and exports(5).
-
On the current linux implementation, the pseudofilesystem is a single real filesystem, identified at export with the fsid=0 option.
+
Technical note: NFSv4 no longer has a separate "mount" protocol. Instead of exporting a number of distinct exports, an NFSv4 client sees the NFSv4 server's exports as existing inside a single filesystem, called the nfsv4 "pseudofilesystem".
-
In the example above, we exported only a single filesystem, which the client mounted as "/". You can provide clients with multiple filesystems to mount, producing NFSv3-like-behavior, by creative use of mount --bind. For example, you could export /usr/local/bin to clients as /bin and /usr/local/etc as /etc as follows:
+
The linux implementation allows you to designate a real filesystem as the pseudofilesystem, identifying that export with the fsid=0 option; we no longer recommend this. Instead, on any recent linux distribution, just list exports in /etc/exports exactly as you would for NFSv2 or NFSv3.
-
mkdir /export
+
== RPC Processes ==
-
mkdir /export/bin
+
-
mkdir /export/etc
+
-
mount --bind /usr/local/bin /export/bin
+
-
mount --bind /usr/local/etc /export/etc
+
-
exportfs -ofsid=0,insecure,no_subtree_check *:/export
+
-
exportfs -orw,nohide,insecure,no_subtree_check *:/export/bin
+
-
exportfs -orw,nohide,insecure,no_subtree_check *:/export/etc
+
-
Note that the paths returned by the "showmount" program are meaningful only to clients using nfs versions 2 and 3; in the above example, "showmount" will list the paths /export, /export/bin/, and /export/etc, but nfsv4 clients should mount yourserver:/, yourserver:/bin, or yourserver:/etc.
+
1. openSUSE after starting the nfs server
-
Mounting and exporting krb5
+
-
To mount a filesystem using krb5, provide the "-osec=krb5" option to mount. To export a filesystem using krb5, export it to the special client named "gss/krb5". Replace "krb5" by "krb5i" or "krb5p" for integrity or for privacy, respectively. The -osec= options should also work for NFSv3 exports and mounts.
+
ps aux | grep rpc
 +
 
 +
root      1231  0.0  0.1  2356  764 ?        Ss  08:20  0:00 /sbin/rpcbind
 +
 
 +
root      3737  0.0  0.0      0    0 ?        S<  08:28  0:00 [rpciod]
 +
 
 +
root      5215  0.0  0.2  3684  1208 ?        Ss  09:26  0:00 /usr/sbin/rpc.svcgssd
 +
 
 +
root      5223  0.0  0.0  2416  344 ?        Ss  09:26  0:00 /usr/sbin/rpc.idmapd
 +
 
 +
root      5227  0.0  0.1  2976  748 ?        Ss  09:26  0:00 /usr/sbin/rpc.mountd --no-nfs-version 2 --no-nfs-version 3
 +
(note the lack of gssd)
 +
 
 +
2. On both openSUSE (after calling rpc.gssd) and Ubuntu with all the necessary processes for successfully exporting the directory with -o sec=krb5<flavour>
 +
 
 +
ps aux | grep rpc
 +
 
 +
root      1231  0.0  0.1  2356  764 ?        Ss  08:20  0:00 /sbin/rpcbind
 +
 
 +
root      3737  0.0  0.0      0    0 ?        S<  08:28  0:00 [rpciod]
 +
 
 +
root      5215  0.0  0.2  3684  1208 ?        Ss  09:26  0:00 /usr/sbin/rpc.svcgssd
 +
 
 +
root      5223  0.0  0.0  2416  344 ?        Ss  09:26  0:00 /usr/sbin/rpc.idmapd
 +
 
 +
root      5227  0.0  0.1  2976  748 ?        Ss  09:26  0:00 /usr/sbin/rpc.mountd --no-nfs-version 2 --no-nfs-version 3
 +
 
 +
root      5294  0.0  0.0  3816  428 ?        Ss  09:31  0:00 rpc.gssd
 +
 
 +
== NFS Installation details ==
 +
 
 +
=== openSUSE ===
 +
Packages:
 +
zypper in nfs-kernel-server nfs-client nfsidmap limal-nfs-server
 +
 
 +
Configuration (Or use the excellent Yast NFS-Server Module)
 +
/etc/sysconfig/nfs
 +
 
 +
USE_KERNEL_NFSD_NUMBER="4"
 +
 
 +
MOUNTD_PORT=""
 +
 
 +
NFS_SECURITY_GSS="yes"
 +
 
 +
NFS3_SERVER_SUPPORT="no"
 +
 
 +
NFS4_SUPPORT="yes"
 +
 
 +
SM_NOTIFY_OPTIONS=""
 +
 
 +
NFS_START_SERVICES="yes"
 +
 
 +
STATD_OPTIONS=""
 +
 
 +
NFSV4LEASETIME=""
 +
 
 +
RPC_PIPEFS_DIR=""
 +
 
 +
SVCGSSD_OPTIONS=""
 +
 
 +
NFSD_OPTIONS=""
 +
 
 +
=== Ubuntu ===
 +
Packages:
 +
 
 +
apt-get install nfs-common nfs-kernel-server
 +
(you may also need to do a: modprobe nfs before starting the nfs server)
 +
 
 +
Configuration
 +
 
 +
/etc/default/nfs-common
 +
 
 +
NEED_STATD=
 +
 
 +
STATDOPTS=
 +
 
 +
NEED_IDMAPD=yes
 +
 
 +
NEED_GSSD=yes
 +
 
 +
 
 +
/etc/default/nfs-kernel-server
 +
 
 +
RPCNFSDCOUNT=8
 +
 
 +
RPCNFSDPRIORITY=0
 +
 
 +
RPCMOUNTDOPTS=--manage-gids
 +
 
 +
NEED_SVCGSSD=yes
 +
 
 +
RPCSVCGSSDOPTS=
 +
 
 +
RPCNFSDOPTS=
 +
 
 +
 
 +
 
 +
=== Needed on both client and server on both openSUSE and Ubuntu ===
 +
 
 +
/etc/idmapd.conf
 +
 
 +
[General]
 +
 
 +
Verbosity = 0
 +
 
 +
Pipefs-Directory = /var/lib/nfs/rpc_pipefs
 +
 
 +
Domain = hh3.site //or whatever your domain is
 +
 
 +
[Mapping]
 +
 
 +
Nobody-User = nobody
 +
 
 +
Nobody-Group = nogroup
 +
 
 +
== Debugging ==
 +
 
 +
Run rpc.idmapd -fvvv and rpc.gssd -fvvv in separate terminals. They will tell you in no uncertain
 +
terms when you've got it wrong. Please see this screenshot:
 +
 
 +
http://3.bp.blogspot.com/-PLpbJDP1vCQ/TygMlNU6k8I/AAAAAAAAAGE/lVXsw6iGrS0/s1600/krb5-ubuntu3.png
 +
 
 +
== A Real Life Example ==
 +
 
 +
hh3.hh3.site is the nfs server and  HH6$@HH3.SITE is the machine principal for the client.
 +
On the client, we issue a:
 +
 
 +
mount -t nfs4 hh3:/foo /bar -o sec=krb5
 +
 
 +
Here is the log on the KDC:
 +
 
 +
Kerberos: ENC-TS Pre-authentication succeeded -- HH6$@HH3.SITE using arcfour-hmac-md5
 +
 
 +
Kerberos: AS-REQ authtime: 2012-02-06T19:44:47 starttime: unset endtime: 2012-02-07T05:44:47 renew till: 2012-02-07T19:44:47
 +
 
 +
Kerberos: Client supported enctypes: aes256-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96, des3-cbc-
 +
sha1, arcfour-hmac-md5, des-cbc-crc, des-cbc-md5, des-cbc-md4, using arcfour-hmac-md5/arcfour-hmac-md5
 +
 
 +
Kerberos: Requested flags: renewable-ok
 +
 
 +
Kerberos: TGS-REQ HH6$@HH3.SITE from ipv4:192.168.1.10:45421 for nfs/hh3.hh3.site@HH3.SITE [canonicalize, renewable]
 +
 
 +
Kerberos: TGS-REQ authtime: 2012-02-06T19:44:47 starttime: 2012-02-06T19:44:47 endtime: 2012-02-07T05:44:47 renew till: 20
 +
 
 +
The nfs server has identified itself, here are the keys in its keytab:
 +
 
 +
klist -ke /etc/krb5.keytab
 +
Keytab name: WRFILE:/etc/krb5.keytab
 +
 
 +
KVNO Principal
 +
  1 nfs/hh3.hh3.site@HH3.SITE (arcfour-hmac) 
 +
  1 HH3$@hh3.site (arcfour-hmac)
 +
 
 +
== Troubleshooting ==
 +
 
 +
http://wiki.linux-nfs.org/wiki/index.php?title=Troubleshooting
 +
 
 +
== References ==
 +
 
 +
1. man rpc.gssd(8)
 +
 
 +
http://linux.die.net/man/8/rpc.gssd
 +
 
 +
2. A hands on practical using modest hardware. Includes screenshots. openSUSE specific.
 +
 
 +
http://linuxcostablanca.blogspot.com/p/samba-4.html
 +
 
 +
3. The same as (3) but with Ubuntu specific differences
 +
 
 +
http://linuxcostablanca.blogspot.com/2012/01/samba-4-ubuntu.html
 +
 
 +
4. The latest unofficial version of this wiki page. Any new findings, I'll post there.
 +
 
 +
http://linuxcostablanca.blogspot.com/2012/02/nfsv4-myths-and-legends.html

Latest revision as of 16:28, 8 June 2017

from CITI documentation.

Contents

NFSv4 configuration

Base configuration

This section may be skipped on most recent distribution. If so, go here

You should have already built and installed the kernel and user utilities and setup krb5.

The following steps need only be done once. It maybe skipped most recent distribution.

Add the following lines to /etc/fstab to mount the rpc_pipefs and nfsd filesystems automatically: 

rpc_pipefs	/var/lib/nfs/rpc_pipefs	rpc_pipefs	defaults	0	0
nfsd	/proc/fs/nfsd	nfsd	defaults	0	0

and create the mountpoint for rpc_pipefs, and mount both filesystems:

  • mkdir /var/lib/nfs/rpc_pipefs
  • mount rpc_pipefs
  • mount nfsd

Modify the fstab line for filesystem you plan to export on the server to add the "acl" option to the mount options.

Create a directory to export on the server:

  • mkdir /export
  • chmod a+rwxt /export

The following steps need to be performed on every boot; this should really be done by init scripts. The necessary init scripts already exist in recent Fedora distributions. See also debian/nfs-common.init and debian/nfs-kernel-server.init in the patched nfs-utils tarball for example init scripts.

On your server, run rpc.mountd, rpc.svcgssd, rpc.idmapd, and rpc.nfsd: 

rpc.mountd
rpc.idmapd
rpc.svcgssd   -> this is obsolete as of nfs-utils version 1.0.8-1
rpc.nfsd 8
      1. Feb 2012: I've added some updates on this in the below

On the client, run rpc.gssd and rpc.idmapd: 

rpc.gssd -m
rpc.idmapd

Exporting directories

Export your server directory (in this case, to anyone, using any authentication flavor), by adding the following lines to /etc/exports 

/export  gss/krb5(rw,fsid=0,insecure,no_subtree_check)
/export  gss/krb5i(rw,fsid=0,insecure,no_subtree_check)
/export  gss/krb5p(rw,fsid=0,insecure,no_subtree_check)

and then running 

exportfs -r

(Note that if your server is connected to the internet the first line will give everyone in the world read and write access to /export and (with a little work) to anything on the same disk partition as /export! See the exports man page for more information.)

To access this export from a client:

Create a mountpoint on your client: 

mkdir /mnt/nfs4/

and mount the server: 

mount -t nfs4 yourserver:/export /mnt/nfs4/

or, to mount with kerberos, 

mount -t nfs4 -o sec=krb5 yourserver:/export /mnt/nfs4/

Play around with the mounted filesystem: 

cd /mnt/nfs4/
echo "Hello World!" >FOO
cat FOO
getfacl FOO

etc.

To mount a filesystem using krb5, provide the "-osec=krb5" option to mount. To export a filesystem using krb5, export it to the special client named "gss/krb5". Replace "krb5" by "krb5i" or "krb5p" for integrity or for privacy, respectively. The -osec= options should also work for NFSv3 exports and mounts.

Exporting and mounting can be automated using /etc/fstab and /etc/exports; see the man pages fstab(5) and exports(5).

Technical note: NFSv4 no longer has a separate "mount" protocol. Instead of exporting a number of distinct exports, an NFSv4 client sees the NFSv4 server's exports as existing inside a single filesystem, called the nfsv4 "pseudofilesystem".

The linux implementation allows you to designate a real filesystem as the pseudofilesystem, identifying that export with the fsid=0 option; we no longer recommend this. Instead, on any recent linux distribution, just list exports in /etc/exports exactly as you would for NFSv2 or NFSv3.

RPC Processes

1. openSUSE after starting the nfs server

ps aux | grep rpc

root 1231 0.0 0.1 2356 764 ? Ss 08:20 0:00 /sbin/rpcbind

root 3737 0.0 0.0 0 0 ? S< 08:28 0:00 [rpciod]

root 5215 0.0 0.2 3684 1208 ? Ss 09:26 0:00 /usr/sbin/rpc.svcgssd

root 5223 0.0 0.0 2416 344 ? Ss 09:26 0:00 /usr/sbin/rpc.idmapd

root 5227 0.0 0.1 2976 748 ? Ss 09:26 0:00 /usr/sbin/rpc.mountd --no-nfs-version 2 --no-nfs-version 3 (note the lack of gssd)

2. On both openSUSE (after calling rpc.gssd) and Ubuntu with all the necessary processes for successfully exporting the directory with -o sec=krb5<flavour>

ps aux | grep rpc

root 1231 0.0 0.1 2356 764 ? Ss 08:20 0:00 /sbin/rpcbind

root 3737 0.0 0.0 0 0 ? S< 08:28 0:00 [rpciod]

root 5215 0.0 0.2 3684 1208 ? Ss 09:26 0:00 /usr/sbin/rpc.svcgssd

root 5223 0.0 0.0 2416 344 ? Ss 09:26 0:00 /usr/sbin/rpc.idmapd

root 5227 0.0 0.1 2976 748 ? Ss 09:26 0:00 /usr/sbin/rpc.mountd --no-nfs-version 2 --no-nfs-version 3

root 5294 0.0 0.0 3816 428 ? Ss 09:31 0:00 rpc.gssd

NFS Installation details

openSUSE

Packages: zypper in nfs-kernel-server nfs-client nfsidmap limal-nfs-server

Configuration (Or use the excellent Yast NFS-Server Module) /etc/sysconfig/nfs

USE_KERNEL_NFSD_NUMBER="4"

MOUNTD_PORT=""

NFS_SECURITY_GSS="yes"

NFS3_SERVER_SUPPORT="no"

NFS4_SUPPORT="yes"

SM_NOTIFY_OPTIONS=""

NFS_START_SERVICES="yes"

STATD_OPTIONS=""

NFSV4LEASETIME=""

RPC_PIPEFS_DIR=""

SVCGSSD_OPTIONS=""

NFSD_OPTIONS=""

Ubuntu

Packages:

apt-get install nfs-common nfs-kernel-server (you may also need to do a: modprobe nfs before starting the nfs server)

Configuration

/etc/default/nfs-common

NEED_STATD=

STATDOPTS=

NEED_IDMAPD=yes

NEED_GSSD=yes


/etc/default/nfs-kernel-server

RPCNFSDCOUNT=8

RPCNFSDPRIORITY=0

RPCMOUNTDOPTS=--manage-gids

NEED_SVCGSSD=yes

RPCSVCGSSDOPTS=

RPCNFSDOPTS=


Needed on both client and server on both openSUSE and Ubuntu

/etc/idmapd.conf

[General]

Verbosity = 0

Pipefs-Directory = /var/lib/nfs/rpc_pipefs

Domain = hh3.site //or whatever your domain is

[Mapping]

Nobody-User = nobody

Nobody-Group = nogroup

Debugging

Run rpc.idmapd -fvvv and rpc.gssd -fvvv in separate terminals. They will tell you in no uncertain terms when you've got it wrong. Please see this screenshot:

http://3.bp.blogspot.com/-PLpbJDP1vCQ/TygMlNU6k8I/AAAAAAAAAGE/lVXsw6iGrS0/s1600/krb5-ubuntu3.png

A Real Life Example

hh3.hh3.site is the nfs server and HH6$@HH3.SITE is the machine principal for the client. On the client, we issue a:

mount -t nfs4 hh3:/foo /bar -o sec=krb5

Here is the log on the KDC:

Kerberos: ENC-TS Pre-authentication succeeded -- HH6$@HH3.SITE using arcfour-hmac-md5

Kerberos: AS-REQ authtime: 2012-02-06T19:44:47 starttime: unset endtime: 2012-02-07T05:44:47 renew till: 2012-02-07T19:44:47

Kerberos: Client supported enctypes: aes256-cts-hmac-sha1-96, aes128-cts-hmac-sha1-96, des3-cbc- sha1, arcfour-hmac-md5, des-cbc-crc, des-cbc-md5, des-cbc-md4, using arcfour-hmac-md5/arcfour-hmac-md5

Kerberos: Requested flags: renewable-ok

Kerberos: TGS-REQ HH6$@HH3.SITE from ipv4:192.168.1.10:45421 for nfs/hh3.hh3.site@HH3.SITE [canonicalize, renewable]

Kerberos: TGS-REQ authtime: 2012-02-06T19:44:47 starttime: 2012-02-06T19:44:47 endtime: 2012-02-07T05:44:47 renew till: 20

The nfs server has identified itself, here are the keys in its keytab:

klist -ke /etc/krb5.keytab Keytab name: WRFILE:/etc/krb5.keytab

KVNO Principal 

  1 nfs/hh3.hh3.site@HH3.SITE (arcfour-hmac)  
  1 HH3$@hh3.site (arcfour-hmac)

Troubleshooting

http://wiki.linux-nfs.org/wiki/index.php?title=Troubleshooting

References

1. man rpc.gssd(8) 

http://linux.die.net/man/8/rpc.gssd

2. A hands on practical using modest hardware. Includes screenshots. openSUSE specific. 

http://linuxcostablanca.blogspot.com/p/samba-4.html

3. The same as (3) but with Ubuntu specific differences 

http://linuxcostablanca.blogspot.com/2012/01/samba-4-ubuntu.html

4. The latest unofficial version of this wiki page. Any new findings, I'll post there. 

http://linuxcostablanca.blogspot.com/2012/02/nfsv4-myths-and-legends.html
Retrieved from "http://wiki.linux-nfs.org/wiki/index.php/Nfsv4_configuration"
Personal tools