PNFS Setup Instructions

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
(New page: <h1>File Layout </h1> <H2>Accessing a storage system with pNFS</H2> <b>Step 0: Get pNFS kernel from git tree and install on all involved servers. </b> <p> <b>Step 1: Setup NFSv4 servers</b...)
 
(9 intermediate revisions not shown)
Line 1: Line 1:
 +
THIS IS OUT OF DATE.  The particular pnfs code described here is no longer under development.
 +
<h1>File Layout </h1>
<h1>File Layout </h1>
<H2>Accessing a storage system with pNFS</H2>
<H2>Accessing a storage system with pNFS</H2>
-
<b>Step 0: Get pNFS kernel from git tree and install on all involved servers. </b>
+
<b>Step 0: Get pNFS kernel from the [[PNFS Development Git tree]] and install on all involved servers. </b>
<p>
<p>
<b>Step 1: Setup NFSv4 servers</b>
<b>Step 1: Setup NFSv4 servers</b>
<UL>
<UL>
-
<li>Create /etc/exports file on all DS's and the MDS.
+
<li>Create <tt>/etc/exports</tt> file on all DS's and the MDS.
<pre>
<pre>
/export  *(rw,sync,fsid=0,insecure,no_subtree_check)
/export  *(rw,sync,fsid=0,insecure,no_subtree_check)
 +
</pre>
 +
Note: starting with 2.6.32-rc1, the "pnfs" export option is required.
 +
<pre>
 +
/export  *(rw,sync,fsid=0,insecure,no_subtree_check,pnfs)
 +
</pre>
 +
Until support for the "pnfs" export option is publicly released,
 +
please build and install <tt>exportfs</tt>, <tt>rpc.mountd</tt>, <tt>rpc.nfsd</tt>, and optionally, <tt>nfsstat</tt> from:
 +
<pre>
 +
git://linux-nfs.org/~bhalevy/pnfs-nfs-utils.git
 +
 +
To install just the required binaries:
 +
cp utils/exportfs/exportfs /usr/sbin/exportfs
 +
cp utils/mountd/mountd /usr/sbin/rpc.mountd
 +
cp utils/mountd/nfsd /usr/sbin/rpc.nfsd
 +
cp utils/nfsstat/nfsstat /usr/sbin/nfsstat
</pre>
</pre>
</li>
</li>
-
<li>Run the following commands on on the storage server<p>
+
<li>Tell the metadata server the IP addresses of the data servers:
<pre>
<pre>
-
rpc.mountd
+
echo "/dev/sdc:192.168.0.1,192.168.0.2" >/proc/fs/nfsd/pnfs_dlm_device
-
rpc.nfsd 8
+
</pre>
-
exportfs -r
+
(replacing /dev/sdc by the name of the device hosting the exported gfs2 filesystem, and the ip addresses by the ip addresses of your data servers.)
 +
</li>
 +
<li>If the nfs service requires starting then run the following commands on the metadata server
 +
<pre>
 +
/etc/init.d/nfs restart
or
or
-
/etc/init.d/nfs start
+
rpc.mountd
-
</pre></p></li>
+
rpc.nfsd 8
 +
exportfs -r
 +
</pre></li>
</UL>
</UL>
-
<b>Step 2: Load File Layout driver on clients</b>
+
<b>Step 2: Load Layout driver on clients</b>
<pre>
<pre>
-
modprobe nfslayoutdriver
+
modprobe nfs_layout_nfsv41_files
</pre>
</pre>
<b>Step 3: Mount pNFS file system.</b>
<b>Step 3: Mount pNFS file system.</b>
<UL>
<UL>
<li>On pNFS client:
<li>On pNFS client:
-
<p><pre>
+
<pre>
-
mount -t nfs4 <i>"mds_server_name"</i>:/ /mnt/pnfs
+
mount -t nfs4 -o minorversion=1 <mds_server>:/ /mnt/pnfs
Note: Each file system has its own way of picking the MDS.  Ensure you mount only the MDS and not a DS.
Note: Each file system has its own way of picking the MDS.  Ensure you mount only the MDS and not a DS.
-
</pre></p></li>
+
</pre></li>
</UL>
</UL>
Line 38: Line 61:
<UL>
<UL>
<li>NFS debugging:  
<li>NFS debugging:  
-
<p><pre>
+
<pre>
echo 32767 > /proc/sys/sunrpc/nfsd_debug
echo 32767 > /proc/sys/sunrpc/nfsd_debug
echo 32767 > /proc/sys/sunrpc/nfs_debug
echo 32767 > /proc/sys/sunrpc/nfs_debug
-
</pre></p></li>
+
</pre></li>

Latest revision as of 15:56, 12 June 2015

THIS IS OUT OF DATE. The particular pnfs code described here is no longer under development.

File Layout

Accessing a storage system with pNFS

Step 0: Get pNFS kernel from the PNFS Development Git tree and install on all involved servers.

Step 1: Setup NFSv4 servers

  • Create /etc/exports file on all DS's and the MDS. 
    /export  *(rw,sync,fsid=0,insecure,no_subtree_check)

    Note: starting with 2.6.32-rc1, the "pnfs" export option is required. 

    /export  *(rw,sync,fsid=0,insecure,no_subtree_check,pnfs)

    Until support for the "pnfs" export option is publicly released, please build and install exportfs, rpc.mountd, rpc.nfsd, and optionally, nfsstat from: 

    git://linux-nfs.org/~bhalevy/pnfs-nfs-utils.git

To install just the required binaries:
cp utils/exportfs/exportfs /usr/sbin/exportfs
cp utils/mountd/mountd /usr/sbin/rpc.mountd
cp utils/mountd/nfsd /usr/sbin/rpc.nfsd
cp utils/nfsstat/nfsstat /usr/sbin/nfsstat
  • Tell the metadata server the IP addresses of the data servers: 
    echo "/dev/sdc:192.168.0.1,192.168.0.2" >/proc/fs/nfsd/pnfs_dlm_device

    (replacing /dev/sdc by the name of the device hosting the exported gfs2 filesystem, and the ip addresses by the ip addresses of your data servers.)

  • If the nfs service requires starting then run the following commands on the metadata server 
    /etc/init.d/nfs restart

or

rpc.mountd
rpc.nfsd 8
exportfs -r

Step 2: Load Layout driver on clients 

modprobe nfs_layout_nfsv41_files

Step 3: Mount pNFS file system.

  • On pNFS client: 
    mount -t nfs4 -o minorversion=1 <mds_server>:/ /mnt/pnfs

Note: Each file system has its own way of picking the MDS.  Ensure you mount only the MDS and not a DS.

Debugging Help

  • NFS debugging: 
    echo 32767 > /proc/sys/sunrpc/nfsd_debug
echo 32767 > /proc/sys/sunrpc/nfs_debug

Retrieved from "http://wiki.linux-nfs.org/wiki/index.php/PNFS_Setup_Instructions"
Personal tools