Peer-to-peer NFS

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
Line 2: Line 2:
=== NFSD ===
=== NFSD ===
==== Register DS ====
==== Register DS ====
-
* NFSD: Implement REGISTER_DS
+
* [NFSD: Implement REGISTER_DS]
* Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request.  Assumes REGISTER_DS_ALL.
* Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request.  Assumes REGISTER_DS_ALL.
==== Unregister DS ====
==== Unregister DS ====
-
* NFSD: Implement UNREGISTER_DS
+
* [NFSD: Implement UNREGISTER_DS]
* Frees the pnfs_p2p_client structure created by REGISTER_DS.
* Frees the pnfs_p2p_client structure created by REGISTER_DS.
==== Layout get ====
==== Layout get ====
-
* NFSD: Find p2p device addresses for layoutget
+
* [NFSD: Find p2p device addresses for layoutget]
* Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set.  I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle.  pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call.  I send the DSs clientid as the device id for GETDEVINFO.
* Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set.  I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle.  pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call.  I send the DSs clientid as the device id for GETDEVINFO.
==== Get device info ====
==== Get device info ====
-
* NFSD: Find device address of a p2p DS
+
* [NFSD: Find device address of a p2p DS]
* Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case.  I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure.
* Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case.  I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure.
==== Put filehandle ====
==== Put filehandle ====
-
* NFSD: Make putfh work with p2p filehandles
+
* [NFSD: Make putfh work with p2p filehandles]
* Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle.  Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure.
* Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle.  Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure.
==== Read ====
==== Read ====
-
* NFSD: Make putfh work with p2p filehandles
+
* [NFSD: Make putfh work with p2p filehandles]
* If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS.
* If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS.
==== Proxy open ====
==== Proxy open ====
-
* NFSD: Implement PROXY_OPEN
+
* [NFSD: Implement PROXY_OPEN]
* Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information.  Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call.
* Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information.  Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call.
==== Expire clients ====
==== Expire clients ====
-
* NFSD: Clean up p2p clients when they expire
+
* [NFSD: Clean up p2p clients when they expire]
* Free up a client's p2p state when it expires.
* Free up a client's p2p state when it expires.
==== Proxy revoke ====
==== Proxy revoke ====
-
* NFSD: most of PROXY_REVOKE
+
* [most of PROXY_REVOKE]
* I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired.  I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops.
* I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired.  I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops.
 +
 +
=== NFS ===
 +
==== Register DS ====
 +
* [NFS: Place a call to REGISTER_DS]
 +
* Generate a new mds identifier using the cl_cb_ident and a static int that is incremented every REGISTER_DS call.  Always sends REGISTER_DS_ALL as the sharing type.  Called from nfs4_remote_mount() if nfs_fs_mount_common() returns success.
 +
==== Unregister DS ====
 +
* [NFS: Place a call to UNREGISTER_DS]
 +
* Called from nfs4_destroy_server() when we are using v4.1
 +
==== Proxy open ====
 +
* [NFS: Place a call to PROXY_OPEN]
 +
* I use the mds identifier to find the struct nfs_server that opened the file for caching and use it to place a call to PROXY_OPEN on the mounted server.  This gives me a translated fh that I use to first find an associated inode (using nfs_delegation_find_inode(), and then a dentry using d_find_any_alias()).  The dentry is returned to the nfsd code.
 +
==== Proxy open ====
 +
* [NFS: Store list of proxy opened files in the delegation]
 +
* Check if we already have the file proxy-opened by storing some extra state in the delegation structure.  If we already have a mapping to the real filehandle, then use that instead of placing a new PROXY_OPEN call.
 +
==== Proxy revoke ====
 +
* [most of PROXY_REVOKE]
 +
* Delete the extra delegation state stored during PROXY_OPEN. This will force a new PROXY_OPEN call (and possibly start some kind of recovery) the next time the file is accessed.

Revision as of 14:32, 12 November 2012

Contents

Implementation

NFSD

Register DS

  • [NFSD: Implement REGISTER_DS]
  • Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request. Assumes REGISTER_DS_ALL.

Unregister DS

  • [NFSD: Implement UNREGISTER_DS]
  • Frees the pnfs_p2p_client structure created by REGISTER_DS.

Layout get

  • [NFSD: Find p2p device addresses for layoutget]
  • Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set. I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle. pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call. I send the DSs clientid as the device id for GETDEVINFO.

Get device info

  • [NFSD: Find device address of a p2p DS]
  • Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case. I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure.

Put filehandle

  • [NFSD: Make putfh work with p2p filehandles]
  • Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle. Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure.

Read

  • [NFSD: Make putfh work with p2p filehandles]
  • If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS.

Proxy open

  • [NFSD: Implement PROXY_OPEN]
  • Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information. Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call.

Expire clients

  • [NFSD: Clean up p2p clients when they expire]
  • Free up a client's p2p state when it expires.

Proxy revoke

  • [most of PROXY_REVOKE]
  • I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired. I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops.

NFS

Register DS

  • [NFS: Place a call to REGISTER_DS]
  • Generate a new mds identifier using the cl_cb_ident and a static int that is incremented every REGISTER_DS call. Always sends REGISTER_DS_ALL as the sharing type. Called from nfs4_remote_mount() if nfs_fs_mount_common() returns success.

Unregister DS

  • [NFS: Place a call to UNREGISTER_DS]
  • Called from nfs4_destroy_server() when we are using v4.1

Proxy open

  • [NFS: Place a call to PROXY_OPEN]
  • I use the mds identifier to find the struct nfs_server that opened the file for caching and use it to place a call to PROXY_OPEN on the mounted server. This gives me a translated fh that I use to first find an associated inode (using nfs_delegation_find_inode(), and then a dentry using d_find_any_alias()). The dentry is returned to the nfsd code.

Proxy open

  • [NFS: Store list of proxy opened files in the delegation]
  • Check if we already have the file proxy-opened by storing some extra state in the delegation structure. If we already have a mapping to the real filehandle, then use that instead of placing a new PROXY_OPEN call.

Proxy revoke

  • [most of PROXY_REVOKE]
  • Delete the extra delegation state stored during PROXY_OPEN. This will force a new PROXY_OPEN call (and possibly start some kind of recovery) the next time the file is accessed.
Personal tools