Peer-to-peer NFS
From Linux NFS
(Difference between revisions)
| Line 2: | Line 2: | ||
=== NFSD === | === NFSD === | ||
==== Register DS ==== | ==== Register DS ==== | ||
| - | * NFSD: Implement REGISTER_DS | + | * [NFSD: Implement REGISTER_DS] |
* Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request. Assumes REGISTER_DS_ALL. | * Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request. Assumes REGISTER_DS_ALL. | ||
==== Unregister DS ==== | ==== Unregister DS ==== | ||
| - | * NFSD: Implement UNREGISTER_DS | + | * [NFSD: Implement UNREGISTER_DS] |
* Frees the pnfs_p2p_client structure created by REGISTER_DS. | * Frees the pnfs_p2p_client structure created by REGISTER_DS. | ||
==== Layout get ==== | ==== Layout get ==== | ||
| - | * NFSD: Find p2p device addresses for layoutget | + | * [NFSD: Find p2p device addresses for layoutget] |
* Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set. I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle. pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call. I send the DSs clientid as the device id for GETDEVINFO. | * Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set. I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle. pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call. I send the DSs clientid as the device id for GETDEVINFO. | ||
==== Get device info ==== | ==== Get device info ==== | ||
| - | * NFSD: Find device address of a p2p DS | + | * [NFSD: Find device address of a p2p DS] |
* Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case. I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure. | * Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case. I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure. | ||
==== Put filehandle ==== | ==== Put filehandle ==== | ||
| - | * NFSD: Make putfh work with p2p filehandles | + | * [NFSD: Make putfh work with p2p filehandles] |
* Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle. Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure. | * Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle. Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure. | ||
==== Read ==== | ==== Read ==== | ||
| - | * NFSD: Make putfh work with p2p filehandles | + | * [NFSD: Make putfh work with p2p filehandles] |
* If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS. | * If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS. | ||
==== Proxy open ==== | ==== Proxy open ==== | ||
| - | * NFSD: Implement PROXY_OPEN | + | * [NFSD: Implement PROXY_OPEN] |
* Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information. Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call. | * Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information. Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call. | ||
==== Expire clients ==== | ==== Expire clients ==== | ||
| - | * NFSD: Clean up p2p clients when they expire | + | * [NFSD: Clean up p2p clients when they expire] |
* Free up a client's p2p state when it expires. | * Free up a client's p2p state when it expires. | ||
==== Proxy revoke ==== | ==== Proxy revoke ==== | ||
| - | * | + | * [most of PROXY_REVOKE] |
* I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired. I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops. | * I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired. I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops. | ||
| + | |||
| + | === NFS === | ||
| + | ==== Register DS ==== | ||
| + | * [NFS: Place a call to REGISTER_DS] | ||
| + | * Generate a new mds identifier using the cl_cb_ident and a static int that is incremented every REGISTER_DS call. Always sends REGISTER_DS_ALL as the sharing type. Called from nfs4_remote_mount() if nfs_fs_mount_common() returns success. | ||
| + | ==== Unregister DS ==== | ||
| + | * [NFS: Place a call to UNREGISTER_DS] | ||
| + | * Called from nfs4_destroy_server() when we are using v4.1 | ||
| + | ==== Proxy open ==== | ||
| + | * [NFS: Place a call to PROXY_OPEN] | ||
| + | * I use the mds identifier to find the struct nfs_server that opened the file for caching and use it to place a call to PROXY_OPEN on the mounted server. This gives me a translated fh that I use to first find an associated inode (using nfs_delegation_find_inode(), and then a dentry using d_find_any_alias()). The dentry is returned to the nfsd code. | ||
| + | ==== Proxy open ==== | ||
| + | * [NFS: Store list of proxy opened files in the delegation] | ||
| + | * Check if we already have the file proxy-opened by storing some extra state in the delegation structure. If we already have a mapping to the real filehandle, then use that instead of placing a new PROXY_OPEN call. | ||
| + | ==== Proxy revoke ==== | ||
| + | * [most of PROXY_REVOKE] | ||
| + | * Delete the extra delegation state stored during PROXY_OPEN. This will force a new PROXY_OPEN call (and possibly start some kind of recovery) the next time the file is accessed. | ||
Revision as of 14:32, 12 November 2012
Contents |
Implementation
NFSD
Register DS
- [NFSD: Implement REGISTER_DS]
- Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request. Assumes REGISTER_DS_ALL.
Unregister DS
- [NFSD: Implement UNREGISTER_DS]
- Frees the pnfs_p2p_client structure created by REGISTER_DS.
Layout get
- [NFSD: Find p2p device addresses for layoutget]
- Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set. I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle. pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call. I send the DSs clientid as the device id for GETDEVINFO.
Get device info
- [NFSD: Find device address of a p2p DS]
- Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case. I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure.
Put filehandle
- [NFSD: Make putfh work with p2p filehandles]
- Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle. Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure.
Read
- [NFSD: Make putfh work with p2p filehandles]
- If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS.
Proxy open
- [NFSD: Implement PROXY_OPEN]
- Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information. Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call.
Expire clients
- [NFSD: Clean up p2p clients when they expire]
- Free up a client's p2p state when it expires.
Proxy revoke
- [most of PROXY_REVOKE]
- I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired. I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops.
NFS
Register DS
- [NFS: Place a call to REGISTER_DS]
- Generate a new mds identifier using the cl_cb_ident and a static int that is incremented every REGISTER_DS call. Always sends REGISTER_DS_ALL as the sharing type. Called from nfs4_remote_mount() if nfs_fs_mount_common() returns success.
Unregister DS
- [NFS: Place a call to UNREGISTER_DS]
- Called from nfs4_destroy_server() when we are using v4.1
Proxy open
- [NFS: Place a call to PROXY_OPEN]
- I use the mds identifier to find the struct nfs_server that opened the file for caching and use it to place a call to PROXY_OPEN on the mounted server. This gives me a translated fh that I use to first find an associated inode (using nfs_delegation_find_inode(), and then a dentry using d_find_any_alias()). The dentry is returned to the nfsd code.
Proxy open
- [NFS: Store list of proxy opened files in the delegation]
- Check if we already have the file proxy-opened by storing some extra state in the delegation structure. If we already have a mapping to the real filehandle, then use that instead of placing a new PROXY_OPEN call.
Proxy revoke
- [most of PROXY_REVOKE]
- Delete the extra delegation state stored during PROXY_OPEN. This will force a new PROXY_OPEN call (and possibly start some kind of recovery) the next time the file is accessed.
