Peer-to-peer NFS

From Linux NFS

Revision as of 14:34, 12 November 2012

Implementation

NFSD

Register DS

• [NFSD: Implement REGISTER_DS]
• Creates a new pnfs_p2p_client structure to store arguments from REGISTER_DS (netid, ip address, mds identifier, stateid) and attaches this structure to the struct nfs4_client sending the request. Assumes REGISTER_DS_ALL.

Unregister DS

• [NFSD: Implement UNREGISTER_DS]
• Frees the pnfs_p2p_client structure created by REGISTER_DS.

Layout get

• [NFSD: Find p2p device addresses for layoutget]
• Use most of pnfsd_lexp_layout_get(), adding a new function call when the device ID is set. I also add on to filelayout_encode_layout() to add the mds identifier to the front of the returned filehandle. pnfs_p2p_find_deviceid() will search the files delegation list and return the first client found that isn't the client placing the LAYOUTGET call. I send the DSs clientid as the device id for GETDEVINFO.

Get device info

• [NFSD: Find device address of a p2p DS]
• Use most of pnfsd_lexp_get_device_info(), but have the p2p code fill out the daddr return value in the p2p case. I can easily translate between devid and clientid since they're the same value, and this allows me to look up the netid and ip address from the client structure.

Put filehandle

• [NFSD: Make putfh work with p2p filehandles]
• Assume that any filehandle of length 36 is a p2p filehandle and split out the mds identifier from the rest of the handle. Don't do any of the state checking normally associated with putfh to avoid a crash since the NFS client doesn't set up an exports structure.

Read

• [NFSD: Make putfh work with p2p filehandles]
• If the filehandle is a p2p handle, then call the NFS client's nfs_proxy_open() function to map the p2pfh into a fh recognized by the DS.

Proxy open

• [NFSD: Implement PROXY_OPEN]
• Generate a new stateid representing the proxy open using the new struct pnfs_p2p_po_stid to store relevant information. Verify the filehandle exists using fh_verify(), since the DS skipped this check during the PUTFH call.

Expire clients

• [NFSD: Clean up p2p clients when they expire]
• Free up a client's p2p state when it expires.

Proxy revoke

• [most of PROXY_REVOKE]
• I send PROXY_REVOKE to clients that have PROXY_OPEN-ed files when a p2pclient (not p2pds) is expired. I haven't figured out the right way to free up the p2p stateid during nfsd4_cb_proxy_revoke_release() yet, my attempts all seem to cause an oops.

NFS

Register DS

• [NFS: Place a call to REGISTER_DS]
• Generate a new mds identifier using the cl_cb_ident and a static int that is incremented every REGISTER_DS call. Always sends REGISTER_DS_ALL as the sharing type. Called from nfs4_remote_mount() if nfs_fs_mount_common() returns success.

Unregister DS

• [NFS: Place a call to UNREGISTER_DS]
• Called from nfs4_destroy_server() when we are using v4.1

Proxy open

• [NFS: Place a call to PROXY_OPEN]
• I use the mds identifier to find the struct nfs_server that opened the file for caching and use it to place a call to PROXY_OPEN on the mounted server. This gives me a translated fh that I use to first find an associated inode (using nfs_delegation_find_inode(), and then a dentry using d_find_any_alias()). The dentry is returned to the nfsd code.

Proxy open

• [NFS: Store list of proxy opened files in the delegation]
• Check if we already have the file proxy-opened by storing some extra state in the delegation structure. If we already have a mapping to the real filehandle, then use that instead of placing a new PROXY_OPEN call.

Proxy revoke

• [most of PROXY_REVOKE]
• Delete the extra delegation state stored during PROXY_OPEN. This will force a new PROXY_OPEN call (and possibly start some kind of recovery) the next time the file is accessed.

VFS

struct super_block -> struct vfsmount

• [NFSD: Implement PROXY_OPEN]
• Look through sb->s_mounts to find a mount instance, inspired by d_find_any_alias(). All vfsmounts in the super_block point to the same filesystem, so we don't care which one is returned as long as NFSD gets something to pass to dentry_open().
• This is the biggest hack I used.