https://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&feed=atom&action=historyNFS for AFS users - Revision history2024-03-29T12:38:36ZRevision history for this page on the wikiMediaWiki 1.16.5https://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5823&oldid=prevBfields: /* PAGS */2022-01-27T20:35:23Z<p><span class="autocomment">PAGS</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:35, 27 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 68:</td>
<td colspan="2" class="diff-lineno">Line 68:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>PAGs: AFS allows a group of processes to share a common identity, different from the local uid, for the purposes of accessing an AFS filesystem: https://docs.openafs.org/AdminGuide/HDRWQ63.html</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>PAGs: AFS allows a group of processes to share a common identity, different from the local uid, for the purposes of accessing an AFS filesystem: https://docs.openafs.org/AdminGuide/HDRWQ63.html</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">So, for example, if you have multiple kerberos identities that you use to access AFS, you can pick which one you want to use at a given time, or even use both, each in a different window. We'd like this for NFS as well.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Dave Howells says: "This is why I added session keyrings. You can run a process in a new keyring</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Dave Howells says: "This is why I added session keyrings. You can run a process in a new keyring</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 81:</td>
<td colspan="2" class="diff-lineno">Line 83:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The key used to open a file would then need to be cached in the file struct</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The key used to open a file would then need to be cached in the file struct</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>private data."</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>private data."</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">So, we have a lot of good kernel infrastructure in place which is designed to do this, but (despite an attempt or two) nobody has managed to quite make it work for NFS yet.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== ACLs ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== ACLs ==</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5822&oldid=prevBfields: /* volume location database and global namespace */2022-01-27T20:30:28Z<p><span class="autocomment">volume location database and global namespace</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:30, 27 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 61:</td>
<td colspan="2" class="diff-lineno">Line 61:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>NFS has standards for DNS discovery of a server from a domain, in theory we could use that. Handling kerberos users across domains would be interesting.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>NFS has standards for DNS discovery of a server from a domain, in theory we could use that. Handling kerberos users across domains would be interesting.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Within one domain, <del class="diffchange diffchange-inline">there's </del>a "Volume Location Database" that keeps track of volumes and where (machine and partition) they're located. You can make a volume for a purpose; give particular people access to it, give it some storage, expand and contract it and move it around. Volumes have quotas.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Within one domain, <ins class="diffchange diffchange-inline">AFS has </ins>a "Volume Location Database" that keeps track of volumes and where (machine and partition) they're located. You can make a volume for a purpose; give particular people access to it, give it some storage, expand and contract it and move it around. Volumes have quotas.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Within </del>a given domain, <del class="diffchange diffchange-inline">We </del>can assemble a namespace out of volumes using referrals. For a higher-level approach more similar to AFS's, there's also [https://wiki.linux-nfs.org/wiki/index.php/FedFsUtilsProject FedFS] which stores the namespace information in a database and provides common protocols for administration tools to manipulate the database.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">With NFS, within </ins>a given domain, <ins class="diffchange diffchange-inline">we </ins>can assemble a namespace out of volumes using referrals. For a higher-level approach more similar to AFS's, there's also [https://wiki.linux-nfs.org/wiki/index.php/FedFsUtilsProject FedFS] which stores the namespace information in a database and provides common protocols for administration tools to manipulate the database<ins class="diffchange diffchange-inline">. That just provides namespace-management facilities. If it were combined with a kerberized distributed volume manager built on top of LVM, that might server as a more complete AFS VLDB replacement</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== PAGS ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== PAGS ==</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5821&oldid=prevBfields: /* replication and migration */2022-01-27T20:27:26Z<p><span class="autocomment">replication and migration</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:27, 27 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 53:</td>
<td colspan="2" class="diff-lineno">Line 53:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* When we distribute new versions, allow servers to keep around older versions and serve files from them in the case filehandle lookups against the new copy fail, to be removed only after applications stop referencing them. Hopefully this can be done space-efficiently if the different versions on the replica servers can be represented as dm snapshots.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* When we distribute new versions, allow servers to keep around older versions and serve files from them in the case filehandle lookups against the new copy fail, to be removed only after applications stop referencing them. Hopefully this can be done space-efficiently if the different versions on the replica servers can be represented as dm snapshots.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>If we use NFSv4 proxies instead, proxies will hold opens or delegations on the files on the original server, which will prevent their being deleted while in use. The problem is server reboots. That's partially worked around with silly-rename. [[Server-side silly<del class="diffchange diffchange-inline">-</del>rename]] would be a more complete solution.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>If we use NFSv4 proxies instead, proxies will hold opens or delegations on the files on the original server, which will prevent their being deleted while in use. The problem is server reboots. That's partially worked around with silly-rename. [[Server-side silly rename]] would be a more complete solution.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== volume location database and global namespace ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== volume location database and global namespace ==</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5820&oldid=prevBfields: /* replication and migration */2022-01-27T20:27:07Z<p><span class="autocomment">replication and migration</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:27, 27 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 45:</td>
<td colspan="2" class="diff-lineno">Line 45:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Latency-hiding may be particularly tricky; delegation and caching policies may need rethinking. Performance will be more complicated to understand compared to AFS-like read-only replicas.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Latency-hiding may be particularly tricky; delegation and caching policies may need rethinking. Performance will be more complicated to understand compared to AFS-like read-only replicas.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>AFS-like volume replication has a problem: when new read-only versions are released, they may <del class="diffchange diffchange-inline">modify or </del>delete <del class="diffchange diffchange-inline">entirely </del>files that are in use by running processes. I'd expect application crashes. I wonder how AFS administrators deal with that now?</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>AFS-like volume replication has a problem: when new read-only versions are released, they may delete files that are in use by running processes. <ins class="diffchange diffchange-inline">Applications probably don't expect ESTALE on in-use files; </ins>I'd expect application crashes. I wonder how AFS administrators deal with that now?</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>My impression is that AFS doesn't reliably prevent this problem, so instead AFS administrators work around it, for example by keeping old versions of binaries in place (and using symlinks to direct users to the newest versions).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>My impression is that AFS doesn't reliably prevent this problem, so instead AFS administrators work around it, for example by keeping old versions of binaries in place (and using symlinks to direct users to the newest versions)<ins class="diffchange diffchange-inline">. So maybe NFS doesn't need to solve this problem either</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Possible approaches to fix the problem if we wanted to:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Possible approaches to fix the problem if we wanted to:</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 53:</td>
<td colspan="2" class="diff-lineno">Line 53:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* When we distribute new versions, allow servers to keep around older versions and serve files from them in the case filehandle lookups against the new copy fail, to be removed only after applications stop referencing them. Hopefully this can be done space-efficiently if the different versions on the replica servers can be represented as dm snapshots.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* When we distribute new versions, allow servers to keep around older versions and serve files from them in the case filehandle lookups against the new copy fail, to be removed only after applications stop referencing them. Hopefully this can be done space-efficiently if the different versions on the replica servers can be represented as dm snapshots.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>If we use NFSv4 proxies instead, proxies will hold opens or delegations on the files on the original server, which will prevent their being deleted while in use. The problem is server reboots. That's partially worked around with silly-rename. Server-side silly-rename would be a more complete solution.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>If we use NFSv4 proxies instead, proxies will hold opens or delegations on the files on the original server, which will prevent their being deleted while in use. The problem is server reboots. That's partially worked around with silly-rename. <ins class="diffchange diffchange-inline">[[</ins>Server-side silly-rename<ins class="diffchange diffchange-inline">]] </ins>would be a more complete solution.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== volume location database and global namespace ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== volume location database and global namespace ==</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5819&oldid=prevBfields: /* replication and migration */2022-01-27T20:23:46Z<p><span class="autocomment">replication and migration</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:23, 27 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 30:</td>
<td colspan="2" class="diff-lineno">Line 30:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* lvmsync: looks possibly unmaintained? We wouldn't want to depend on this. But possibly it could be a proof of concept or starting point.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>* lvmsync: looks possibly unmaintained? We wouldn't want to depend on this. But possibly it could be a proof of concept or starting point.</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">Between LVM and (container-respecting) knfsd, we have a lot of the necessary pieces, but there's at a minimum a lot of tooling and documentation to write before this is usable.</del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Clients could be configured to mount particular servers by hand, or they could mount any server and then use [https://tools.ietf.org/html/rfc5661#section-11.9 fs_locations], [https://tools.ietf.org/html/rfc5661#section-11.10 fs_locations_info], or maybe even [https://datatracker.ietf.org/doc/rfc8435/ pnfs flexfiles] to get lists of servers hosting replicas and pick one. They would need some heuristics to make the right choice. It would also be nice if clients could fail over to a different replica when one goes down.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Clients could be configured to mount particular servers by hand, or they could mount any server and then use [https://tools.ietf.org/html/rfc5661#section-11.9 fs_locations], [https://tools.ietf.org/html/rfc5661#section-11.10 fs_locations_info], or maybe even [https://datatracker.ietf.org/doc/rfc8435/ pnfs flexfiles] to get lists of servers hosting replicas and pick one. They would need some heuristics to make the right choice. It would also be nice if clients could fail over to a different replica when one goes down.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5784&oldid=prevBfields: /* quotas */2021-04-01T19:56:05Z<p><span class="autocomment">quotas</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:56, 1 April 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 124:</td>
<td colspan="2" class="diff-lineno">Line 124:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could get similar functionality with LVM thin provisioning or XFS with project quotas. (There is some work needed there to treat projects as separate exports, but that's very doable.)</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could get similar functionality with LVM thin provisioning or XFS with project quotas. (There is some work needed there to treat projects as separate exports, but that's very doable.)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Note NFS, ext4, xfs, and other filesystems all support per-user (and other) quotas. That's not something AFS has, as far as I know. Some <del class="diffchange diffchange-inline">[[</del>notes on NFSv4 quota support]].</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Note NFS, ext4, xfs, and other filesystems all support per-user (and other) quotas. That's not something AFS has, as far as I know. Some notes on <ins class="diffchange diffchange-inline">[[</ins>NFSv4 quota support]].</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5783&oldid=prevBfields: /* quotas */2021-04-01T19:55:47Z<p><span class="autocomment">quotas</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:55, 1 April 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 120:</td>
<td colspan="2" class="diff-lineno">Line 120:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== quotas ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== quotas ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">NFSv3 had the RQUOTA protocol</del>. <del class="diffchange diffchange-inline">NFSv4 has three attributes (</del>see <del class="diffchange diffchange-inline">https://tools</del>.<del class="diffchange diffchange-inline">ietf</del>.<del class="diffchange diffchange-inline">org/html/rfc5661#section-5.8.2.28 and following</del>) <del class="diffchange diffchange-inline">that support quotas</del>, <del class="diffchange diffchange-inline">but Linux has not implemented them</del>. <del class="diffchange diffchange-inline">Maybe we should</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">AFS has per-volume quotas</ins>. <ins class="diffchange diffchange-inline">There are no per-user quotas that I can </ins>see<ins class="diffchange diffchange-inline">; instead, AFS administrators create volumes for individual users (e</ins>.<ins class="diffchange diffchange-inline">g</ins>.<ins class="diffchange diffchange-inline">, for individual home directories</ins>), <ins class="diffchange diffchange-inline">and set quotas on those</ins>. <ins class="diffchange diffchange-inline">Volumes can share the same storage, and it's fine for quotas on volumes to add up to more than the available storage</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">The linux implementation of RQUOTA seems to live along </del>with <del class="diffchange diffchange-inline">the local filesystem quota utilites at https://sourceforge.net/projects/linuxquota/. The same "quota" tool can make calls to either local filesystems </del>or <del class="diffchange diffchange-inline">RQUOTA calls to the server</del>. <del class="diffchange diffchange-inline">One of the goals of the NFSv4 protocol was to move all functionality into the single NFS protocol </del>(<del class="diffchange diffchange-inline">among other reasons, </del>to <del class="diffchange diffchange-inline">simplify firewall traversal)</del>, <del class="diffchange diffchange-inline">so we should really be shifting people to using the NFSv4 attributes instead of the separate RQUOTA protocol. https://sourceforge.net/p/linuxquota/code/ci/master/tree/doc/quotadoc.sgml looks like one place to start understanding how local filesystems deal with quotas</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">We could get similar functionality </ins>with <ins class="diffchange diffchange-inline">LVM thin provisioning </ins>or <ins class="diffchange diffchange-inline">XFS with project quotas</ins>. (<ins class="diffchange diffchange-inline">There is some work needed there </ins>to <ins class="diffchange diffchange-inline">treat projects as separate exports</ins>, <ins class="diffchange diffchange-inline">but that's very doable</ins>.<ins class="diffchange diffchange-inline">)</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Some local filesystems quotas seem to be managed using an actual on-disk file whose format is understood by both the kernel filesystem code </del>and <del class="diffchange diffchange-inline">the userspace "quota" tool. That isn't an appropriate interface for the NFS filesystem. But </del>filesystems <del class="diffchange diffchange-inline">are also able to hide the on</del>-<del class="diffchange diffchange-inline">disk quota information </del>and <del class="diffchange diffchange-inline">system calls to get and set </del>quotas. <del class="diffchange diffchange-inline">Perhaps NFS could use some subset of that interface (see quotactl(2)), or perhaps it needs to define its own simpler calls.</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Note NFS, ext4, xfs, </ins>and <ins class="diffchange diffchange-inline">other </ins>filesystems <ins class="diffchange diffchange-inline">all support per</ins>-<ins class="diffchange diffchange-inline">user (</ins>and <ins class="diffchange diffchange-inline">other) </ins>quotas. <ins class="diffchange diffchange-inline">That</ins>'s <ins class="diffchange diffchange-inline">not something AFS has</ins>, <ins class="diffchange diffchange-inline">as far as I know</ins>. <ins class="diffchange diffchange-inline">Some [[notes on NFSv4 quota support]]</ins>.</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">AFS appears to only have quotas for volumes; per-user quotas are implemented using per-user volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">So, really, NFS/xfs/ext4 quotas may not be what</del>'s <del class="diffchange diffchange-inline">needed. Instead</del>, <del class="diffchange diffchange-inline">we can probably get similar functionality with thin provisioning and "df"</del>. <del class="diffchange diffchange-inline">XFS project quotas could also do the same job</del>. <del class="diffchange diffchange-inline"> (Note some work is needed there to treat projects as separate exports, but that's very doable.)</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5782&oldid=prevBfields: /* quotas */2021-04-01T14:03:32Z<p><span class="autocomment">quotas</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 14:03, 1 April 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 128:</td>
<td colspan="2" class="diff-lineno">Line 128:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>AFS appears to only have quotas for volumes; per-user quotas are implemented using per-user volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>AFS appears to only have quotas for volumes; per-user quotas are implemented using per-user volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>So, really, NFS/xfs/ext4 quotas may not be what's needed. Instead, we can probably get similar functionality with thin provisioning and "df".</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>So, really, NFS/xfs/ext4 quotas may not be what's needed. Instead, we can probably get similar functionality with thin provisioning and "df". <ins class="diffchange diffchange-inline"> XFS project quotas could also do the same job. (Note some work is needed there to treat projects as separate exports, but that's very doable.)</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5781&oldid=prevBfields: /* quotas */2021-04-01T01:51:29Z<p><span class="autocomment">quotas</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 01:51, 1 April 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 126:</td>
<td colspan="2" class="diff-lineno">Line 126:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Some local filesystems quotas seem to be managed using an actual on-disk file whose format is understood by both the kernel filesystem code and the userspace "quota" tool. That isn't an appropriate interface for the NFS filesystem. But filesystems are also able to hide the on-disk quota information and system calls to get and set quotas. Perhaps NFS could use some subset of that interface (see quotactl(2)), or perhaps it needs to define its own simpler calls.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Some local filesystems quotas seem to be managed using an actual on-disk file whose format is understood by both the kernel filesystem code and the userspace "quota" tool. That isn't an appropriate interface for the NFS filesystem. But filesystems are also able to hide the on-disk quota information and system calls to get and set quotas. Perhaps NFS could use some subset of that interface (see quotactl(2)), or perhaps it needs to define its own simpler calls.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>AFS appears to only have quotas for volumes; per-user quotas are implemented using per-<del class="diffchange diffchange-inline">quota </del>volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>AFS appears to only have quotas for volumes; per-user quotas are implemented using per-<ins class="diffchange diffchange-inline">user </ins>volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>So, really, NFS/xfs/ext4 quotas may not be what's needed. Instead, we can probably get similar functionality with thin provisioning and "df".</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>So, really, NFS/xfs/ext4 quotas may not be what's needed. Instead, we can probably get similar functionality with thin provisioning and "df".</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=NFS_for_AFS_users&diff=5780&oldid=prevBfields: /* quotas */2021-04-01T01:31:09Z<p><span class="autocomment">quotas</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 01:31, 1 April 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 124:</td>
<td colspan="2" class="diff-lineno">Line 124:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The linux implementation of RQUOTA seems to live along with the local filesystem quota utilites at https://sourceforge.net/projects/linuxquota/. The same "quota" tool can make calls to either local filesystems or RQUOTA calls to the server. One of the goals of the NFSv4 protocol was to move all functionality into the single NFS protocol (among other reasons, to simplify firewall traversal), so we should really be shifting people to using the NFSv4 attributes instead of the separate RQUOTA protocol. https://sourceforge.net/p/linuxquota/code/ci/master/tree/doc/quotadoc.sgml looks like one place to start understanding how local filesystems deal with quotas.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The linux implementation of RQUOTA seems to live along with the local filesystem quota utilites at https://sourceforge.net/projects/linuxquota/. The same "quota" tool can make calls to either local filesystems or RQUOTA calls to the server. One of the goals of the NFSv4 protocol was to move all functionality into the single NFS protocol (among other reasons, to simplify firewall traversal), so we should really be shifting people to using the NFSv4 attributes instead of the separate RQUOTA protocol. https://sourceforge.net/p/linuxquota/code/ci/master/tree/doc/quotadoc.sgml looks like one place to start understanding how local filesystems deal with quotas.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Hooking up </del>local filesystem quota <del class="diffchange diffchange-inline">support </del>to <del class="diffchange diffchange-inline">NFSv4, </del>and <del class="diffchange diffchange-inline">then figuring out how </del>to <del class="diffchange diffchange-inline">expose that stuff on the client side, could be a reasonable project</del>. <del class="diffchange diffchange-inline">We would also need </del>to <del class="diffchange diffchange-inline">look at </del>AFS <del class="diffchange diffchange-inline">filesystem </del>quotas and <del class="diffchange diffchange-inline">compare</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Some </ins>local <ins class="diffchange diffchange-inline">filesystems quotas seem to be managed using an actual on-disk file whose format is understood by both the kernel </ins>filesystem <ins class="diffchange diffchange-inline">code and the userspace "</ins>quota<ins class="diffchange diffchange-inline">" tool. That isn't an appropriate interface for the NFS filesystem. But filesystems are also able </ins>to <ins class="diffchange diffchange-inline">hide the on-disk quota information </ins>and <ins class="diffchange diffchange-inline">system calls </ins>to <ins class="diffchange diffchange-inline">get and set quotas</ins>. <ins class="diffchange diffchange-inline">Perhaps NFS could use some subset of that interface (see quotactl(2)), or perhaps it needs </ins>to <ins class="diffchange diffchange-inline">define its own simpler calls.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>AFS <ins class="diffchange diffchange-inline">appears to only have </ins>quotas <ins class="diffchange diffchange-inline">for volumes; per-user quotas are implemented using per-quota volumes: https://docs.openafs.org/AdminGuide/HDRWQ234.html</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">So, really, NFS/xfs/ext4 quotas may not be what's needed. Instead, we can probably get similar functionality with thin provisioning </ins>and <ins class="diffchange diffchange-inline">"df"</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>= migrating existing AFS installations to NFS =</div></td></tr>
</table>Bfields