https://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&feed=atom&action=historyServer-side silly rename - Revision history2024-03-28T10:37:04ZRevision history for this page on the wikiMediaWiki 1.16.5https://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5815&oldid=prevBfields at 20:27, 18 January 20222022-01-18T20:27:57Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:27, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 36:</td>
<td colspan="2" class="diff-lineno">Line 36:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Approaches I (bfields) considered and rejected for now:</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Approaches I (bfields) considered and rejected for now:</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">- </del>Create a link in the new directory on every open, and remove it on every close. But open may be a frequent operation, and we'd need to actually sync that link to disk on every operation, so it could be pretty slow. But maybe, with cooperation of the filesystem, we could *just* do the link on open, and delay waiting for the sync until there's an unlink.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">* </ins>Create a link in the new directory on every open, and remove it on every close. But open may be a frequent operation, and we'd need to actually sync that link to disk on every operation, so it could be pretty slow. But maybe, with cooperation of the filesystem, we could *just* do the link on open, and delay waiting for the sync until there's an unlink.</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">- </del>Filesystems already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. I considered hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. It would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So in the end maybe it wouldn't work.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">* </ins>Filesystems already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. I considered hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. It would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So in the end maybe it wouldn't work.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5814&oldid=prevBfields at 20:27, 18 January 20222022-01-18T20:27:38Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:27, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 34:</td>
<td colspan="2" class="diff-lineno">Line 34:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The extra hidden link will mean that the st_nlink (for local users) and the numlinks attribute (for NFSv4 GETATTR callers) are wrong. We could fix up the latter, at least, by checking for this specific case.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The extra hidden link will mean that the st_nlink (for local users) and the numlinks attribute (for NFSv4 GETATTR callers) are wrong. We could fix up the latter, at least, by checking for this specific case.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">--</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Approaches I (bfields) considered and rejected for now:</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Another possibility I considered was just creating </del>a link in the new directory on every open, and <del class="diffchange diffchange-inline">removing </del>it on every close. But open may be a frequent operation, and we'd need to actually sync that link to disk on every operation, so it could be pretty slow. But maybe, with cooperation of the filesystem, we could *just* do the link on open, and delay waiting for the sync until there's an unlink.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">- Create </ins>a link in the new directory on every open, and <ins class="diffchange diffchange-inline">remove </ins>it on every close. But open may be a frequent operation, and we'd need to actually sync that link to disk on every operation, so it could be pretty slow. But maybe, with cooperation of the filesystem, we could *just* do the link on open, and delay waiting for the sync until there's an unlink.</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>- <ins class="diffchange diffchange-inline">Filesystems </ins>already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. I considered hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. It would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So in the end maybe it wouldn't work.</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>-<del class="diffchange diffchange-inline">-</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Another possibility: filesystems </del>already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. I considered hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. It would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So in the end maybe it wouldn't work.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5813&oldid=prevBfields at 20:26, 18 January 20222022-01-18T20:26:38Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:26, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 26:</td>
<td colspan="2" class="diff-lineno">Line 26:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Then we need the close code to check whether we're closing one of these files and, if so, to also unlink it from the hidden directory.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Then we need the close code to check whether we're closing one of these files and, if so, to also unlink it from the hidden directory.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>And, finally the laundromat code, after it ends the grace period, needs to walk through the hidden directory and remove any files that haven't been opened. Maybe nfsd4_recdir_purge_old() would <del class="diffchange diffchange-inline">do</del>. This is usually the kind of thing we try not to do from the kernel, but I don't see a clean way to do it from userspace.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>And, finally the laundromat code, after it ends the grace period, needs to walk through the hidden directory and remove any files that haven't been opened. Maybe <ins class="diffchange diffchange-inline">code like that in </ins>nfsd4_recdir_purge_old() would <ins class="diffchange diffchange-inline">work</ins>. This is usually the kind of thing we try not to do from the kernel, but I don't see a clean way to do it from userspace.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>That done, if we wanted to also make this work for unlinks by non-NFSv4 clients, we'd need some way to intercept all the unlinks to a given filesystem. We might need to modify the individual exported filesystems.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>That done, if we wanted to also make this work for unlinks by non-NFSv4 clients, we'd need some way to intercept all the unlinks to a given filesystem. We might need to modify the individual exported filesystems.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5812&oldid=prevBfields at 19:13, 18 January 20222022-01-18T19:13:05Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:13, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 14:</td>
<td colspan="2" class="diff-lineno">Line 14:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence. (Update: see <del class="diffchange diffchange-inline">]</del>https://lore.kernel.org/linux-nfs/20220118190251.55526-1-olga.kornievskaia@gmail.com/T/#u].)</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence. (Update: see <ins class="diffchange diffchange-inline">[</ins>https://lore.kernel.org/linux-nfs/20220118190251.55526-1-olga.kornievskaia@gmail.com/T/#u].)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5811&oldid=prevBfields at 19:12, 18 January 20222022-01-18T19:12:47Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:12, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 14:</td>
<td colspan="2" class="diff-lineno">Line 14:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence. (Update: see https://lore.kernel.org/linux-nfs/20220118190251.55526-1-olga.kornievskaia@gmail.com/T/#u .)</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence. (Update: see <ins class="diffchange diffchange-inline">]</ins>https://lore.kernel.org/linux-nfs/20220118190251.55526-1-olga.kornievskaia@gmail.com/T/#u<ins class="diffchange diffchange-inline">]</ins>.)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5810&oldid=prevBfields at 19:07, 18 January 20222022-01-18T19:07:06Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:07, 18 January 2022</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 14:</td>
<td colspan="2" class="diff-lineno">Line 14:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4.1 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence. <ins class="diffchange diffchange-inline"> (Update: see https://lore.kernel.org/linux-nfs/20220118190251.55526-1-olga.kornievskaia@gmail.com/T/#u .)</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5809&oldid=prevBfields at 21:13, 30 November 20212021-11-30T21:13:32Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:13, 30 November 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 18:</td>
<td colspan="2" class="diff-lineno">Line 18:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The server side looks harder.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>One complication is knfsd doesn't get exclusive use of exported filesystems: other applications may also be using them. A file opened by an NFS client could be unlinked by a local application, and we'd like the file not to disappear after reboot in that case. That said, the current behavior doesn't handle that case--it doesn't even handle the case when the unlink is done by a different client than the open--so for a first implementation I think it'd be fine to ignore that case.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>One complication is <ins class="diffchange diffchange-inline">that </ins>knfsd doesn't get exclusive use of exported filesystems: other applications may also be using them. A file opened by an NFS client could be unlinked by a local application, and we'd like the file not to disappear after reboot in that case. That said, the current behavior doesn't handle that case--it doesn't even handle the case when the unlink is done by a different client than the open--so for a first implementation I think it'd be fine to ignore that case.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>My rough plan for knfsd is to create a hidden directory in the root of the exported filesystem and modify nfsd4_remove() to check whether the file to be unlinked is open by an NFSv4 client, and if so to instead rename it to that hidden directory. The name shouldn't matter--just use a counter or something.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>My rough plan for knfsd is to create a hidden directory in the root of the exported filesystem and modify nfsd4_remove() to check whether the file to be unlinked is open by an NFSv4 client, and if so to instead rename it to that hidden directory. The name shouldn't matter--just use a counter or something.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5808&oldid=prevBfields at 21:12, 30 November 20212021-11-30T21:12:49Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 21:12, 30 November 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 6:</td>
<td colspan="2" class="diff-lineno">Line 6:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>What we did to make open file removal work on remote files was check in the client VFS remove operation if the file is open, and if so rename it instead of removing it. This makes it (sort of) invisible to the client and still allows reading and writing. The client kernel then removes the new name when the vnode becomes inactive. We call this the 3/4 solution because if the client crashes between the rename and remove a garbage file is left on the server. An entry to cron can be added to clean up on the server.</p></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>What we did to make open file removal work on remote files was check in the client VFS remove operation if the file is open, and if so rename it instead of removing it. This makes it (sort of) invisible to the client and still allows reading and writing. The client kernel then removes the new name when the vnode becomes inactive. We call this the 3/4 solution because if the client crashes between the rename and remove a garbage file is left on the server. An entry to cron can be added to clean up on the server.</p></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div></blockquote></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div></blockquote></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2"> </td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td></tr>
<tr><td colspan="2" class="diff-lineno">Line 13:</td>
<td colspan="2" class="diff-lineno">Line 12:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4 protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>that would preserve the file after reboot as well. (And it could use a separate directory for the purpose, and avoid the rmdir). We even added a bit to the NFSv4<ins class="diffchange diffchange-inline">.1 </ins>protocol so that the server can tell the client it does this, allowing the client to skip sillyrename (see references to OPEN4_RESULT_PRESERVE_UNLINKED in [https://tools.ietf.org/html/rfc8881].)</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>My rough plan for knfsd is to create a <del class="diffchange diffchange-inline">"</del>hidden<del class="diffchange diffchange-inline">" </del>directory in the root of the exported filesystem and <del class="diffchange diffchange-inline">create </del>a <del class="diffchange diffchange-inline">link there </del>for <del class="diffchange diffchange-inline">each </del>open file <del class="diffchange diffchange-inline">on</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">I suspect the client side implementation of this wouldn't be hard--it'd need to watch for the OPEN4_RESULT_PRESERVE_UNLINKED flag and skip silly rename in its presence.</ins></div></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">open</del>, <del class="diffchange diffchange-inline">deleted on </del>close. <del class="diffchange diffchange-inline"> On restart</del>, the <del class="diffchange diffchange-inline">server would wait till </del>the <del class="diffchange diffchange-inline">end of its </del>grace period <del class="diffchange diffchange-inline">for clients </del>to <del class="diffchange diffchange-inline">reclaim </del>any files <del class="diffchange diffchange-inline">they had open before</del>, <del class="diffchange diffchange-inline">and then delete any leftover links</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">The server side looks harder.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">One complication is knfsd doesn't get exclusive use of exported filesystems: other applications may also be using them. A file opened by an NFS client could be unlinked by a local application, and we'd like the file not to disappear after reboot in that case. That said, the current behavior doesn't handle that case--it doesn't even handle the case when the unlink is done by a different client than the open--so for a first implementation I think it'd be fine to ignore that case.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>My rough plan for knfsd is to create a hidden directory in the root of the exported filesystem and <ins class="diffchange diffchange-inline">modify nfsd4_remove() to check whether the file to be unlinked is open by an NFSv4 client, and if so to instead rename it to that hidden directory. The name shouldn't matter--just use </ins>a <ins class="diffchange diffchange-inline">counter or something.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">I think we can use something like the logic at the start of nfsd4_process_open2 to look up a struct nfs4_file from the filehandle, and then use that to check </ins>for <ins class="diffchange diffchange-inline">nfsv4 opens. We also need to prevent the race where a new </ins>open <ins class="diffchange diffchange-inline">comes in after we decide to unlink the </ins>file <ins class="diffchange diffchange-inline">but before we're done unlinking it--I'm not sure how. Also we need to think about the possibility of filehandle aliasing</ins>, <ins class="diffchange diffchange-inline">in which case there may exist two nfs4_files for a given file.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Then we need the </ins>close <ins class="diffchange diffchange-inline">code to check whether we're closing one of these files and, if so, to also unlink it from the hidden directory</ins>.</div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">And</ins>, <ins class="diffchange diffchange-inline">finally </ins>the <ins class="diffchange diffchange-inline">laundromat code, after it ends </ins>the grace period<ins class="diffchange diffchange-inline">, needs </ins>to <ins class="diffchange diffchange-inline">walk through the hidden directory and remove </ins>any files <ins class="diffchange diffchange-inline">that haven't been opened. Maybe nfsd4_recdir_purge_old() would do. This is usually the kind of thing we try not to do from the kernel</ins>, <ins class="diffchange diffchange-inline">but I don't see a clean way to do it from userspace.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">That done, if we wanted to also make this work for unlinks by non-NFSv4 clients, we'd need some way to intercept all the unlinks to a given filesystem. We might need to modify the individual exported filesystems</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We may want to think about how exactly to hide that directory. Maybe we could get some kind of help from the filesystem.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We may want to think about how exactly to hide that directory. Maybe we could get some kind of help from the filesystem.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del class="diffchange diffchange-inline">Filesystems </del>already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. <del class="diffchange diffchange-inline">Another possibility </del>I considered <del class="diffchange diffchange-inline">was </del>hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. <del class="diffchange diffchange-inline">But that </del>would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So <del class="diffchange diffchange-inline">I think that would be too complicated and fragile</del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">The extra hidden link will mean that the st_nlink (for local users) and the numlinks attribute (for NFSv4 GETATTR callers) are wrong. We could fix up the latter, at least, by checking for this specific case.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">--</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Another possibility I considered was just creating a link in the new directory on every open, and removing it on every close. But open may be a frequent operation, and we'd need to actually sync that link to disk on every operation, so it could be pretty slow. But maybe, with cooperation of the filesystem, we could *just* do the link on open, and delay waiting for the sync until there's an unlink.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">--</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins class="diffchange diffchange-inline">Another possibility: filesystems </ins>already have to deal with the case where the system crashes while there are unlinked open files. I believe they keep a list of such files so they can free them in fsck or next mount. I considered hooking into that process somehow--perhaps the server could be given an interface allowing it to discover those orphaned files. <ins class="diffchange diffchange-inline">It </ins>would require nfsd to be involved in the mount process (currently we mount first, then export). And we'd have to figure out how to perform clean shutdowns without losing those files. And we'd have to worry about losing them any time an administrator fsck'd or mounted without running nfsd. So <ins class="diffchange diffchange-inline">in the end maybe it wouldn't work</ins>.</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5754&oldid=prevBfields at 16:03, 5 January 20212021-01-05T16:03:16Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 16:03, 5 January 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 10:</td>
<td colspan="2" class="diff-lineno">Line 10:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>NFSv4 actually does have open and close calls, and our server won't free a file until last close--unless the server reboots, at which point the file will disappear even <del class="diffchange diffchange-inline">though </del>an application on the client is still using it. NFS is supposed to keep working normally across server reboots<del class="diffchange diffchange-inline">. So </del>the client still does silly rename even in the v4 case.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>NFSv4 actually does have open and close calls, and our server won't free a file until last close--unless the server reboots, at which point the file will disappear even <ins class="diffchange diffchange-inline">if </ins>an application on the client is still using it. NFS is supposed to keep working normally across server reboots<ins class="diffchange diffchange-inline">, so </ins>the client still does silly rename even in the v4 case.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td></tr>
</table>Bfieldshttps://wiki.linux-nfs.org/wiki/index.php?title=Server-side_silly_rename&diff=5753&oldid=prevBfields at 16:02, 5 January 20212021-01-05T16:02:41Z<p></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 16:02, 5 January 2021</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 10:</td>
<td colspan="2" class="diff-lineno">Line 10:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Silly rename is indeed an imperfect solution. Another case when users sometimes notice the ".nfsXXXX" files is when they try to remove a directory that contains them. Also, it doesn't help if a file is unlinked by a different client than the one that holds it open.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>NFSv4 actually does have open and close calls, and our server won't free a file until last close<del class="diffchange diffchange-inline">. Until </del>the server reboots, at which point the file will disappear even though an application on the client is still using it. NFS is supposed to keep working normally across server reboots. So the client still does silly rename even in the v4 case.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>NFSv4 actually does have open and close calls, and our server won't free a file until last close<ins class="diffchange diffchange-inline">--unless </ins>the server reboots, at which point the file will disappear even though an application on the client is still using it. NFS is supposed to keep working normally across server reboots. So the client still does silly rename even in the v4 case.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>We could move the responsibility for silly rename to the server--the server could keep a hardlink to the file after unlink, and</div></td></tr>
</table>Bfields