Alternate Data Streams

From Linux NFS

Revision as of 19:50, 23 March 2016

The NFSv4 protocol includes "named attributes" (RFC 7530 section 5.3). Microsoft uses the term "Alternate Data Streams" (ADSs) for a similar NTFS feature. (See, for example, https://blogs.technet.microsoft.com/askcore/2013/03/24/alternate-data-streams-in-ntfs/).

They are sometimes confused with the Linux "extended attribute" feature, but the two are quite different:

Extended attributes are smaller. (XXX: maximum size varies, maybe around 64K on Linux?) The contents can only be replaced atomically, using special system calls; see xattr(7). See https://en.wikipedia.org/wiki/Extended_file_attributes for other platforms.

Alternate data streams (or named attributes) are files that happen to be looked up in a different way. You can hold references to them and read and write to them as you would a file, and they can be much larger. The lookup interface differs. NFSv4 provides an OPENATTR operation which takes a file and returns a directory filehandle; READDIR and LOOKUP on that filehandle can be used to get the filehandles of the alternate data streams. On Solaris openat with the O_XATTR flag performs a similar operation. (See fsattr(5), which confusingly calls these "extended attributes".) Windows seems to use a colon character to separate filename from ADS name, with ":" otherwise not allowed to appear in path names. For OSX, see https://en.wikipedia.org/wiki/Resource_fork (and note: "However, as forks can make it difficult to exchange files with other operating systems, this feature is not in common use. Even in OS X, resource forks are seldom used anymore."). See also https://en.wikipedia.org/wiki/Fork_%28file_system%29 ?

Linux has support for extended attributes, but not for alternate data streams or NFSv4 named attributes (either on client or server).

People have occasionally suggested implementing these interfaces in terms of each other. (E.g. implementing the extended attribute interface on the NFS client by using named attributes under the cover, or implementing named attributes on the server by using the exported filesystem's extended attribute support.) Problems with that:

• the above differences make conversion messy: e.g. write to an extended attribute must be implemented as a read-modify-write operation with setxattr and getxattr.
• that rules out compatibility with other platforms (like Windows) that keep the two separate.

The main motivation for implementing alternate data streams on Linux would probably be mainly Windows and OSX compatibility: for example, Samba (and to a lesser extent NFSv4) servers want to store them for Windows clients, and we might want to be able to access them on NTFS filesystems.

Implementing alternate data streams would mean designing and implementing the interface to look them up.

Reiserfs previously attempted to implement a similar feature (incorrectly), which promoted a previous thread "possible design issues for hybrids" that may be useful:

http://mid.gmane.org/20040826212853.GA21964@parcelfarce.linux.theplanet.co.uk

(Note on the use of the word "hybrid" here: allowing lookups of named attributes associated with a file is a lot like looking up names in a directory, making the file somewhat of a hybrid of a file and a directory.)

More recent discussion prompted by an attempt to provide an interface to ZFSOnLinux (a problematic project due to licensing issues):

https://en.wikipedia.org/wiki/Resource_fork