ChucksProjects

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
(Unifying storage management on Linux)
 
(21 intermediate revisions not shown)
Line 3: Line 3:
== Mount ==
== Mount ==
-
* Add scalable mounting (more efficient use of network resources; mountd connection caching)
+
=== Testing and documentation ===
-
* Rewrite nfs(5)
+
* Add real unit testing
* Add real unit testing
 +
* Construct a set of specifications for mount behavior
 +
* Rewrite nfs(5)
 +
* Test "-o remount" -- handled by /usr/sbin/mount instead of /sbin/mount.nfs?
 +
=== UI clean up ===
* Make error reporting more useful
* Make error reporting more useful
 +
** Some error messages are too specific ("RPC Error: RPC program not available" is meaningless to most users)
 +
** Some error messages are too general ("mount.nfs: mount failed" doesn't explain how to rectify the problem)
* Complete internationalization
* Complete internationalization
-
* Properly implement of break-back retries
+
 
 +
=== New features ===
* Implement support for IPv6
* Implement support for IPv6
* Implement support for server failover options
* Implement support for server failover options
-
* Test "-o remount" -- handled by /usr/sbin/mount instead of /sbin/mount.nfs?
+
* Add scalable mounting (more efficient use of network resources; mountd connection caching)
 +
* Properly implement break-back retries
[[MountNotes|More notes]]
[[MountNotes|More notes]]
 +
 +
== NFS support for IPv6 networking ==
 +
 +
* Get the delegation / NLM stuff working
 +
* Rewrite the referral logic in user-space (DNS resolution can be needed to handle an NFSv4 referral)
 +
* Review svcauth_unix stuff to remove open-coded IPv4 address dependencies
 +
* Track where we are with user-space tools and libraries
 +
* Existing NFS IPv6 code for the client
 +
 +
== NFS client performance tools ==
 +
 +
* Replace jiffies-based timestamps with ktime in the RPC client for more precise latency measurement
 +
* Consider adding existing Python utilities to nfs-utils
 +
* Update vmstat and iostat tools to provide statistics about NFS mount points
 +
* Construct new tools that provide fine-grained data about NFS mount points
 +
 +
== Eliminate BKL from NFS client ==
 +
 +
* Remove BKL dependency from attribute cache
 +
* Remove BKL usage from XDR layer
 +
* Remove BKL usage from RPC client and GSS
 +
* Eliminate BKL in NFS client proper
 +
 +
=== From OmniOutliner ===
 +
 +
* Bruce found that the BKL is held while calling normal XDR encoding and decoding functions
 +
** Why did I miss the existence BKL around XDR encoding?
 +
** Generic xdr.c: look for anything caching an xdr_buf, especially during decode
 +
** Still need an audit of NFSv4 XDR
 +
*** Copying state ids
 +
*** Migration - file handles can change
 +
*** Theoretically the client ID is in danger as well
 +
* Trond has RCU'd lots of stuff shared with NFSv4 XDR
 +
* and also the RPC auth cache
 +
* Radix trees are now RCU -- so write path locking might be relaxed
 +
 +
== Alternate socket-based transport implementations ==
 +
 +
* One UDP socket per client
 +
* Two TCP sockets per rpc_clnt (one for sync, one for async requests; could be done by an NFS client change instead)
 +
* One TCP socket per client CPU
 +
* One TCP socket per server, shared by mount points (that's probably an NFS client change, not a transport change)
 +
* DMA-engine based TCP socket transport
 +
* SCTP-based transport (server-side switch and NFS on SCTP spec needed for that)
== Rebuild the NFS server and client XDR layer from the ground up ==
== Rebuild the NFS server and client XDR layer from the ground up ==
Line 54: Line 105:
[[StorageUnificationTool|More notes]]
[[StorageUnificationTool|More notes]]
-
== Construct productized performance tools based on /proc/self/mounstats ==
+
== Linux De-duplication appliance ==
 +
 
 +
* Provide high capacity near-line storage for inexpensive disk-based backup
 +
* Based on content-addressable storage on top of RAID 6
 +
* Looks like one or more very large OSD LUNs or a very large NFS server
 +
 
 +
[[LinuxDeDupNotes|More notes]]
 +
 
 +
== Object Storage Devices ==

Latest revision as of 16:19, 4 December 2007

Stuff I'm playing with, thinking about, or watching admiringly...

Contents

Mount

Testing and documentation

  • Add real unit testing
  • Construct a set of specifications for mount behavior
  • Rewrite nfs(5)
  • Test "-o remount" -- handled by /usr/sbin/mount instead of /sbin/mount.nfs?

UI clean up

  • Make error reporting more useful
    • Some error messages are too specific ("RPC Error: RPC program not available" is meaningless to most users)
    • Some error messages are too general ("mount.nfs: mount failed" doesn't explain how to rectify the problem)
  • Complete internationalization

New features

  • Implement support for IPv6
  • Implement support for server failover options
  • Add scalable mounting (more efficient use of network resources; mountd connection caching)
  • Properly implement break-back retries

More notes

NFS support for IPv6 networking

  • Get the delegation / NLM stuff working
  • Rewrite the referral logic in user-space (DNS resolution can be needed to handle an NFSv4 referral)
  • Review svcauth_unix stuff to remove open-coded IPv4 address dependencies
  • Track where we are with user-space tools and libraries
  • Existing NFS IPv6 code for the client

NFS client performance tools

  • Replace jiffies-based timestamps with ktime in the RPC client for more precise latency measurement
  • Consider adding existing Python utilities to nfs-utils
  • Update vmstat and iostat tools to provide statistics about NFS mount points
  • Construct new tools that provide fine-grained data about NFS mount points

Eliminate BKL from NFS client

  • Remove BKL dependency from attribute cache
  • Remove BKL usage from XDR layer
  • Remove BKL usage from RPC client and GSS
  • Eliminate BKL in NFS client proper

From OmniOutliner

  • Bruce found that the BKL is held while calling normal XDR encoding and decoding functions
    • Why did I miss the existence BKL around XDR encoding?
    • Generic xdr.c: look for anything caching an xdr_buf, especially during decode
    • Still need an audit of NFSv4 XDR
      • Copying state ids
      • Migration - file handles can change
      • Theoretically the client ID is in danger as well
  • Trond has RCU'd lots of stuff shared with NFSv4 XDR
  • and also the RPC auth cache
  • Radix trees are now RCU -- so write path locking might be relaxed

Alternate socket-based transport implementations

  • One UDP socket per client
  • Two TCP sockets per rpc_clnt (one for sync, one for async requests; could be done by an NFS client change instead)
  • One TCP socket per client CPU
  • One TCP socket per server, shared by mount points (that's probably an NFS client change, not a transport change)
  • DMA-engine based TCP socket transport
  • SCTP-based transport (server-side switch and NFS on SCTP spec needed for that)

Rebuild the NFS server and client XDR layer from the ground up

  • More complete type checking and error reporting
  • Each function called with a simplified virtual API
  • All functions re-entrant so we can remove the BKL

NFS client build testing

  • Builds kernel with "allmodconfig" and "allyesconfig"
  • Automated - no user intervention once started
  • Tests all patches in a series
  • Later add more static analysis for each patch
  • Need to add post-analysis tool that compares the build logs
  • Need to eliminate as many spurious warnings generated by the tool chain as possible

More notes

Advanced file sync tool

  • multi-purpose generic file system syncer - a file synchronization tool that runs on most Unix-flavored operating systems
  • one-way or bidirectional data synchronization
  • syncs file changes periodically or continuously
  • supports disconnected operation (later playback of changes)
  • can use advanced file system features to improve efficiency
  • the basic idea is to use an inotify-like mechanism to make the detection of file and directory modification more efficient


More notes

Unifying storage management on Linux

  • A easy-to-use graphical tool that can manage storage devices and pools, and file systems
  • Includes a wizard (like Roxio or Toast) that can start different applications depending on what is to be accomplished
  • Make it easy to create, manage, and back up very large data storage pools
  • Make it easy to export and secure data
  • Provide a framework for thinking about and documenting data management

More notes

Linux De-duplication appliance

  • Provide high capacity near-line storage for inexpensive disk-based backup
  • Based on content-addressable storage on top of RAID 6
  • Looks like one or more very large OSD LUNs or a very large NFS server

More notes

Object Storage Devices

Personal tools