P2P Design Specification
From Linux NFS
| Line 6: | Line 6: | ||
= Overview = | = Overview = | ||
| - | + | Peer-to-peer pNFS is designed to solve the "boot storm" problem that happens when several clients in a cluster boot and attempt to read the same set of files from a single NFS server all at the same time. This could overload the server's bandwidth, slowing down operations on most client machines. The idea behind p2p NFS is to allow clients to act as an adhoc read-only pNFS data server that serves files out of their data cache. This should spread out network usage across all machines, rather than focusing all activity on a single node. Server and desired DS machines will need to be modified but any pNFS-enabled client already has the code required to read from adhoc DSs. | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
= Related Documents = | = Related Documents = | ||
* draft-myklebust-nfsv4-pnfs-backend-protocol-01.txt | * draft-myklebust-nfsv4-pnfs-backend-protocol-01.txt | ||
* [http://tools.ietf.org/html/rfc5661 RFC 5661] | * [http://tools.ietf.org/html/rfc5661 RFC 5661] | ||
| + | |||
= Dependencies = | = Dependencies = | ||
== This design needs the following from others: == | == This design needs the following from others: == | ||
Revision as of 20:11, 3 December 2012
| 3 December 2012 | DRAFT | bjschuma@netapp.com |
|---|
Contents |
Overview
Peer-to-peer pNFS is designed to solve the "boot storm" problem that happens when several clients in a cluster boot and attempt to read the same set of files from a single NFS server all at the same time. This could overload the server's bandwidth, slowing down operations on most client machines. The idea behind p2p NFS is to allow clients to act as an adhoc read-only pNFS data server that serves files out of their data cache. This should spread out network usage across all machines, rather than focusing all activity on a single node. Server and desired DS machines will need to be modified but any pNFS-enabled client already has the code required to read from adhoc DSs.
Related Documents
- draft-myklebust-nfsv4-pnfs-backend-protocol-01.txt
- RFC 5661
Dependencies
This design needs the following from others:
| Item | Description of Dependency or Issue | Affected Group | Contact |
|---|---|---|---|
| 1 | Linux pNFS server development code | Bryan Schumaker | Benny Halevy |
| 2 | pNFS nfs utils needs to be installed on the NFSD server so it can export a filesystem over pNFS. | Bryan Schumaker | Benny Halevy |
Assumptions
- Enable the following .config options for the pNFS client and pNFS ds machines:
- CONFIG_NFS_V4_1
- CONFIG_PNFS_FILE_LAYOUT
- Enable the following .config option for the pNFS ds machine:
- CONFIG_NFS_P2P
- Enable the following .config options for the pNFS server and pNFS ds machines:
- CONFIG_PNFSD
- CONFIG_PNFSD_LOCAL_EXPORT
- CONFIG_PNFSD_P2P
- Install pnfs-nfs-utils on the pNFS server
- Add "pnfs" to the export options of a local filesystem on the pNFS server
- pNFS DS should have nfsd running, but does not need to edit /etc/exports to share files
Design
DESCRIBE YOUR DESIGN IN THIS SECTION
This section is typically the largest section. Since designs are highly specific, the template cannot provide much in the way of guidelines here. Information which is relevant to the sections below should not be discussed here.
This is the main place where customizing the template for each particular team can really pay off. Teams are encouraged to add a section for the design considerations their own particular area needs to address.
The Design specification describes how the functionality is implemented. Intended readers are:
- Engineering (current and future)
- QA; given this spec, QA should understand the design enough to be able to create white-box type tests for the various parts.
- Overall design
This document should describe:- How it works, in detail.
- Module breakdown
- Major data paths through the code. (Referring to the use cases might be useful here)
- Process structure.
- Major data structures.
- Concurrency, parallelism, and mutual exclusion.
- Class hierarchy, if your design uses object-oriented notions of inheritance and polymorphism. This applies to, but is not limited to, development done in object-oriented languages such as C++ and Java.
- A UML diagram may be the easiest and most precise way of describing the relationship between the various abstractions supported by your design.
- Any state machines.
- What persistent storage is used? For Data ONTAP this might be files in the root, rdb databases, registry entries, and the like. For other products, it might be a client filesystem, a NetApp system somewhere, or dedicated hardware. What happens when (not if) these are lost due to failure or hardware replacement?
- Resources used, how they’re controlled, what we do when we run out, recovery steps
- What languages are involved.
- Document how the consistency model is maintained. (NG, CFO, consistency points, etc.)
- Licenses
- Describe how licenses are used, especially if the license checking must be done before the licensing infrastructure is initialized.
- Upgrade/revert
- Describe how upgrade and revert work.
- Discuss how these modules interact with CFO and SFO, data motion, and data replication.
- Install/uninstall
- Describe how the product is installed and uninstalled.
- Versioning/compatibility
- Describe how the versioning checks are implemented.
- If wire- or disk-layout is important, discuss tools (like IDL’s) used to achieve that.
- Internationalization/language support
- Branding and brand or vendor-neutral implementation.
- Configurations
- Describe algorithms related to the platform or architecture type.
- Describe algorithms affected by user configuration.
- Packaging
- Does it change the build/release/install process in any way (e.g. adds new build types, new build steps, new build files, new files to be shipped in the tar bundle, etc.) If so, describe how these are implemented.
- Online documentation
- Describe implementations of documentation of any form (for example, tools which process commentary and create other documents)
Feature Interaction Dependencies and Impacts
- nfsd <-> nfs
- The machine acting as a pNFS DS needs to be running both the nfs server and the nfs client.
Performance
Describe what if any aspects of the design impact the performance?
- What bottlenecks, limitations, or unpredictable performance effects may result from the design, and why?
- Discuss resource limitations and sizing issues as they apply to performance.
Scalability
Provide details about how scalability goals identified in the related Architecture and Functional Specifications will be met.
Provide descriptions of data structures, algorithms, and programmatic interfaces between Data ONTAP components, or between client and server, which are needed to achieve a scalable solution.
For example, fast lookup of a logical object may involve replacing use of a linear based search, with use of a hash table or btree based search.
Open Issues
Record in this section issues that you are aware of, but which are not yet resolved in the specification. If you discover issues after the specification is approved, you may record them here, and then re-review the specification after you address the issues.
| Item | Date | Name | Issue | Resolution | Date Resolved |
|---|---|---|---|---|---|
| 1 | Date the issue was raised. | Who raised it? | Describe the issue. | Describe what you did to resolve the issue. | Date |
Approvals
Approvers
| Name | Role | Target Approval Date | Approval Date |
|---|---|---|---|
| Trond Myklebust | NFS Client Maintainer | Date | Date |
Reviewers
| Name | Role | Target Approval Date | Approval Date |
|---|---|---|---|
| Jeffrey Heller | Bryan's Manager | Date | Date |
