Testing methodologies
From Linux NFS
Revision as of 00:32, 3 June 2005
The NFSv4 Test Matrix itemizes a number of tasks for testing the client, server, and related technologies in a variety of different ways. In many cases it should be evident how to conduct the testing, but in other cases the tester may benefit from some additional detail or guidance for what exactly to look for or how to properly conduct the testing.
Normally, in formal testing you'd create a 'test plan', detailing everything the tester must do, the tools to use, the format of reports to be written, the specs for the hardware to use, etc. etc. But such documents are time consuming to write and make for dry reading. Fortunately, you probably can get 80% of the benefit for 20% of the work by just giving a 'test checklist' that gives the tester some general hints as to what to do. If they need something more formal, they can extrapolate from there.
Thus, the purpose of this HOWTO document is to explain how to make a test checklist for NFSv4.
Contents |
Example Test Methodology
For the impatient, here's an example. If this gives you a good enough idea what we're talking about, feel free to skip to the end where we say thank you and good luck. ;-)
Serviceability Testing
For any given service function (showmount, umount -f, etc. etc.) use the following checklist for verifying the functionality:
1. The feature reports accurate information
a) reports correctly during exception case situations
2. The feature does not cause misbehavior (ie an oops) when used
3. The feature is consistent with other standard reporting facilities
a) it uses recognized (POSIX) error codes
b) it is internationalized appropriately
c) it matches existing network and disk reporting tools
d) it has a man page
4. The feature does not cause undue performance impact
a) it does not cause memory leaks
5. The feature produces error messages that are useful
(self-explanatory, not too much noise, not unnecessarily alarming)
Philosophy
The basic purpose of these test checklists is to help guide a tester towards being able to do a good job. This helps all of us by ensuring a high level of quality. We'll have a better idea of how the tester did the work, and a better chance of recreating any issues they run across.
We assume the tester has a relatively good understanding of NFSv4; if they don't, we can expect that they can tap into other documentation to gain that knowledge.
Test Matrix Scope
The test matrix covers a broad range of testing activities. Some require sysadmin skills, some need development skills, and some need analytical skills. Obviously, different types of people are needed for these different roles. Thus, make sure your test checklist identifies the types of skills needed.
System testing activities are ones that involve running an existing test or performing a canned set of actions against a variety of configurations. The tester might be varying hardware, invoking failure cases, or altering the software composition of the system under test. The deliverable for this type of testing is a report summarizing the findings.
Test development activities augment existing test suites or create new tests for things that are not already covered. For instance, many existing tests do not yet include some of the newer NFSv4 features. A developer augments an existing test to add this functionality. The deliverable is a patch or tarball that other testers can use or that the test maintainer can incorporate.
Analyst activities require deep review and consideration of the item under test. For example, if there is a performance flaw, this would seek to identify and understand the cause. The analyst's deliverable is a set of recommendations to the developers to help characterize the problem and suggest how to fix it.
1. export filesystem to client. 2. let client allow to mount it. 3. remove all entries from rmtab on server. 4. restart all nfs daemons. 5. Now if client access filesystem , it gets permission denied error.Client can access filesystem only after remounting it. 6. This show that mountd still uses rmtab to check if client had mounted filesystem or not. No entry in rmtab for a client means it has not mounted filesystem.
Differences between various test items
- Assume the tester knows what the heck they're doing. Don't bother explaining basic stuff that the reader can figure out on their own with a little research. Do give elaboration to particular elements relevant to this task, that aren't well documented or that are highly important to understand.
- Give a reference if you have one.
Good Luck!
Thank you for giving consideration to helping the NFSv4 testing effort. Your test checklist will help enable the community to test your chosen test item more completely and accurately.
