Testing tools

From Linux NFS

(Difference between revisions)
Jump to: navigation, search
(Overview)
m (Reverted edits by MtmG2i (Talk); changed back to last version by Bfields)
 
(15 intermediate revisions not shown)
Line 1: Line 1:
-
All this programs are provided under the GPL, excepted SpecSFS. This tools are mainly issued from two projects :
+
All this programs are provided under the GPL, except SpecSFS. This tools are mainly issued from two projects :
*[http://lse.sf.net/ L.S.E]Linux Scalability Effort
*[http://lse.sf.net/ L.S.E]Linux Scalability Effort
*[http://ltp.sf.net/L.T.P Linux Tests Projects]
*[http://ltp.sf.net/L.T.P Linux Tests Projects]
Line 354: Line 354:
<tr>
<tr>
-
<td>[[#PyNFS|PyNFS]]</td>
+
<td>[[#Newpynfs|Newpynfs]]</td>
<td>
<td>
Line 384: Line 384:
<tr>
<tr>
-
<td>[[#Connectathon|connectathon]]</td>
+
<td>[[#Connectathon|Connectathon]]</td>
<td>
<td>
Line 489: Line 489:
=='''[http://www.coker.com.au/bonnie++/ Bonnie]'''==
=='''[http://www.coker.com.au/bonnie++/ Bonnie]'''==
A classical performances evaluation tool tests. The main program tests database type access to a single file (or a set of files if you wish to test more than 1G of storage), and it tests creation, reading, and deleting of small files which can simulate the usage of programs such as Squid, INN, or Maildir format email.
A classical performances evaluation tool tests. The main program tests database type access to a single file (or a set of files if you wish to test more than 1G of storage), and it tests creation, reading, and deleting of small files which can simulate the usage of programs such as Squid, INN, or Maildir format email.
-
=='''[http://samba.org/ftp/tridge/dbench/ DBench]'''
+
 
-
Dbench was written to allow independents developers to debug and test SAMBA.
+
'''Relevance to NFS:''':  [[Performance testing]], workload
 +
 
 +
=='''[http://samba.org/ftp/tridge/dbench/ DBench]'''==
 +
Dbench was written to allow independent developers to debug and test SAMBA.
It is heavily inspired of the original SAMBA tool : [http://lse.sourceforge.net/benchmarks/netbench/ NetBench]
It is heavily inspired of the original SAMBA tool : [http://lse.sourceforge.net/benchmarks/netbench/ NetBench]
Line 497: Line 500:
*improve the network load independently of the disk IO
*improve the network load independently of the disk IO
*Measure performances
*Measure performances
-
But it does not need as many hardware resources as NetBench to run.
+
But it does not need as much hardware resources as NetBench to run.
 +
 
 +
'''Relevance to NFS:''':
 +
 
 +
-Animesh Choudhury ,
 +
9986007415 .
 +
Datalifecycle.com Pvt. Ltd.
=='''[http://www.iozone.org/ IOZone]'''==
=='''[http://www.iozone.org/ IOZone]'''==
-
Performances tests suites. POSIX and 64 bits compliant. This tests is the file system test from the L.S.E. Main features
+
Performance tests suite. POSIX and 64 bits compliant. This tests is the file system test from the L.S.E. Main features
-
*POSIX async I/O, Mmap() file I/O, Normal file I/O
+
* POSIX async I/O, Mmap() file I/O, Normal file I/O
-
*Single stream measurement, Multiple stream measurement, Distributed
+
* Single stream measurement, Multiple stream measurement, Distributed file server measurements (Cluster)
-
file server measurements (Cluster)
+
* POSIX pthreads, Multi-process measurement
-
*POSIX pthreads, Multi-process measurement
+
* Selectable measurements with fsync, O_SYNC
-
*Selectable measurements with fsync, O_SYNC
+
* Latency plots
-
*Latency plots
+
 
 +
'''Relevance to NFS:''':  [[Performance testing]].  Good for exercising a given mount point under various load conditions.
=='''[http://weather.ou.edu/%7Eapw/projects/ Stress]'''==
=='''[http://weather.ou.edu/%7Eapw/projects/ Stress]'''==
This tool allow to set up a user specified amount of CPU, Disks IO, RAM and HDD load, and then to launch the test suite. Any error is detected and reported. It may be used to evaluate how well their systems will scale, by kernel programmers to evaluate perceived performance characteristics, and by systems programmers to expose the classes of bugs which only or more frequently manifest themselves when the system is under heavy load.
This tool allow to set up a user specified amount of CPU, Disks IO, RAM and HDD load, and then to launch the test suite. Any error is detected and reported. It may be used to evaluate how well their systems will scale, by kernel programmers to evaluate perceived performance characteristics, and by systems programmers to expose the classes of bugs which only or more frequently manifest themselves when the system is under heavy load.
 +
 +
'''Relevance to NFS:''':
=='''[http://www.cs.duke.edu/ari/fstress/ Fstress]'''==
=='''[http://www.cs.duke.edu/ari/fstress/ Fstress]'''==
-
A very flexible tool designed to evaluate NFS performances. fstress exports control over several dimensions in both data set and workload, enabling a wide range of tests for fundamental evaluation of file service scalability,
+
A very flexible tool designed to evaluate NFS performance. fstress exports control over several dimensions in both data set and workload, enabling a wide range of tests for fundamental evaluation of file service scalability,
sizing, configuration, and other factors.
sizing, configuration, and other factors.
Fstress includes several important "canned" workloads.
Fstress includes several important "canned" workloads.
-
=='''[http://www.spec.org/benchmarks.html#nfs Spec SFS]'''==
+
'''Relevance to NFS:''':  [[Robustness testing]], [[Performance testing]]
 +
 
 +
=='''[http://www.spec.org/benchmarks.html#nfs SpecSFS]'''==
This tool is an industry standard designed to test a NFS server by emulating the NFS client and verifying the correctness of the result. Since the low level protocol is verified, the tool is protocol dependent and must be
This tool is an industry standard designed to test a NFS server by emulating the NFS client and verifying the correctness of the result. Since the low level protocol is verified, the tool is protocol dependent and must be
adapted to the NFSv4 and IPv6 support.
adapted to the NFSv4 and IPv6 support.
Dbench, fstress and SpecSFS are directly designed and used to test networks file systems
Dbench, fstress and SpecSFS are directly designed and used to test networks file systems
 +
 +
'''Relevance to NFS:''':  Does not support NFSv4 capabilities; probably of limited use to NFSv4 testing.
=='''[http://ltp.sf.net/ OpenPOSIX]'''==
=='''[http://ltp.sf.net/ OpenPOSIX]'''==
OpenPOSIX is a tool designed to test the POSIX conformance. Currently, the test allow to test pthread, timers and ipc related functions. However, a few filesystems functions can be tested (open, close).
OpenPOSIX is a tool designed to test the POSIX conformance. Currently, the test allow to test pthread, timers and ipc related functions. However, a few filesystems functions can be tested (open, close).
-
=='''[http://www.nfsv4.org/ NFSv4 IETF project/connectathon]'''==
+
'''Relevance to NFS:'''
-
This test suite is the official suite of the connectathon to test clients/servers interoperability.
+
-
=='''[http://www.citi.umich.edu/projects/nfsv4/pynfs/ PyNFS]'''==
+
=='''[http://www.nfsv4.org/ Connectathon]'''==
-
PyNFS is the tool used to debug the Linux NFSv4 server. It maybe adapted to verify the RFC compliance.
+
This test suite from the NFSv4 IETF project is the official suite of the connectathon to test clients/servers interoperability.
 +
 
 +
'''Relevance to NFS:''':  [[Functionality testing]], [[Interoperability testing]].  Standard NFS test.  Needs updating to incorporate NFSv4 features, but still good for regression checking.
 +
 
 +
=='''[http://www.citi.umich.edu/projects/nfsv4/pynfs/ Newpynfs]'''==
 +
Newpynfs is rewritten from PyNFS, which is a useful test tool used to debug the Linux NFSv4 server. It maybe adapted to verify the RFC compliance.
 +
 
 +
'''Relevance to NFS:''':  [[Functionality testing]].  Includes NFSv4 specific features.  Good for regression checking.
=='''[http://www.namesys.com/benchmarks/mongo_readme.html Mongo]'''==
=='''[http://www.namesys.com/benchmarks/mongo_readme.html Mongo]'''==
Mongo is the filesystem suite from the Reiser4 filesystem project. It is a set of the programs to test linux filesystems for performance and functionality. The main program is mongo.pl script which creates the set of statistics for the file system variations specified by special mongo options. The mongo_parser.pl script parses those statistics and creates for them comparative html-table.
Mongo is the filesystem suite from the Reiser4 filesystem project. It is a set of the programs to test linux filesystems for performance and functionality. The main program is mongo.pl script which creates the set of statistics for the file system variations specified by special mongo options. The mongo_parser.pl script parses those statistics and creates for them comparative html-table.
 +
 +
'''Relevance to NFS:''':
=='''[http://www.netapp.com/tech_library/3022.html PostMark]'''==
=='''[http://www.netapp.com/tech_library/3022.html PostMark]'''==
Line 544: Line 568:
*Read file or Append file
*Read file or Append file
-
=='''[http://sourceforge.net/projects/ffsb/ Flexible File System Benchmark]'''==
+
'''Relevance to NFS:''':
 +
 
 +
=='''[http://sourceforge.net/projects/ffsb/ FFSB]'''==
The Flexible Filesystem Benchmark (FFSB) is a cross-platform filesystem performance measurement tool. It uses customizable profiles to measure of different workloads, and it supports multiple groups of threads across multiple filesystems. Depending on the profile It can be used for robustness and stress testing as well as performances testing.
The Flexible Filesystem Benchmark (FFSB) is a cross-platform filesystem performance measurement tool. It uses customizable profiles to measure of different workloads, and it supports multiple groups of threads across multiple filesystems. Depending on the profile It can be used for robustness and stress testing as well as performances testing.
-
=='''[http://ltp.sf.net/ (N)FSx]'''==
+
'''Relevance to NFS:''':  [[Performance testing]] and [[Robustness_testing|robustness testing]], depending on the profile used.  Serves similar role as Iozone and fsstress with different workloads.
-
NFSx is the nfs vrsion of fsx. That is the fs torture suite used by the [http://ltp.sf.net/ LTP] to test the linux filesystem robustness. Initialisations scripts from the ltp are yet modified to support the NFSv4 style syntax.
+
 
 +
=='''[http://ltp.sf.net/ NFSx]'''==
 +
NFSx is the nfs version of fsx. That is the fs torture suite used by the [http://ltp.sf.net/ LTP] to test the linux filesystem robustness. Initialisations scripts from the ltp are yet modified to support the NFSv4 style syntax.
 +
 
 +
'''Relevance to NFS:''':  [[Robustness testing]]

Latest revision as of 02:28, 19 August 2007

All this programs are provided under the GPL, except SpecSFS. This tools are mainly issued from two projects :


Contents

Overview

</tr>
Non Reg capability Performance evaluation Robustness evaluation NFSv3/v4 support IPv4/IPv6 POSIX conformance Interoperability evaluation RFC 3530 conformance

Bonnie++

yes

yes

no

v3/v4

IPv4/IPv6 no

yes

no
FFSB yes

yes

yes v3/v4

IPv4/IPv6

no yes no
NetBench yes yes yes

v3/v4

IPv4/IPv6

no

yes

no
DBench

yes

yes yes

v3/v4

IPv4/IPv6

no yes

no

IOZone yes yes no

v3/v4

IPv4/IPv6

no

yes

no
Stress
???
no yes

v3/v4

IPv4/IPv6

no yes no
fstress yes

yes

yes

v3/v4

IPv4/IPv6

no yes no
SpecSFS no yes no v3

IPv4

no

yes

yes

(N)FSx

yes

no yes

v3/v4

IPv4/IPv6

no

yes

no

Mongo ??? yes no

v3/v4

IPv4/IPv6

no

yes no
Newpynfs no no yes

v4

IPv4

no no yes
Connectathon

no

no

no v4 IPv4 no yes

-

OpenPOSIX no no no

v3/v4

IPv4/IPv6

partial

no

no

PostMark

???

yes

yes

v3/v4

IPv4/IPv6

no yes

no

Nota : All tests are interoperability evaluation capable, by testing all NFSv4 clients/servers combinations. To support NFSv4 and IPv6 the SpecSFS tool must be updated. According to a first sizing, 11 month may be necessary.

Description

Bonnie

A classical performances evaluation tool tests. The main program tests database type access to a single file (or a set of files if you wish to test more than 1G of storage), and it tests creation, reading, and deleting of small files which can simulate the usage of programs such as Squid, INN, or Maildir format email.

Relevance to NFS:: Performance testing, workload

DBench

Dbench was written to allow independent developers to debug and test SAMBA. It is heavily inspired of the original SAMBA tool : NetBench

As NetBench it allow to:

  • torture the file system
  • improve the network load independently of the disk IO
  • Measure performances

But it does not need as much hardware resources as NetBench to run.

Relevance to NFS::

-Animesh Choudhury , 9986007415 . Datalifecycle.com Pvt. Ltd.

IOZone

Performance tests suite. POSIX and 64 bits compliant. This tests is the file system test from the L.S.E. Main features

  • POSIX async I/O, Mmap() file I/O, Normal file I/O
  • Single stream measurement, Multiple stream measurement, Distributed file server measurements (Cluster)
  • POSIX pthreads, Multi-process measurement
  • Selectable measurements with fsync, O_SYNC
  • Latency plots

Relevance to NFS:: Performance testing. Good for exercising a given mount point under various load conditions.

Stress

This tool allow to set up a user specified amount of CPU, Disks IO, RAM and HDD load, and then to launch the test suite. Any error is detected and reported. It may be used to evaluate how well their systems will scale, by kernel programmers to evaluate perceived performance characteristics, and by systems programmers to expose the classes of bugs which only or more frequently manifest themselves when the system is under heavy load.

Relevance to NFS::

Fstress

A very flexible tool designed to evaluate NFS performance. fstress exports control over several dimensions in both data set and workload, enabling a wide range of tests for fundamental evaluation of file service scalability, sizing, configuration, and other factors.

Fstress includes several important "canned" workloads.

Relevance to NFS:: Robustness testing, Performance testing

SpecSFS

This tool is an industry standard designed to test a NFS server by emulating the NFS client and verifying the correctness of the result. Since the low level protocol is verified, the tool is protocol dependent and must be adapted to the NFSv4 and IPv6 support.

Dbench, fstress and SpecSFS are directly designed and used to test networks file systems

Relevance to NFS:: Does not support NFSv4 capabilities; probably of limited use to NFSv4 testing.

OpenPOSIX

OpenPOSIX is a tool designed to test the POSIX conformance. Currently, the test allow to test pthread, timers and ipc related functions. However, a few filesystems functions can be tested (open, close).

Relevance to NFS::

Connectathon

This test suite from the NFSv4 IETF project is the official suite of the connectathon to test clients/servers interoperability.

Relevance to NFS:: Functionality testing, Interoperability testing. Standard NFS test. Needs updating to incorporate NFSv4 features, but still good for regression checking.

Newpynfs

Newpynfs is rewritten from PyNFS, which is a useful test tool used to debug the Linux NFSv4 server. It maybe adapted to verify the RFC compliance.

Relevance to NFS:: Functionality testing. Includes NFSv4 specific features. Good for regression checking.

Mongo

Mongo is the filesystem suite from the Reiser4 filesystem project. It is a set of the programs to test linux filesystems for performance and functionality. The main program is mongo.pl script which creates the set of statistics for the file system variations specified by special mongo options. The mongo_parser.pl script parses those statistics and creates for them comparative html-table.

Relevance to NFS::

PostMark

PostMark was designed to create a large pool of continually changing files and to measure the transaction rates for a workload approximating a large Internet electronic mail server. It generates an initial pool of random text files ranging in size from a configurable low bound to a configurable high bound. This file pool is of configurable size and can be located on any accessible file system. Once the pool has been created (also producing statistics on continuous small file creation performance), a specified number of transactions occurs. Each transaction consists of a pair of smaller transactions:

  • Create file or Delete file
  • Read file or Append file
Relevance to NFS::

FFSB

The Flexible Filesystem Benchmark (FFSB) is a cross-platform filesystem performance measurement tool. It uses customizable profiles to measure of different workloads, and it supports multiple groups of threads across multiple filesystems. Depending on the profile It can be used for robustness and stress testing as well as performances testing.

Relevance to NFS:: Performance testing and robustness testing, depending on the profile used. Serves similar role as Iozone and fsstress with different workloads.

NFSx

NFSx is the nfs version of fsx. That is the fs torture suite used by the LTP to test the linux filesystem robustness. Initialisations scripts from the ltp are yet modified to support the NFSv4 style syntax.

Relevance to NFS:: Robustness testing

Personal tools