From Linux NFS

Section III

Contents 1 ROBUSTNESS TESTING 1.1 Basic stability assessments 1.2 Resource limit testing 1.3 Stress load testing 1.4 Scalability (robustness) 1.5 Recovery from problems while under light/normal/heavy loads

ROBUSTNESS TESTING

Basic stability assessments

	test	tool test	status	Owner	notes
III.A.1	Run iozone for 2 weeks on basic client/server operations, using: Both data and metadata options Cached and direct I/O Various mount options	IOzone	done	BULL	Now testing with fsstress and FFSB
III.A.2	Run automounter use case for 2 weeks on amd, autofs, and autong, using: Large number of maps Randomly mount and run workloads on an automounted partition use a variety of workloads, such as randomly chosen fs tests	e.g. Crashme more	New	none	none
III.A.3	Run NFS server for 2 wks with random configuration changes, using: Interrupt server in various ways (reboot, power cycle, lan fail) Change/reexport export rules at random Trigger a client workload at arbitrary times Analyze client recovery behaviors		OPEN	OSDL
III.A.4	Run connectathon locking tests against NFS server for 2 weeks, using: Multiple client machines Reboot at random Analyze client cache coherency behaviors Analyze locking behaviors		NEW
III.A.5	Run fsstress 2 weeks on basic client/server operations, using: Long list random operations (1000 operations) hight number of process (100)	fsstress	Done	BULL	1 week
III.A.6	Run FFSB 1 day on basic client/server operations in stress configuration, using: 1 200 000 files 100 directories	ffsb	Done	BULL	1 day

Resource limit testing

• Test stability of client in out of pid situation
• Test stability of client in out of disk space on server situation Done
• Test stability of client in out of inode situation
• Test stability of client in out of swap space situation
• Test stability of server in out of pid situation
• Test stability of server in out of memory situation
• Test stability of server in out of disk space situation Done
• Test stability of server in out of inode situation
• Test stability of server in out of swap space situation

Stress load testing

• Run LTP NFS fstress in a std config on each release fsx, fsstress (1hour), ffsb (1hour) In Progress
• Analyze load balancing, failure modes, etc. under different stress loads New
• Destructive testing by measuring point of failure for various loads New

Scalability (robustness)

• Find maximum number of connections to Linux IA-32 server Fsstress, fsx New
• Find maximum number of files for Linux IA-32 exported file system Fsstress, fsx
• Find maximum file size on Linux IA-32 Fsstress, fsx New
• Find maximum number of mounted file systems on client Fsstress, fsx New
• Test robustness on NUMA when scaling CPU, mem, NIC, or disk count New
• Test robustness on SMP when scaling CPU, mem, NIC, or disk count New
• Test correctness of NFS client when backed by a large (>100GB) cachefs New
• Find maximum number exported file systems on server New
• Find maximum size of exported file systems on server New

Recovery from problems while under light/normal/heavy loads

• Test short & long term local network failure (unplugged cable, ifdown eth0, etc.) Open OSDL
• Test short & long duration remote network partition Open OSDL
• Test behavior during crash/reboot of server with clients holding various states Open OSDL more
• Test multiple clients using, locking, etc. same files New
• Test behavior of server with failed storage device New
• Test behavior during crash of client with open delegations and locks New
• Test recovery from denied permission New
• Test recovery from JUKEBOX/DELAY New
• Test recovery from ESTALE New