From Linux NFS

Section III

Contents 1 ROBUSTNESS TESTING 1.1 Basic stability assessments 1.2 Resource limit testing 1.3 Stress load testing 1.4 Scalability (robustness) 1.5 Recovery from problems while under light/normal/heavy loads

ROBUSTNESS TESTING

Basic stability assessments

• Run iozone for 2 weeks on basic client/server operations, using:
  • Both data and metadata options
  • Cached and direct I/O
  • Various mount options

• Run automounter use case for 2 weeks on amd, autofs, and autong, using:
  • Large number of maps
  • Randomly mount and run workloads on an automounted partition
  • use a variety of workloads, such as randomly chosen fs tests e.g. Crashme http://people.delphiforums.com/gjc/crashme.html

• Run NFS server for 2 wks with random configuration changes, using: Open OSDL
  • Interrupt server in various ways (reboot, power cycle, lan fail)
  • Change/reexport export rules at random
  • Trigger a client workload at arbitrary times
  • Analyze client recovery behaviors

• Run connectathon locking tests against NFS server for 2 weeks, using: New
  • Multiple client machines
  • Reboot at random
  • Analyze client cache coherency behaviors
  • Analyze locking behaviors

• Run fsstress 2 weeks on basic client/server operations, using: Done (1 week)
  • Long list random operations (1000 operations)
  • hight number of process (100)

• Run FFSB 1 day on basic client/server operations, using: Done (1 day)
  • 1 200 000 files
  • 100 directories

Resource limit testing

• Test stability of client in out of pid situation
• Test stability of client in out of disk space on server situation Done
• Test stability of client in out of inode situation
• Test stability of client in out of swap space situation
• Test stability of server in out of pid situation
• Test stability of server in out of memory situation
• Test stability of server in out of disk space situation Done
• Test stability of server in out of inode situation
• Test stability of server in out of swap space situation

Stress load testing

• Run LTP NFS fstress in a std config on each release fsx, fsstress (1hour), ffsb (1hour) In Progress
• Analyze load balancing, failure modes, etc. under different stress loads New
• Destructive testing by measuring point of failure for various loads New

Scalability (robustness)

• Find maximum number of connections to Linux IA-32 server Fsstress, fsx New
• Find maximum number of files for Linux IA-32 exported file system Fsstress, fsx
• Find maximum file size on Linux IA-32 Fsstress, fsx New
• Find maximum number of mounted file systems on client Fsstress, fsx New
• Test robustness on NUMA when scaling CPU, mem, NIC, or disk count New
• Test robustness on SMP when scaling CPU, mem, NIC, or disk count New
• Test correctness of NFS client when backed by a large (>100GB) cachefs New
• Find maximum number exported file systems on server New
• Find maximum size of exported file systems on server New

Recovery from problems while under light/normal/heavy loads

• Test short & long term local network failure (unplugged cable, ifdown eth0, etc.) Open OSDL
• Test short & long duration remote network partition Open OSDL
• Test behavior during crash/reboot of server with clients holding various states Open OSDL more
• Test multiple clients using, locking, etc. same files New
• Test behavior of server with failed storage device New
• Test behavior during crash of client with open delegations and locks New
• Test recovery from denied permission New
• Test recovery from JUKEBOX/DELAY New
• Test recovery from ESTALE New