Jenkins CI

From Linux NFS

Revision as of 20:31, 3 June 2011

Jenkins Setup

Before doing anything, make sure the required environment variables are set to correct values (see the section "Required Environment Variables" below).

Begin by cloning the jenkins-nfs git repository:

	git clone git://git.linux-nfs.org/projects/bjschuma/jenkins-nfs.git

The cloned directory should be placed in a location exported by the NFS server running on the host machine. The full path up to and including this directory is referred to as $NFS_WORKSPACE throughout this document and various scripts.

Once you have the latest version of the scripts, cd into $NFS_WORKSPACE/jenkins-nfs and run ./setup to download the Jenkins executable, required plugins, and to configure the jenkins environment. In addition, a launcher script called start_jenkins will be placed in $NFS_WORKSPACE. Jenkins is then started by running $NFS_WORKSPACE/start_jenkins.

Before you can use Jenkins to test kernels, you must first configure a "default" server. This is the server hosting the Jenkins installation, and will be used to install kernels on the virtual machine clients. To add the default server, run the "Admin - Add Server" job and enter "default" for the "NAME" parameter.

You must also configure what tests to run and virtual machines to run them on. See the sections below for instructions on how to do this. Once everything is configured, you can begin a new run by starting the job "Compile and Install".

Required Environment Variables

These need to be set before starting Jenkins

• NFS_SSH_USER
  • The username of the virtual machine user that will install the compiled kernel and run the tests. SSH should be configured to avoid prompting for a password.
• NFS_MOUNT_POINT
  • The path on the virtual machine that NFS_SERVER is mounted on.

Adding a new architecture

Jenkins is configured for i386 and x86_64 by default. Other architectures can be added, but manual configuration is required. You will have to modify both "Compile and Install" and "Run Tests". Select each job from the dashboard and then click "Configure" on the left. Scroll down to the "Configuration Matrix" and add the new architecture to the list of possible values for the ARCH variable.

Adding a new .config

Copy the config file to $NFS_WORKSPACE/config/ using a unique name. Then modify "Compile and Install" and "Run Tests" to add the new file to the CONFIG variable in the "Test Matrix".

Adding a new virtual machine client

Create the new VM in virt-manager. Once it has been created, start the "Initialize VM" job to configure passwordless SSH to the VM and install various tests (such as connectathon). See the section "Virtual machine setup" below for help creating new VMs.

Adding a new server

Set up the server (either as a VM or a real machine), and then run the "Add Server" job to create a server config file. When the job has finished, edit "Run Tests" and add the server name to the SERVER variable of the "Test Matrix"

Adding a new NFS version

Edit the "Run Tests" job and add the new version to the list of possible values for the NFS variable. NFS v2, v3, and v4 are configured by default. Adding v4.1 will require modification of the mount command to add in the "minorversion=1" mount option.

Adding a new test

First design your test and edit $NFS_WORKSPACE/tests.map to add a mapping from name -> command. The general format of this file is:

        name:where:command

     name: The name of the test
    where: Where the command will be run (either "remote" or "local")
  command: The command to run the test

Once the new test has been added to the tests.map file, add the new test to the "Run Tests" job.

Blacklisting a combination

Both "Compile and Install" and "Run Tests" are configured as Jenkins multi-configuration projects. This means that every combination of possible variable values is tested. Some combination of variables may not make sense, such as running 32bit tests on a 64bit virtual machine. The files in $NFS_WORKSPACE/filters/ can be used to blacklist (or whitelist) various combinations. These files are simple shell scripts that should exit 0 if the combination can be tested. See the files in $NFS_WORKSPACE/filters/ for more information.

Testing NFS v4.1

NFS v4.1 is still considered to be experimental and requires manual intervention to enable on both the client and server. As a result, running tests over NFS v4.1 has been disabled by default.

Once you have your client and server configured correctly, edit the file $NFS_WORKSPACE/filters/runtests and remove or comment out the block of code under "#### Disable NFS v4.1 tests ####"

$NFS_WORKSPACE/scripts/functions

This file provides many convenient functions for controlling virtual machines. You can boot, halt, and reboot machines. You can also do more complicated actions like sending files, mounting the nfs server and running remote commands. Before using any of these functions, you have to tell your script what machine everything should be run on. This is done through the set_machine function. If you want to ls the $NFS_SSH_USER's home directory on the machine jenkins-tests, you could do the following:

	#!/bin/bash
	. functions
	set_machine jenkins-tests
	boot
	run_cmd "ls"
	halt

Virtual machine setup

There are a few requirements for a virtual machine to work correctly for the NFS Jenkins setup.

1. Must have NFS client running
2. Must be able to run connectathon tests
3. Must be able to run xfs tests
4. Passwordless sudo for NFS_SSH_USER user
5. Passwordless ssh configured

The sample virtual machines come preconfigured with a valid user. Log in with:

   username: jenkins
   password: jenkins

If you find you need root access:

   username: root
   password: jenkins

Be sure to change the root password before hosting virtual machines publicly! You can download sample machines here:

• 32-bit sample machine
• 64-bit sample machine

Passwordless ssh can be configured by running $NFS_WORKSPACE/scripts/init_vm or by manually starting the job "Admin - Initialize VM". This script will check if ~/.ssh/id_rsa.pub (your public key) exists. If it doesn't, then ssh-keygen will be used to generate a key. The public key will then be copied over to the virtual machine and appended to the ~/.ssh/authorized_keys file.

This process will require you to enter your password twice, once for the file transfer and once for appending to the authorized_keys file. Due to a limitation in the Jenkins software, the password must be entered in the same shell that Jenkins was initially started from.

You will also need to set the environment variable MNTOPTIONS on each virtual machine so that the connectathon tests will work properly. To do this, SSH needs to allow user environments to be set up. This can be done by editing /etc/ssh/sshd_config and setting PermitUserEnvironment to "yes". When mounting through the scripts in $NFS_WORKSPACE/scripts/, $MNTOPTIONS will be stored in /home/$NFS_SSH_USER/.ssh/environment to be sourced on future logins.

The install step will mount the server through NFS v4, so at the very least NFS v4 needs to be working on the client and server.

Archlinux and Fedora virtual machines are known to work properly.