HTCondorWiki: Continuous Build And Tests In Nmi

Page History

The results of the per-commit builds can be seen here.

Overview

We build Condor and run the regression test suite on each push to the Git repository. The reasons for these frequent builds are:

Provide quick feedback if a commit breaks the builds or tests
Provide historical coverage to help trace back and find which commit broke tests/builds. It is hard to track backwards from a broken test to the commit which broke the test if there are many commits between builds. The more frequent the tests are, the smaller the commit windows is to find the guilty commit.
Help spot race conditions in tests that fail sporadically, especially when subsequent runs of the test produce different results against the same code

Current implementation

The per-commit runs are submitted from submit-2.batlab.org in the /home/condorauto/per-commit directory. A single Crondor job exists which wakes up every few minutes and checks for new commits to build. This job simply calls condor_nmi_submit to do the dirty work. Occasionally, someone will condor_rm all of condorauto's jobs, forgetting about this Crondor job. When that happens, the Crondor job will get removed too, stopping the per-commit builds from running (but don't worry - it won't lose state) and someone needs to re-submit the job (see below for instructions).

Note that condor_nmi_submit sets the JobPrio such that the most recently submitted build (and test) has the highest priority. Because tests are slow, they usually fall behind in the course of a day, but the most recently submitted ones are started first. Even if it takes overnight for a test run to finish, it can still be very valuable when tracking backwards to find what commit broke a test.

Setting up the per-commit builds

The per-commit build "code" is in Subversion at /p/condor/software/svn/condor/per-commit-builds

Follow the README file found in that directory.

Re-submitting the per-commit builds

Login to submit-2.batlab.org and sudo to the condorauto user
cd /home/condorauto/per-commit
condor_submit submit

Per-commit Native Package Repositories

...

The Build and Test Dashboard

Build and Test dashboard

The code is in PHP. It is available in the same SVN repository as the per-commit build scripts: /p/condor/software/svn/build-dashboard/

Currently it is made available on submit-2.batlab.org by placing it in /home/condorauto/public/html/results and placing a symlink in /var/www/html

[kronenfe@submit-2 html]$ ll /var/www/html
...
lrwxrwxrwx 1 root         root            36 Sep  8  2011 results -> /home/condorauto/public/html/results
lrwxrwxrwx 1 root         root            42 Sep 22  2011 results-devel -> /home/condorauto/public/html/results-devel
...