The Imake rules know on each platform what compilers exist. For each compiler it
finds on a platform, it creates a new directory underneath condor_tests that is the name of that compiler, and creates basically a symlink and copy image of everything else in condor_tests for that compiler. Then it goes through each compiler subdirectory and builds each test with that compiler, usually with the syscall library and a vanilla version of that test. Last it does any building of non-Standard Universe test in the top level.
-To actually run the tests, you start out in condor_tests and run
-./batch_test.pl -b
-
-batch_test.pl by default assumes you want to test with daemons found in your path and assumes they are currently running. This allows having a daemon in a debugger during the testing if needed.
+batch_test.pl by default assumes you want to test with daemons found in your path and assumes they are currently running with your current configurations. This allows having a daemon in a debugger during the testing if needed.
batch_test.pl knows how to create a new test personal condor from the binaries in your path using slightly modified generic configuration files. It creates the
directory condor_tests/TestingPersonalCondor and sets up to run there. Sometimes to restart tests you may need to remove this directory so it knows to restart the personal condor.
batchtest.pl runs tests serially one after another.
+**Variations within batch_test.pl**
+
To run the entire test suite with your existing daemons:
./batch_test.pl
@@ -59,6 +58,14 @@
The files that -s and -f are one test per line of things to run or skip.
+**Simple tests vs wrapped tests**
+
+**Concurrent Testing**
+
+**Core file and ERROR detection**
+
+**General expectations**
+
"Tests" are perl scripts that end in .run. batch_test.pl takes the list of
tests it's supposed to run, forks them off. It sets a timer of 18 hours
that the test must complete in or it will automatically exit. stdout of the
@@ -72,26 +79,26 @@
if they passed or succeeded. It looks like this:
-submitting gcc tests.......
-no_supp succeeded
-loop succeeded
-bigenv-scheduler succeeded
-syscalltester succeeded
-coredump succeeded
-printer succeeded
-big succeeded
-
-submitting g++ tests........
-no_supp succeeded
-loop succeeded
-bigenv-scheduler succeeded
-lseek (job never checkpointed)
-syscalltester succeeded
-coredump succeeded
-printer succeeded
-big succeeded
+ submitting gcc tests.......
+ no_supp succeeded
+ loop succeeded
+ bigenv-scheduler succeeded
+ syscalltester succeeded
+ coredump succeeded
+ printer succeeded
+ big succeeded
+
+ submitting g++ tests........
+ no_supp succeeded
+ loop succeeded
+ bigenv-scheduler succeeded
+ lseek (job never checkpointed)
+ syscalltester succeeded
+ coredump succeeded
+ printer succeeded
+ big succeeded
-14 successful, 1 failed
+ 14 successful, 1 failed
The exit status is the number of tests that failed. If everything passed,
@@ -108,9 +115,9 @@
READ the README files! The naming is supposed to be concise by rules in them. This helps everyone understand what the test is doing. Additionally understand classes! The default class run by batch_test.pl is list_quick. On windows the default list of tests run comes from the file Windows_list.
To add a new test to the test suite, you have to do three steps:
-1. Write the code for the test program itself
-2. Decide how your test should be built
-3. Write a .run file that can run your test and collect the output.
+1: Write the code for the test program itself
+2: Decide how your test should be built
+3: Write a .run file that can run your test and collect the output.
Actually writing your test code is very easy. The first thing to think about is
what are you trying to test. Most of the compiler directory tests in the current
@@ -140,35 +147,37 @@
interface to submitting Condor jobs and receiving callbacks when events happen. (It monitors the UserLog file the job submit file specifies to discover this
information.) CondorTest.pm adds several new functions on top of Condor.pm, mostly for managing expected output. Again, a .run file is just a Perl script, and as such does not have to use the Condor perl modules or even use Condor at all.
+**Sample Test**
+
Here is an example .run file, for dirtest.c - dirtest.c is designed to check that the dirent() functions still work, so it creates a couple of directories and looks at them:
-#!/usr/bin/env perl
+ #!/usr/bin/env perl
-use CondorTest;
+ use CondorTest;
-$cmd = 'dirtest.cmd';
-$testname = 'getdirentries Test';
+ $cmd = 'dirtest.cmd';
+ $testname = 'getdirentries Test';
-@CondorTest::expected_output = ( 'ENTRY: .',
-'ENTRY: ..',
-'ENTRY: first',
-'ENTRY: second',
-'ENTRY: third',
-'completed successfully' );
+ @CondorTest::expected_output = ( 'ENTRY: .',
+ 'ENTRY: ..',
+ 'ENTRY: first',
+ 'ENTRY: second',
+ 'ENTRY: third',
+ 'completed successfully' );
-CondorTest::RegisterExitedSuccess( $testname,
+ CondorTest::RegisterExitedSuccess( $testname,
\&CondorTest::DefaultOutputTest );
-if( CondorTest::RunTest($testname, $cmd, 0) ) {
+ if( CondorTest::RunTest($testname, $cmd, 0) ) {
print "$testname: SUCCESS\n";
exit(0);
-} else {
+ } else {
die "$testname: CondorTest::RunTest() failed\n";
-}
+ }
-A quick walkthrough:
+**Sample test walkthrough**
The first thing to notice is the @CondorTest::expected_output array - this is the text that dirtest.c will output, and that we should be looking for. If there are lines that your test will produce that aren't invariant (say you print out the time of day at the beginning of your test) you can register certain output lines as "lines to be skipped":
@@ -196,7 +205,8 @@
Oftentimes, the job will finish before it successfully checkpoints. This sort of
testing in a distributed system is hard.
-Added tidbits:
+**Added tidbits:**
+
There are many examples to look at in src/condor_tests/Imakefile. We also have a
perl module called CondorPersonal.pm which allows us to setup tests which require a modified environment so as to not change the environment of the personal condor running all of the tests. It allows us to emulate most pool configurations. Look at the job_flocking_to, job_condorc_ab_van and cmd_status_shows-avail for more complicated examples.
@@ -204,10 +214,9 @@
*: Install windows msi
*: Add path to condor binaries(bin and sbin) to PATH env variable
-*: Add NMI_PLATFORM = x86_winnt_5.1 env variable
*: Install cygwin with cygwin perl
*: Install Active perl
-*: Bring down sources and move condor_examples, condor_scripts and condor_tests to c:\
+*: Bring down sources for condor_examples, condor_scripts and condor_tests to c:\
*: Cp *.pm and batch_test.pl from condor_scripts to condor_tests
*: Cd to condor_tests
*: ./batch_test.pl --no-error -d . -b
