batch_test.pl knows how to create a new test personal condor from the binaries in your path using slightly modified generic configuration files. It creates the directory condor_tests/TestingPersonalCondor and sets up to run there. Sometimes to restart tests you may need to remove this directory so it knows to restart the personal condor. -batchtest.pl runs tests serially one after another. +batchtest.pl runs tests serially one after another by default. **Variations within batch_test.pl** @@ -37,31 +37,61 @@ ./batch_test.pl -b -The above run is setting up a test environment in -condor_tests/TestingPersonalCondor running the quick class: +To run tests concurrently you specify how many to try to keep +running as it does each directory. Do all the tests and when a directory +is being being processes start 20 if they are there and maintain that level +until tests are all running. -./batch_test.pl -f list_quick ?b +./batch_test.pl -e 20 -b + +To set up a test environment in +condor_tests/TestingPersonalCondor and run the quick class: + +./batch_test.pl -f list_quick -b batch_test.pl takes multiple arguments: --d <directory> Run the tests in this directory, ie "-d gcc/" --s <filename> Skip all the tests listed in <filename> --f <filename> Run all of the tests listed in <filename> --a <count> Run each test this many times --t <test> Run just this test --pretest Just set up the testing personal condor + -b <build&test> Use generic config files from condor examples + and start a personal condor. + -d <directory> Run the tests in this directory, ie "-d gcc/" + -s <filename> Skip all the tests listed in <filename> + -f <filename> Run all of the tests listed in <filename> + -a <count> Run each test this many times + -t <test> Run just this test + -p <pretest> Just set up the testing personal condor + -e <evenly> Run the tests concurrently + -w <wrap> Wrap each test in a personal condor or with -t just one + +Run all the top level tests( standard way for windows ) with +a pristine personal condor. -Run all the top level tests -./batch_test.pl -d . +./batch_test.pl -d . -b + +Run just one test and then do the same but run it five times. -Run just one test and then do the same but run it twice ./batch_test.pl -d . -t job_condorc_ab_van -./batch_test.pl -d . -t job_condorc_ab_van -a 2 +./batch_test.pl -d . -t job_condorc_ab_van -a 5 The files that -s and -f are one test per line of things to run or skip. **Simple tests vs wrapped tests** +We wrap tests because we want to change the environment(config settings) used while this test is running without impacting the other tests which are either running now or will run yet in the main personal condor. It also ensures that +there will be a log directory showing only what is going on during this test. +With many tests running in the starting personal condor it can be hard to see what is going on. There are two ways to get a test wrapped: + +1: Run test with batch_test.pl -d . -w -t job_core_output_van -b +2: Write the test to start a personal condor and then run its test + +The following files are the most basic test which is wrapped and can be run ./batch_test.pl -d . -t wrapped_test -b. See "Adding a test" below for help in writing the actual test. Below are the pieces: + +*: wrapped_test.cmd - Submit file for the test +*: wrapped_test.run - Script run to perform the test +*: x_wrapped_test_locconfigpostsrc - The items you want to be at the end of the local config file +*: wrapped_test.pl - The core test itself with call to RunTest +*: x_param.wrapped_test - Parameter file to control personal condor and which daemons to run + + **Concurrent Testing** 1: Under V7_2-branch ./batch_test.pl -e 30 -b means try to keep thirty running at once if there are that many in the current directory(or left to do). @@ -207,11 +237,11 @@ CondorTest::RunTest ultimately submits the job to Condor, and blocks until the job has left the queue. The arguments are the name of the test, the submit file to use, and whether or not Condor should try and force a checkpoint. If you can't use ckpt_and_exit, but you need to checkpoint for your test to be valid, set this flag to be 1. However, realize that your job will need to run for "long enough" to make sure that: -1. The Condor.pm module sees that your job is executing -2. It has a chance to send the 'condor_vacate' to the remote machine -3. The remote machine has a chance to get the 'condor_vacate' command, and +1: The Condor.pm module sees that your job is executing +2: It has a chance to send the 'condor_vacate' to the remote machine +3: The remote machine has a chance to get the 'condor_vacate' command, and decide that you're authorized to send a 'condor_vacate' to it -4. The remote machine has a chance to actually send the SIGUSR2 to the job +4: The remote machine has a chance to actually send the SIGUSR2 to the job Oftentimes, the job will finish before it successfully checkpoints. This sort of testing in a distributed system is hard. @@ -230,4 +260,4 @@ *: Bring down sources for condor_examples, condor_scripts and condor_tests to c:\ *: Cp *.pm and batch_test.pl from condor_scripts to condor_tests *: Cd to condor_tests -*: ./batch_test.pl --no-error -d . -b +*: ./batch_test.pl -d . -b
