Page History

Turn Off History

You might want to submit a job repeatedly, varying the arguments for each run. The core technique is to queue multiple jobs in a single submit file, using submit macros and ClassAd expressions to calculate the argument.

Monotonically increasing count

You can use $(Process) in your submit file to get a monotonically increasing count. This is also useful if you need an arbitrary unique identifier. For example, to submit 10 jobs, where the first job will get the argument "0", the second job "1", and so on, you could use: 

executable = myprogram.exe
arguments  = $(Process)
queue 10

As of HTCondor 8.3.5 you can do simple math in the submit file using $INT() or $REAL(). The technique is to use a temporary variable to declare the mathematical expressions, and then refer to the temporary variable using the $INT() or $REAL() macro expansion to evaluate and print the result. The $INT() and $REAL() expansions take an optional argument to control the formatting of the result. 

executable = myprogram.exe
tmp1 = $(Process) + 1
tmp2 = $(Process) * 2.0
arguments = $INT(tmp1) $REAL(tmp2,%.3f)
queue 10

Prior to 8.3.5 you can do math by using the deferred expansion of $$([]), in which case you'll use $(ProcId) instead of $(Process). For example, this passes two arguments, the first starts a 1 and increases by 1 for each job, the second starts at 0 and increases by 2 for each job: 

executable = myprogram.exe
arguments  = $$([ProcId+1]) $$([ProcId*2])
queue 10

Selection from a list

In versions of HTCondor starting with 8.3.5, you can use the Process variable to select an item from a list using the $CHOICE() macro.

In this example, there will be 6 jobs submitted, the first will have arguments=Alan, the second will have arguments=Betty, etc. 

executable = myprogram.exe
arguments = $CHOICE(Process, Alan, Betty, Claire, Dan, Eva, Frank)
queue 6

The list can also be a submit variable like this. This example produces the same resulting jobs as the above example. 

executable = myprogram.exe
arguments = $CHOICE(Process, Names)
Names = Alan, Betty, Claire, John, Harry, Sally
queue 6

Random selection

If you want each job to have a randomly assigned argument, you can use $RANDOM_CHOICE. The random selection is not guaranteed to be cryptographically strong, nor necessarily suitable for scientific use.

In this example, HTCondor assigns each job two random choices; an integer from 1 to 3, and a random name: 

executable = myprogram.exe
arguments  = $RANDOM_CHOICE(1,2,3) $RANDOM_CHOICE(Alan, Betty, Claire)
queue 10

Permute a list (pre 8.3.5 method)

Prior to 8.3.5 If you want to permute some other element, you can use $$([ ]) to evalute arbitrary ClassAd expressions. This is particularly useful combined with lists. If you wanted to permute the 3 names, you could do this: 

executable = myprogram.exe
arguments  = $$([names[ProcId])
+names     = {"Alan", "Betty", "Claire"}
queue 3

Note that if you try to index into a non-existant list element, HTCondor will be unable to determine the final argument string and the job will be put on hold. If you want to loop over the list, you can use modulus arthimatic and the list of the list: 

executable = myprogram.exe
arguments  = $$([names[ProcId%size(names)]])
+names     = {"Alan", "Betty", "Claire"}
queue 50