Page History

Turn Off History

Is the idea that you want to require all the nodes of a 4-way or 8-way parallel universe job to run on the same physical machine (i.e. to be certain a parallel job never has to communicate over the network) ? If so, one easy option may be to simply create a unique PARALLEL_SCHEDULING_GROUP for each machine, e.g. append in condor_config: 
  ParallelSchedulingGroup = "$(FULL_HOSTNAME)"
  STARTD_ATTRS = $(STARTD_ATTRS), ParallelSchedulingGroup
and then add to your submit file: 
+WantParallelSchedulingGroups = True
For more info on parallel scheduling groups, see section 3.13.10.4, Condor's Dedicated Scheduling section in the Condor Manual.

If you don't wish to require packing all nodes of a parallel job onto the same physical machine, you could tell Condor to attempt to use the least number of physical machines to run a parallel job as follows: 

##  The NEGOTIATOR_PRE_JOB_RANK expression overrides all other ranks
##  that are used to pick a match from the set of possibilities.
##  The following expression matches jobs to unclaimed slots
##  whenever possible, regardless of the job-supplied rank.
##  For all slots, prefer to give out the highest MachineId
##  available to parallel universe jobs in order to try to pack parallel
##  jobs onto the the minimum number of physical machines.
NEGOTIATOR_PRE_JOB_RANK =(1000000*RemoteOwner =?= UNDEFINED) \
                          + ( (TARGET.JobUniverse=?=11)*MachineId )

##  Now, if the job is a parallel universe job, tell condor_submit to
##  append a job rank expression in the job classads that prefers to use
##  the highest MachineId available. Note that for parallel universe
##  jobs it is not enough to set NEGOTIATOR ranking alone to use
##  the highest MachineId available, we must also put it into the job
##  classad itself because the dedicated scheduler does its own
##  re-sorting of jobs to machine mappings after it gets its matches
##  from the negotiator.
APPEND_RANK = (JobUniverse=?=11)*MachineId

The above example assumes you have a global unique ordering of your physical machines as an integer in machine ad attribute MachineId. If you cannot easily add such a global ordering, perhaps you could generate one via a classad regex on the IP address or some such. I suppose the above examples could be improved by also having non-parallel universe jobs prefer the lowest numbered MachineId machines.

BTW, if you come up with a policy that combines both parallel scheduling groups AND job ranks, be aware - I don't think that using both of these mechanisms at the same time worked until Condor v7.5.6.