**How to match only Multicore jobs in recently drained slots** +Known to work with HTCondor version: 8.2.8 + Since the purpose of the defrag daemon is to drain jobs on a p-slot so multi-core jobs can begin to match, it would be best to implement a policy where recently drained p-slots can *insist* on matching only multicore jobs for a period of time. Unfortunately, there is no attribute that uniquely identifies a recently drained slot - but there are two candidate attributes that come close with some caveats. @@ -11,31 +13,29 @@ *This is the preferred policy - use this policy if you use the defrag daemon* {code} - # This knob must be set in the negotiator's configuration. it prevents - # slots from going into matched state, which is works fine and is necessary + # This knob must be set in the negotiator's configuration. It prevents + # slots from going into matched state, which works fine and is necessary # to make the $(StateTimer) trigger only when we leave draining state. NEGOTIATOR_INFORM_STARTD = false - # A job counts as multicore when it requests 4 or more core, or when - # it has already matched to a slot that has less than 4 cores. + # Set OnlyMulticoreInterval to the number of seconds the startd should + # only allow multi-core jobs. Default to about 2 negotiation cycles. +OnlyMulticoreInterval = 2*$(NEGOTIATOR_INTERVAL:300) + # If this slot has more than 4 cores free, then insist on jobs that + # request at least 4 core or more. IsMulticore = RequestCpus >= IfThenElse(Cpus<4,1,4) - # We want to insist on multicore jobs for about 2 negotiation cycles -OnlyMulticoreInterval = 2*$(NEGOTIATOR_INTERVAL:60) # This is already defined in the default config, included for here for clarity StateTimer = (time() - EnteredCurrentState) - # Ignore the policy for non p-slots + # We only want this policy to apply to partitionable slots (pslots) IsntUnmatchedPSlot = PartitionableSlot=!=true || State=="Matched" -OnlyMulticoreJobsAfterDrain = $(IsntUnmatchedPSlot) || $(IsMulticore) || $(StateTimer) > $INT(OnlyMulticoreInterval) +OnlyMulticoreJobsAfterDrain = $(IsntUnmatchedPSlot) || $(IsMulticore) || $(StateTimer) > $(OnlyMulticoreInterval) START = $(START) && ( $(OnlyMulticoreJobsAfterDrain) ) {endcode} The policy says that for 2 negotiation cycles after the STARTD starts up or leaves draining state, the slot should only match jobs that want at least 4 cores. Once the slot has less than 4 available Cpus remaining, it will match single-core jobs. We only want this portion of the START expression to be evaluated by the Negotiator, which is what the $(IsntUnmatchedPSlot) sub-expression does. -Note for 8.2 and earlier versions: change =$INT(OnlyMulticoreInterval)= to =$(OnlyMulticoreInterval)= - - *Alternate policy that triggers whenever the p-slot fully drains for any reason - but doesn't work at all if draining is only partial* This policy using =ExpectedMachineGracefulDrainingCompletion= triggers the preference for multi-core automatically whenever a p-slot has no child d-slots, but, it reverts back to matching any job as soon as there are child d-slots. @@ -50,7 +50,7 @@ # note that if the result is < 0, draining is in the future. DrainStateTimer = (time() - ExpectedMachineGracefulDrainingCompletion) -OnlyMulticoreJobsAfterDrain = PartitionableSlot=!=true || $(IsMulticore) || $(DrainStateTimer) > $INT(OnlyMulticoreInterval) || $(DrainStateTimer) < 0 +OnlyMulticoreJobsAfterDrain = PartitionableSlot=!=true || $(IsMulticore) || $(DrainStateTimer) > $(OnlyMulticoreInterval) || $(DrainStateTimer) < 0 START = $(START) && ( $(OnlyMulticoreJobsAfterDrain) ) {endcode}