Ticket #2826: HGQ + CC-Limits + CLAIM_PARTITIONABLE_LEFTOVERS

test config:
########################################################
# pSlot overload
NUM_CPUS=32

SLOT_TYPE_1 = cpus=8
NUM_SLOTS_TYPE_1 = 1
SLOT_TYPE_1_PARTITIONABLE=TRUE

SLOT_TYPE_2 = cpus=8
NUM_SLOTS_TYPE_2 = 1
SLOT_TYPE_2_PARTITIONABLE=TRUE

SLOT_TYPE_3 = cpus=8
NUM_SLOTS_TYPE_3 = 1
SLOT_TYPE_3_PARTITIONABLE=TRUE

SLOT_TYPE_4 = cpus=8
NUM_SLOTS_TYPE_4 = 1
SLOT_TYPE_4_PARTITIONABLE=TRUE

########################################################
# Test for concurrency limits
XSW_LIMIT=4

########################################################
## Test for HGQ
GROUP_NAMES = group_one, group_two
GROUP_QUOTA_group_one = 8
GROUP_QUOTA_group_two = 8

GROUP_AUTOREGROUP_group_one = TRUE
GROUP_AUTOREGROUP_group_two = TRUE

Step 1: Submit job with cc-limits

executable=/bin/sleep
arguments=300
concurrency_limits = XSW
+AccountingGroup = "group_one.user"

Result of Step 1: Will hand out pSlot Claim for all 4 slots.

Step 2: Submit same job w/o cc-limits e.g. user+group == same

executable=/bin/sleep
arguments=300
+AccountingGroup = "group_one.user"

Results of step 2: Schedd uses claim to match pSlot and uses all slots blowing past HGQ

[Append remarks]

Remarks:

2012-Feb-27 14:37:01 by tstclair:
In running through the matchmaker it's pretty clear that the accountant does not use slot weight when +/-'ing limits. It seems doable to pass details into AddMatch which will update this.

Also LoadLimits will need to account for this during it's next run.

Of Note: This all happens in phase 4.1 of the negotiation cycle, for ease of tracing set : NEGOTIATOR_DEBUG = D_MATCH | D_ACCOUNTANT | D_FULLDEBUG


2012-Mar-20 13:34:32 by tannenba:
Some bold assertions:

  1. If you are using dynamic slots, it only makes sense to have accounting (including HGQ accounting) done using slot weights.
  2. As we move towards dynamic slots becoming more common, we should change so that slot accounting is no longer the default, but instead the default should be slotweight accounting were the default slotweight is simply number of cpus.

Anyone disagree?


2012-Dec-11 16:37:58 by johnkn:
Bulk change of target version from v070902 to v070903 using ./ticket-target-mover.


2013-Jan-18 09:10:44 by tstclair:
See #3435 for implementation fix.
[Append remarks]

Properties:

Type: defect           Last Change: 2013-Jan-18 09:10
Status: abandoned          Created: 2012-Feb-14 11:08
Fixed Version:            Broken Version: v070705 
Priority:          Subsystem: Daemons 
Assigned To: tstclair           Derived From: #2818
Creator: tstclair  Rust:  
Customer Group: users  Visibility: public 
Notify: tstclair@redhat.com, eje@cs.wisc.edu, tannenba@cs.wisc.edu  Due Date:  

Derived Tickets:

#3435   negotiator-side resource consumption with consumption policies