*: siblings having received remainder least recently are favored in round robin - siblings are ordered by time of last receipt of a remainder
*: remainder unused at a level is sent up to parent
-The following steps are iterated GROUP_QUOTA_MAX_ALLOCATION_ROUNDS time, default of 3:
+{subsubsection: Allocation Rounds}
-For each node in the graph, we assume that all running jobs associated with that group
-will stay running, and any idle jobs will match, and if this sum is more than the
-quota, we declare that number the surplus, and distribute the surplus bottom up to groups
-that are configured accept surplus. ???
-hen, we sort the submitters in "starvation order", by GROUP_SORT_EXPR, defaults to the ratio
-of resources used to the quota (either weighted)?
+Allocation rounds are a method to address the scenario where jobs submitted under an accounting group do not satisfy mutual job/slot requirements for enough slots to achieve their quota. When GROUP_QUOTA_MAX_ALLOCATION_ROUNDS > 1, then each group that has not met its allocated quota has its 'requested' value re-set to be equal to whatever its current (weighted) usage is. (i.e. it is assumed that no further jobs under that group will match slots until next negotiation cycle). This frees up the unused quota for other groups that may be able to use it as surplus.
+
+The following steps are iterated GROUP_QUOTA_MAX_ALLOCATION_ROUNDS times:
+1: (starting after 1st round) re-set 'requested' values to current usage
+2: (re)compute quota allocations
+3: allow all groups to renegotiate
+
+{subsubsection: Round Robin Rate}
+Round robin rate is a method to address the 'overlapping effective pool' problem: this is a scenario where the jobs in two or more accounting groups are in fact competing for a subset of the total available resources. For example, if a pool has 100 linux machines and 100 windows machines, and 200 jobs from 2 accounting groups are competing only for the linux machines. Without intervention, the first group to negotiate can acquire all 100 linux machines and starve the 2nd group.
+
+{subsubsection: accounting group negotiation order}
+we sort the submitters in "starvation order", by GROUP_SORT_EXPR, defaults to the ratio of current group usage / configured group quota
Finally, we negotiate with each group in that order, with a quota limited as calculated above.
{subsection: Questions}
-How common is it to have demand (submitters) in interior nodes?
-What about non-homogenous pools?
-Is there a way to do this without relying on the submitter ad's # of idle/running jobs?
-How should this behave in the face of flocking?
-Weighted slots?
+*: How common is it to have demand (submitters) in interior nodes?
+*:: some downstream customers are known to be interested in jobs submitted against interior nodes
+*: What about non-homogenous pools?
+*: Is there a way to do this without relying on the submitter ad's # of idle/running jobs?
+*:: There may be alternative approaches to surplus-sharing to address this behavior, but it is an open question
+*: How should this behave in the face of flocking?
+*: Weighted slots?
+*:: I have a few thoughts on how weighted slots should be thought about here:
+*::: http://erikerlandson.github.com/blog/2012/11/15/rethinking-the-semantics-of-group-quotas-and-slot-weights-claim-capacity-model/
+*:: and here: https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=3435
