Page History
- 2012-Apr-02 18:04 wenger
- 2012-Apr-02 17:43 wenger
- 2012-Apr-02 16:58 wenger
- 2012-Apr-02 16:55 wenger
- 2012-Apr-02 16:45 wenger
- 2012-Apr-02 16:38 wenger
- 2012-Apr-02 16:26 wenger
- 2012-Apr-02 16:22 wenger
- 2012-Apr-02 16:20 wenger
- 2012-Apr-02 15:40 wenger
- 2012-Mar-18 18:34 wenger
- 2012-Feb-19 22:19 wenger
- 2012-Feb-19 20:39 wenger
The way this is handled in the code is probably not as clean as it could be, because PRE and POST scripts got added to DAGMan after the initial code was written. Because of that, I guess, PRE scripts are handled outside of the ready queue data structure, which is kind of awkward.
There are a number of important data structures relating to job ordering:
Dag::_jobs: a list of all Job objects (nodes) in the DAG.Dag::_readyQ: a queue of all jobs that are ready to be submitted (all of their parent nodes have finished, and their PRE scripts, if any, have also finished).Job::_queues[Q_WAITING]: a list of Job objects (nodes) that this Job is waiting on (Jobs are removed from this queue as they finish). (The Job object also has parents and children queues.)
When DAGMan starts up, the ready queue is empty. Dag::Bootstrap() calls Dag::StartNode() on all jobs with empty waiting queues. Dag::StartNode() either runs the node's PRE script (if there is one) or puts the node into the ready queue.
condor_event_timer() in dagman_main.cpp gets called every five (by default) seconds. In that function, we call Dag::SubmitReadyJobs() to submit any jobs that are ready; ready any new node job events (see ???); output the status of the DAG; and check whether the DAG is finished.
- Submission/event reading loop -- every 5 sec (default) via daemoncore
- ready queue -- Job objects go into it when ready to submit
- pre scripts are handled kind of kludgily -- might make more sense for Job objects to go into the ready queue when the PRE script should be submitted
- document list of parents, not done parents in Job objects
- Jobs have lists of parents, pending parents, etc.
- Each job has parents, waiting, and children lists (parents and children lists don't change during run; waiting does)
- submit failure puts job back into ready queue
- pre scripts are handled separately from ready queue -- kind of goofs things up...
- maybe explain how things work w/o pre/post scripts, then add in that info