HTCondorWiki: Sub Dags Vs Splices

Page History

Note: In the process of creating this document (2016-06-13)

Note: This document is valid for HTCondor version 8.5.5.

This is a comparison of the features of external sub-DAGs and splices http://research.cs.wisc.edu/htcondor/manual/v8.5/2_10DAGMan_Applications.html#SECTION003108900000000000000

http://research.cs.wisc.edu/htcondor/manual/v8.5/2_10DAGMan_Applications.html#SECTION0031081000000000000000

Feature External sub DAGs Splices Notes

Incorporate separate DAG files yes yes

Rescue DAGs yes yes

DAGMan recovery yes yes

Multiple DAGMan instances yes no Can be good or bad

Possible combinatorial explosion of dependencies no yes Until we implement socket nodes for splices

Dynamic creation of sub-workflows yes no

PRE/POST scripts on sub-workflows yes no Until we implement socket nodes for splices

Retries of sub-workflows yes no

Workflow-wide throttling no yes

Per-sub-workflow throttling yes no

Node priorities on sub-workflows yes no

Reduce memory footprint of large workflows yes? no If used properly

Per-sub-workflow file final nodes yes no

Abort sub-workflows individually yes no

Variables associated with sub-workflows yes no

Separate configuration for sub-workflows yes no Can be good or bad

One node status file, etc., for entire workflow no yes

Feature	External sub DAGs	Splices	Notes
Incorporate separate DAG files	yes	yes
Rescue DAGs	yes	yes
DAGMan recovery	yes	yes
Multiple DAGMan instances	yes	no	Can be good or bad
Possible combinatorial explosion of dependencies	no	yes	Until we implement socket nodes for splices
Dynamic creation of sub-workflows	yes	no
PRE/POST scripts on sub-workflows	yes	no	Until we implement socket nodes for splices
Retries of sub-workflows	yes	no
Workflow-wide throttling	no	yes
Per-sub-workflow throttling	yes	no
Node priorities on sub-workflows	yes	no
Reduce memory footprint of large workflows	yes?	no	If used properly
Per-sub-workflow file final nodes	yes	no
Abort sub-workflows individually	yes	no
Variables associated with sub-workflows	yes	no
Separate configuration for sub-workflows	yes	no	Can be good or bad
One node status file, etc., for entire workflow	no	yes