Ticket #2428: allow macros in dagman log files to assist templating
Mentioned on condor-users:
Universe = standard
requirements = (Arch == "X86_64")&& (OpSys == "LINUX")
Executable = /path/to/binary
transfer_executable = TRUE
should_transfer_files = IF_NEEDED
when_to_transfer_output = ON_EXIT
Arguments = $(param)
Output = $(id).stdout
Error = $(id).stderr
Log = $(id).condor_log
JOB RT000-rt0.01 itersave.condor_job
VARS RT000-rt0.01 param="000 GT"
VARS RT000-rt0.01 id="RT000-rt0.01"
RETRY RT000-rt0.01 3
JOB RT001-rt0.01 itersave.condor_job
VARS RT001-rt0.01 param="001 GT"
VARS RT001-rt0.01 id="RT001-rt0.01"
RETRY RT001-rt0.01 3
MultiLogFiles: macros ('$(...') not allowed in log file name
($(id).condor_log) in DAG node submit files
DAGMan strictly protects against possibly changing UserLog files because of the presence of macros. This covers $(Cluster) and $(Process). However, it also covers against VARS ID=ABC, as in this example, a macro that is not variable and instead allows for template submit files.
Proposal is to more narrowly disallow macros (nix Process/Cluster/anything-that-evals-at-submit-time-with-Process-or-Cluster-or-Time-etc) or give users who know what they are doing an escape hatch, a way to say "yes, i'm using macros properly, trust me"
2011-Nov-17 12:56:19 by wenger:
I'm marking this as stalled because if we actually implement #2645, this ticket will become irrelevant.
2011-Nov-21 16:20:02 by wenger:
Deferring this because of the ideas in #2645 -- if we implement some of those things, DAGMan may no longer have to parse the log file names.
2012-Oct-04 14:23:07 by wenger:
This is pretty much no longer an issue because of #2807 being done. I'm going to leave it unresolved for now, but move it down to priority 5. I can probably be abandoned once the new "one log file" feature is widely used.
Macro in node job log file name should be fatal error|
Check-in : edits of version history items and defn of knob DAGMAN_ALWAYS_USE_NODE_LOG ===GT=== #2428 ===GT=== #2807 (By Karen Miller )|
Check-in : Let submit use recursive macros ===GT=== #2807 This has been suggested a lot. This is a partial solution to #406 and #2428 (By Nathan W. Panike )|
Check-in : Use new macro capability in DAGman DAGman now requests the schedd and shadow to write each event to the default log, i.e., <DAG file>.nodes.log, along with whatever log is specified by the user. [...]
(By Nathan W. Panike )|
Check-in : Working on gittrac #2428: cleaned up the VARS code in preparation for allowing macros in the log file specifications; also started on the first test for this. (By Kent Wenger )|