sys.exit(77)
{endfile}
2: HTCondor recognizes the unique exit code and does file transfer; this file transfer presently behaves as it would if the job were being evicted. Thus, your job must set =when_to_transfer_output= to =ON_EXIT_OR_EVICT=.
-3: After the job's =transfer_output_files= are successfully sent to the submit node (and are stored in the schedd's spool, as normal for file transfer on eviction), HTCondor restarts the job exactly as it started it the first time.
+3: After the job's =transfer_output_files= are successfully sent to the submit node (and are stored in the schedd's spool, as normal for file transfer on eviction), HTCondor restarts the job on the same machine in exactly the same way as it started it the first time. The job does not become idle -- it stays in the running state while the checkpoint is transferred.
4: If something interrupts the job, HTCondor reschedules it as normal. As for any job with =ON_EXIT_OR_EVICT= set, HTCondor will restart the job with the files stored in the schedd's spool.
By storing the job's checkpoint on the submit node as soon as it is created, this method allows the job to resume even if the execute node becomes unavailable. Transferring the checkpoint before the job is interrupted is also more reliable, not just because not all interruptions permit file transfer, but because job lifetimes are typically much longer than eviction deadlines, so slower transfers are much more likely to complete.
