*:  How frequently your job "naturally" checkpoints.  For instance, a particular simulation may compute a time step roughly every five minutes on an average machine.  If the simulator can be configured to write a checkpoint every ten time-steps, that gives the simulator and HTCondor ten minutes to write and then transfer the checkpoint.  (Of course, if it actually takes that long, your job will take a sixth longer, which may be too slow; see below.)  If instead each job is a batch of smaller tasks (e.g., run six different computations on the same input data), it may be most convenient to checkpoint after each computation completes.  If that takes about forty-five minutes for some and ninety for others, that's fine.  (Just because a job which writes checkpoints frequently doesn't mean it has to exit that often, but the details of writing or altering a script to do that are beyond the scope of this HOWTO.)
 *:  How frequently your job is interrupted.  For instance, if the max job lifetime on your pool is six hours, taking a checkpoint every three hours could easily result in losing almost (but not quite actually) three hours of work.
+*:  How long it takes to take and transfer checkpoints.  Measuring the first may be difficult (how do you know when the job started taking a checkpoint?), but the second, in practice, can only be done experimentally.  (HTCondor versions 8.9.1 and later record file transfer events in the user job log (the =log= submit command), including for file transfers on checkpoint, so this duration should be easy to determine.)  Unfortunately, you generally want to checkpoint less frequently (if possible) when checkpoints take longer, so as to maintain efficient job progress, the longer between intervals, the more progress the job will lose when interrupted.  The timing of interruptions isn't (in general) predictable, and (in practice) varies from pool to pool, and can also only be predicated assuming its similarity to past experiments.  The big exception to this is above: max job runtimes.
+
 
 {subsection: Debugging Checkpoints}