Page History

Turn Off History

This page describes a configuration useful primarily for virtual machines which should be shut down when they stop being useful (doing work).

We start by configuring the startd to shut down if it hasn't been claimed for five minutes: 

STARTD_NOCLAIM_SHUTDOWN = 300.

We then configure the master to shut down when the startd shuts down. Note that we give the startd sixty seconds to start up the first time; this time-out would be unnecessary if STARTD_StartTime were guaranteed to be unset rather than zero before the startd starts up. 

MASTER.DAEMON_SHUTDOWN_FAST = ( STARTD_StartTime == 0 ) && ((CurrentTime - DaemonStartTime) > 60)

Then, when the master shuts down, we want to run a script to shut down the machine. All that this configuration knob does is define the "SHUTDOWN" name; you can pick another. 

MASTER_SHUTDOWN_SHUTDOWN = /etc/condor/shutdown

To activate the shutdown script named "SHUTDOWN", use the condor_set_shutdown command with the -exec option. This command MUST be run while the master is alive. A shell script like the following may help resolve timing issues; be sure that the user you run it as is privileged (probably root). 

while ! condor_set_shutdown -exec shutdown; do sleep 1; done

Because of the sleeps and the retries, you can run that shell script fragment immediately after starting (or restarting) the master and not worry about the master not being ready to accept commands.

The script (/etc/condor/shutdown in this example) must of course be executable (and have the #! line), and is run by the master as root when the master shuts down. The following script may be useful as an example to work from. 

#!/bin/sh

# We start out trying to determine if this shutdown was voluntary or not.
# In this example, we don't do anything useful with this information, so
# you can skip it if you like.
#
# I'm not sure if /sbin/runlevel is available on systemd systems, so you
# check to make sure that this is normally nonzero and zero when shutting down.
SHUTDOWN=`/sbin/runlevel | /usr/bin/awk '{print $2}'`
SHUTDOWN_MESSAGE='because instance is being terminated'
if [ ${SHUTDOWN} -ne 0 ]; then
    SHUTDOWN_MESSAGE='for lack of work'
fi

# The following line is probably AWS-specific; you can omit it and the
# ${INSTANCE_ID} from the following line entirely.
INSTANCE_ID=$(/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id)
MESSAGE="$(/bin/date) Instance ${INSTANCE_ID} shutting down ${SHUTDOWN_MESSAGE}."

# For testing.  You could do something cleverer here, if you'd like.
echo ${MESSAGE}

# Shut the machine down.  Comment this line out if you're just testing,
# of course. :)
/sbin/shutdown -h now