Page History

Turn Off History

Using HTCondor Annex for the First Time

We assume you already have an AWS account, as well as a log-in account on a Linux machine with a public IP address and an administrator who's willing to open a port for you. If so, you can follow the instructions for UsingCondorAnnexForTheFirstTime.

What You'll Need to Know

To create a HTCondor annex with on-demand instances, you'll need to figure out four things:

  1. A name for it. "MyFirstAnnex" is a fine name for your first annex.
  2. Which instance type you want to use. Each type of instance has a different number (and type) of CPU cores, amount of RAM, size of local storage, and the like. If you're not sure, we recommend starting with 'm4.large', which has 2 CPU cores and 8 GiB of RAM.
  3. How many instances you want. For your first annex, when you're checking to make sure things work, you may only want one instance.
  4. The duration of the annex, expressed as decimal number of hours. By default, this is 0.83, or 50 minutes. condor_annex will arrange for your annex's instances to be terminated after its duration has passed. Once it's in place, this lease doesn't depend on your machine, but it's only checked every five minutes, so give your deadlines a lot of cushion to make you don't get charged for an extra hour.

Start an Annex

If you agree with the suggestions above, entering the following will start an annex. 

$ condor_annex -duration 0.83 -count 1 -annex-name MyFirstAnnex -odi-instance-type m4.large
MyFirstAnnex[something-something-something]

This command will return after HTCondor has set up the 3000-second (50 minute) lease and requested that Amazon start 1 instance. The output can be used on AWS to identify annex instances; you won't have to do that unless something goes wrong.

The sharp-eyed among you may have spotted m4.large on both the command line and the in the HTCondor configuration; specifying -odi-instance-type is in fact optional, and condor_annex will use the configured default if you don't specify one on the command-line.

Likewise, if fifty minutes is long enough for you, you don't need to use -duration, either.

Monitor your Annex

You can find out if that instance has successfully joined the pool in the following way. 

$ condor_status -annex MyFirstAnnex
slot1@ip-172-31-48-84.ec2.internal  LINUX     X86_64 Unclaimed Idle 0.640 3767
slot2@ip-172-31-48-84.ec2.internal  LINUX     X86_64 Unclaimed Idle 0.640 3767

              Total Owner Claimed Unclaimed Matched Preempting Backfill  Drain
 X86_64/LINUX     2     0       0         2       0          0        0      0
        Total     2     0       0         2       0          0        0      0

This example shows that the annex instance you requested has joined your pool. (The default annex image configures one static slot for each CPU it finds on start-up.)

Stop an Annex

The following command shuts HTCondor off on each instance in the annex; if you're using the default annex image, doing so causes each instance to shut itself down. 

$ condor_off -master -annex MyFirstAnnex
Sent "Kill-Daemon" command for "master" to master ip-172-31-48-84.ec2.internal

Advanced Usage

The information is this section is for advanced users and may not apply (or make sense) to everyone.

Configure the Idle Time

By default, annex instances will shut themselves off it they've been idle (that is, in the HTCondor unclaimed state) for more than fifteen minutes. If you'd like to change that value, pass condor_annex the -idle flag, which takes a decimal number of hours. For instance, the default is equivalent to condor_annex -idle 0.25.

Configure the Annex

You can customize the configuration of your annex. If you pass the full path to a directory (for example, /home/annex/config.d) to condor_annex using the -config-dir option, condor_annex will copy the files in that directory to the HTCondor config directory on each annex instance. This does not replace the customization that condor_annex is already doing to configure security and tell the annex instances which pool to join; those changes will be laid down on top of (a temporary copy of) the directory you specified before being transferred to the instances.