Page History

Turn Off History

description

The annex daemon will be the production implementation of the condor_annex tool. (See ExperimentalCondorAnnex.) At present, it provides only the ability to provision leased AWS instances efficiently (in bulk).

The lease implementation requires an AWS Lambda function; rather than upload it every time, for efficiency the annex daemon must be provided the function's ARN. This may be automated in the future; see the installation instructions, below, for the manual process.

installation

Install the pre-release package(s) as normal.

Add the following three lines to your HTCondor configuration: 

# Turn the annex daemon on.
DAEMON_LIST = $(DAEMON_LIST) ANNEXD

# Optional: configure the default endpoints.  All three endpoints need to be the same region.
ANNEX_DEFAULT_EC2_URL = https://ec2.us-east-1.amazonaws.com
ANNEX_DEFAULT_CWE_URL = https://events.us-east-1.amazonaws.com
ANNEX_DEFAULT_LAMBDA_URL = https://lambda.us-east-1.amazonaws.com

Lambda function

The examples directory includes a CloudWatch template [FIXME] (the file template-3.json) you can use to create the necessary AWS Lambda function (and grant it the permissions necessary for it do its job). After logging into the AWS web console, do the following for each region you intend to use:

  1. Switch to the region. (Use the second drop-down box in from the upper right.)
  2. Switch to CloudFormation. (In the Services menu, under Management.)
  3. Click the "Create Stack" button.
  4. Upload the template using the "Browse..." button
  5. Click the "Next" button.
  6. Name the stack; "HTCondorLeaseImplementation" is a good name.
  7. Click the "Next" button. (You don't to change anything on the options screen.)
  8. Check the box next to "I acknowledge" (down near the bottom) and click the "Create" button (where the "Next" button was).
  9. AWS should return the list of stacks; select the one you just created and select the "outputs" tab.
  10. Copy the long string labelled "LeaseFunctionARN"; you'll need it to for each invocation of condor_annex. It may take some time for that string to appear (you may need to reload, as well.) Wait the stack to enter the 'CREATE_COMPLETE' state before using the LeaseFunctionARN (see below).

usage

Attachments: