Page History
- 2017-Jan-04 14:48 tlmiller
- 2017-Jan-04 14:08 tlmiller
- 2017-Jan-04 14:01 tlmiller
- 2017-Jan-04 13:56 tlmiller
- 2017-Jan-04 13:46 tlmiller
- 2016-Dec-30 15:08 tlmiller
- 2016-Dec-30 15:00 tlmiller
- 2016-Dec-30 14:57 tlmiller
- 2016-Dec-30 14:50 tlmiller
- 2016-Dec-30 14:46 tlmiller
- 2016-Dec-30 12:33 tlmiller
- 2016-Dec-22 15:39 tlmiller
- 2016-Dec-22 15:25 tlmiller
- 2016-Dec-22 12:12 tlmiller
- 2016-Dec-22 12:08 tlmiller
- 2016-Dec-19 14:47 tlmiller
- 2016-Dec-17 22:12 tlmiller
- 2016-Dec-17 22:02 tlmiller
- 2016-Dec-17 21:43 tlmiller
- 2016-Dec-16 11:32 tlmiller
- 2016-Dec-16 11:12 tlmiller
description
The annex daemon will be the production implementation of the condor_annex tool. (See ExperimentalCondorAnnex.) At present, it provides only the ability to provision leased AWS instances efficiently (in bulk).
The lease implementation requires an AWS Lambda function; rather than upload it every time, for efficiency the annex daemon must be provided the function's ARN. This may be automated in the future; see the installation instructions, below, for the manual process.
installation
Install the pre-release package(s) as normal.
Add the following three lines to your HTCondor configuration:
# Turn the annex daemon on. DAEMON_LIST = $(DAEMON_LIST) ANNEXD # Optional: configure the default endpoints. All three endpoints need to be the same region. ANNEX_DEFAULT_EC2_URL = https://ec2.us-east-1.amazonaws.com ANNEX_DEFAULT_CWE_URL = https://events.us-east-1.amazonaws.com ANNEX_DEFAULT_LAMBDA_URL = https://lambda.us-east-1.amazonaws.com
Lambda function
Do the following for each region you intend to use.
[FIXME] The examples directory will include a file, template-3.json.
usage
Attachments:
- template-3.json 3993 bytes added by tlmiller on 2016-Dec-30 21:09:32 UTC.
This template creates the lease infrastructure for your AWS account.
- generate-role 2006 bytes added by tlmiller on 2016-Dec-30 21:11:58 UTC.
This script generates a template. When instantiated, that template provides an instance profile. Instances run under that profile can introspectively discover that they were granted permission to download a specific file in S3.
- 49ec2-instance.sh 4334 bytes added by tlmiller on 2017-Jan-04 20:07:23 UTC.
This script sets EC2PublicIP and EC2InstanceID (for later configuration to use) and also downloads and extracts the configuration file pointed to by the instance's role. (It also turns off nonroot access to the metadata, so that user jobs don't get access to the role's privileges.)