You will need to provide HTCondor with an access key/secret key pair. For security reasons, you specify the location of a file containing the secret key instead of specifying the secret key directly; the same goes for the access key. If you don't already have these keys, you can create new pair from the AWS web console; the process varies depending on which kind of account you have. (FIXME: (link to) Instructions for the root account.) -You'll need access to a private S3 bucket. (FIXME: (link to) Instructions for creating private bucket.) +{subsubsection: Create a private S3 bucket} -To avoid having to upload it every time, the annex assumes that the Lambda function its needs already exists and is configured to run as a role with the required permissons. We've provided a CloudFormation template that will create and configure the Lambda function for you [FIXME: where?]. Instructions follow for readers who haven't created a stack from a template file before. After logging into the AWS web console, do the following for each region you intend to use (start with us-east-1, since that has the example AMI): +You'll need access to a private S3 bucket. (FIXME: (link to) Instructions for creating private bucket.) In the following instructions, we'll call this bucket 'privateBucketName'; replace that string, when you see it, with the actual name of the private bucket you created in this step. + +{subsubsection: Prepare the lease machineery} + +To avoid having to upload it every time, the annex assumes that the Lambda function its needs already exists and is configured to run as a role with the required permissons. We've provided a CloudFormation template that will create and configure the Lambda function for you [FIXME: where?]. Instructions follow for readers who haven't created a stack from a template file before. After logging into the AWS web console, do the following for each region you intend to use (you do just 'us-east-1' to start, since that has the example AMI): 1: Switch to the region. (The second drop-down box in from the upper right.) 1: Switch to CloudFormation. (In the _Services_ menu, under _Management_.) @@ -54,13 +58,17 @@ 1: AWS should return the list of stacks; select the one you just created and select the "outputs" tab. 1: Copy the long string labelled "LeaseFunctionARN"; you'll need it to configure _condor_annex_. It may take some time for that string to appear (you may need to reload the page, as well.) Wait the stack to enter the 'CREATE_COMPLETE' state before using the LeaseFunctionARN (see below). -For the same reason, you'll have to create a role for the annex instances, so they (but nobody else) can access the private S3 bucket. [FIXME: This should probably just be CF parameter?] Use the =generate-role= script to create a CloudFormation template: +{subsubsection: Prepare the dynamic configuration machinery} + +For the same reason, you'll have to create a role for the annex instances, so they (but nobody else) can access the private S3 bucket. [FIXME: This should probably just be CF parameter?] Use the =generate-role= script, distributed FIXME, to create a CloudFormation template: {term} -$ generate-role privateBucketName config.targ.gz > role.json +$ generate-role privateBucketName config.tar.gz > role.json {endterm} -Create a stack by uploading =role.json=; its output will be named "InstanceConfigurationProfile", and you'll need its value later. +Create a stack by uploading =role.json=, but otherwise following the instructions from the previous section; the stack's output will be named "InstanceConfigurationProfile", and you'll need its value later. + +{subsubsection: Create Spot Fleet role} If this account you're using has never created a Spot Fleet, create one now: @@ -71,6 +79,8 @@ 1: Click the "Next" button. 1: [FIXME: automagic creating the IAM Fleet Role]. +{subsubsection: Create Security Group} + You'll also need a security group that allows HTCondor (and SSH, just in case) through the firewall: 1: Click on "Security Groups" (under _NETWORK & SECURITY_).