-One approach to using cloud resources to run HTCondor jobs is to use the =condor_annex= tool to expand an existing pool onto the cloud (see HowToUseCondorAnnexWithOnDemandInstances). Another approach, documented here, is to create a new HTCondor pool entirely in the cloud. The HTCondor team maintains an AWS Marketplace entry to help simplify the process[, but target-audience disclaimer]. These instructions assume you already have an AWS account and a key pair. +One approach to using cloud resources to run HTCondor jobs is to use the =condor_annex= tool to expand an existing pool onto the cloud (see HowToUseCondorAnnexWithOnDemandInstances). Another approach, documented here, is to create a new HTCondor pool entirely in the cloud. The HTCondor team maintains an AWS Marketplace entry to help simplify the process[, but FIXME target-audience disclaimer]. These instructions assume you already have an AWS account and a key pair. {section: Overview} @@ -6,12 +6,16 @@ [FIXME: subsections, linkage] -1: Starting a Head Node -2: Adding Cloud Resources to Your New Pool -3: Running Jobs -4: Cleaning Up +1: Start a Head Node +2: Add Cloud Resources to Your New Pool +2:: Log into your Head Node +2:: Obtain an Access Key +2:: Prepare Account +2:: Add Cloud Resources +3: Run Jobs +4: Clean Up -{section: Starting a Head Node} +{section: Start a Head Node} 1: Open HTCondor's {link: https://aws.amazon.com/marketplace/pp/B073WHVRPR Marketplace entry} in another tab. 1: Click the orange 'Continue' button to the right. You may need to log in to AWS at this point. @@ -19,7 +23,7 @@ 1: This step is where you will start spending money. Scroll back up; there will be section to the right titled "Price for your Selections." That's what Amazon will charge starting when you click the orange 'Launch with one-click' button. We'll remind you in these instructions, but you'll have to stop or terminate the instance your about to start on your own. 1: On the next page, click on the "Your Software" link in the green box. -{section: Adding Cloud Resource to Your New Pool} +{section: Add Cloud Resources to Your New Pool} Your head node (by default) starts with two CPUs and 8 GiB of RAM. Immediately after you log in -- see the next section -- you can start submitting and running jobs (see the section "Running Jobs", below). However, with only two CPUs, you'll only be able to run two jobs at a time. To add cloud resources to your pool, you'll use =condor_annex=. To do that, you'll need to obtain an access key for =condor_annex=, so it can act interact with AWS on your behalf. You'll only need to do that once. Likewise, =condor_annex= has to do some one-time set-up for each account. Once that's done, you can run =condor_annex= to add cloud resources as often as you like. @@ -81,9 +85,9 @@ You should keep a copy both files somewhere safe, so you don't have to recreate them every time you start a new pool in the cloud. -{subsection: Running the Set-Up Command} +{subsection: Prepare your Account} -The following command will set up your AWS account. It will create a number of persistent components, none of which will cost you anything to keep around. These components can take quite some time to create; =condor_annex= checks each for completion every ten seconds and prints an additional dot (past the first three) when it does so, to let you know that everything's still working. +The following command will prepare your AWS account for =condor_annex=. It will create a number of persistent components, none of which will cost you anything to keep around. These components can take quite some time to create; =condor_annex= checks each for completion every ten seconds and prints an additional dot (past the first three) when it does so, to let you know that everything's still working. {term} $ condor_annex -setup @@ -94,9 +98,9 @@ Setup successful. {endterm} -{subsection: Checking the Setup} +{subsubsection: Verify Account Preparation} -You can verify at this point (or any later time) that the setup procedure completed successfully by running the following command. +You can verify at this point (or any later time) that the account-preparation procedure completed successfully by running the following command. {term} $ condor_annex -check-setup @@ -124,7 +128,7 @@ {link: http://research.cs.wisc.edu/htcondor/manual/v8.7/6_Cloud_Computing.html Complete documentation} is also available. -{section: Running Jobs} +{section: Run Jobs} To run on your new resources, a job's submit file must contain the following line: @@ -138,7 +142,7 @@ should_transfer_files = TRUE {endverbatim} -{section: Cleaning Up} +{section: Clean Up} {subsection: The Cloud Resources}