description
condor_annex was always intended to work with multiple cloud providers, and this experiment is about supporting the Google Cloud Platform (GCP).
the rules of acquisition
Until further notice, the binaries for this experiment are available by invitation only.
limitations and restrictions
The current (2020-05-15) version of this experiment is extremely limited. All that it does is start instances, which, if they're Debian 10, will join the annex pool. These instances will have AnnexName defined in their machine ad, but will otherwise not behave as expected. In particular, this includes, but is not limited to:
- The instance will not terminate itself when HTCondor is turned off (via condor_off -annex, usually).
- The instance will not terminate itself because it is idle.
- The instance will not terminate itself, nor be terminated, because the annex's duration has expired.
- The instance will not restrict itself to running jobs from the annexing user, or to jobs with MayUseAWS set.
- The corresponding command-line options will be ignored.
- Instances not in the pool will not be tracked.
- The instance will not prevent user jobs from accessing the instance metadata, which includes things like the pool password and the instance's credentials (if any).
set up
The current (2020-05-15) version of this experiment does not do any GCP set-up for you. In particular, you must:
- arrange for instances started by condor_annex to have a firewall rule allowing inbound port 9618;
- and provide condor_annex with the necessary user credentials.
The GCP web console's default firewall editing appears to apply to all instances in the project, current and future.
To obtain the necessary user credentials, download and install Google's gcloud tool (see https://cloud.google.com/compute/docs/gcloud-compute/) and then complete the gcloud init process (including logging in).
usage
Running
condor_annex -gcp -count N
will complain about missing command-line options. You can set defaults for the missing command-line options, and will probably want to, by modifying the user config file, as usual for condor_annex. An example follows:
~/.condor/user_config
# These should be the same for everybody. ANNEX_DEFAULT_GCP_SERVICE_URL = https://compute.googleapis.com/compute/v1 ANNEX_DEFAULT_GCP_STARTUP_SCRIPT = $(RELEASE_DIR)/etc/examples/gce-annex-configure.sh # If you pick a different default zone, adjust the rest of the config appropriately. ANNEX_DEFAULT_GCP_ZONE = us-central1-a us_central1_a.ANNEX_DEFAULT_GCP_MACHINE_TYPE = projects/<project ID>/zones/us-central1-a/machineTypes/n1-standard-1 us_central1_a.ANNEX_DEFAULT_GCP_IMAGE_NAME = projects/debian-cloud/global/images/debian-10-buster-v20200413 us_central1_a.ANNEX_DEFAULT_GCP_PROJECT = <project ID> us_central1_a.ANNEX_DEFAULT_GCP_AUTH_FILE = /home/<username>/.config/gcloud/credentials.db # Deliberately left blank. May be useful if you've logged into more than one account with gcloud. us_central1_a.ANNEX_DEFAULT_GCP_ACCOUNT =