Creating a personal HTCondor from your own build of HTCondor or from prebuilt binaries
Building HTCondor
Note this section is optional -- if you are on a CSL machine, you can skip to the "Settting up your Personal HTCondor" section
Pretend you have your build in /scratch/roy/v66.
Pretend you have you want to set up a personal HTCondor in /scratch/roy/personal-condor.
Substitute pathnames appropriately.
First, build HTCondor as described in here. After you run make, do "make install" (note: in older versions of the code, this used to be "make release"). You'll find that you have a new directory /scratch/roy/v66/release_dir, and if you look inside it, you will see something like:
> ls /scratch/roy/v66/release_dir/ bin/ include/ lib/ man/ sbin/ src/
Setting up your Personal HTCondor
There are lots of ways to set up a personal HTCondor: I'll show you one way. You want to set up your PATH to point at the bin and sbin directories in your release directory. You want to set CONDOR_CONFIG to point at a condor_config file. You want to edit your condor_config file appropriately. Here is what I do:
Make a link to your release directory
cd /scratch/roy/personal-condor ln -s /scratch/roy/v66/src/release_dir .
src/ part of the path in the above ln command, (for versions of HTCondor later than 7.5.5.)
Or, if you want to use the pre-built binaries in the CS pool, the release directory is in "/unsup/condor", and you don't need to make the subdirectories below.
Make some directories for HTCondor
mkdir execute mkdir log mkdir spool
Set up your environment
This commands are for tcsh: you'll need to edit them for bash.
cat > setup
setenv PATH /scratch/roy/personal-condor/release_dir/bin:${PATH}
setenv PATH /scratch/roy/personal-condor/release_dir/sbin:${PATH}
setenv CONDOR_CONFIG /scratch/roy/personal-condor/condor_config
(control-d)
Create your HTCondor configuration file
cp /scratch/roy/v66/src/condor_examples/condor_config.generic condor_config touch condor_config.local
condor_config file. Search for the following configuration variables, and set them as follows (using the correct pathnames and email address for you):
CONDOR_HOST = $(FULL_HOSTNAME) RELEASE_DIR = /scratch/roy/personal-condor/release_dir LOCAL_DIR = /scratch/roy/personal-condor LOCAL_CONFIG_FILE = $(LOCAL_DIR)/condor_config.local CONDOR_ADMIN = your-email@cs.wisc.edu UID_DOMAIN = $(FULL_HOSTNAME) FILESYSTEM_DOMAIN = $(FULL_HOSTNAME) COLLECTOR_HOST = $(CONDOR_HOST):11000 # Not necessary for late versions of HTCondor NEGOTIATOR_HOST = $(CONDOR_HOST):11001 # choose a policy for running jobs. # as of 8.2 the default policy is always_run_jobs #USE POLICY : ALWAYS_RUN_JOBS #USE POLICY : DESKTOP DAEMON_LIST = MASTER, STARTD, SCHEDD, COLLECTOR, NEGOTIATOR
Note: It may be useful to set CONDOR_HOST = localhost or CONDOR_HOST = localhost.localdomain if the machine running this does not have a valid IP address, is a virtual machine, or the administrator wants HTCondor to look only at that machine.
This looks like a lot of things to edit. Why is it so hard? Well, our installation script normally sets things up for you, but because you are using a build environment, it is harder to use the installation script. Also, it is useful for you to be familiar with these settings, so you understand better how HTCondor works.
Run HTCondor
First, make sure that you will run the correct HTCondor. Which should tell you the one you expect to see:
> source setup > which condor_master /scratch/roy/personal-condor/release_dir/sbin/condor_master
condor_master
ps -x | grep condor
condor_off -master
Advanced configuration and problem solving
If you are debugging a daemon, you may want to have additional debugging information. For instance, if you are debugging the collector, you can edit COLLECTOR_DEBUG. For instance, you might do:
COLLECTOR_DEBUG = D_CAT D_FULLDEBUG D_COMMAND:1