When we say "Personal HTCondor" we mean two things which are actually independent, but which normally go together 1: A single node HTCondor pool 1: A HTCondor pool in which all of the daemons and jobs run as a single user These instructions show you how to set up HTCondor to be both of these things, but keep in mind that they are actually independent, and you can easily change the configuration to have a multi-node pool running as a single user, or a single-node pool where the daemons run as root (and jobs run unprivileged). These instructions are for Linux, Unix and OSX. {wiki: CreatingPersonalHtcondorWin Instructions for Windows} {section: Get HTCondor binaries} Downloading the tarball/zip release *: {link: https://research.cs.wisc.edu/htcondor/downloads/ Download Official HTCondor Releases} *: {link: http://batlab.chtc.wisc.edu/results/continuous.php Download Daily HTCondor builds} (click on the row you want, and then on the link under the platform you want) Or build HTCondor and tar/zip the release_dir directory from the build *: {wiki: BuildingHtcondorOnUnix Build HTCondor} and then tar it up. {term} # assuming that you build HTCondor V8_4_0-branch in /scratch/build/condor cd /scratch/build/condor/release_dir tar czf ../condor-8.4.0-x86_64_RedHat6-unstripped.tar.gz . {endterm} For Personal HTCondor we use the tarball/zip release because this lets you control where the binaries will be installed, and will allow you have more than a single version of HTCondor running at once. {section: Setting up your Personal HTCondor} The remainder of this guide will assume that you downloaded or built the condor-8.4.0-x86_64_RedHat6-unstripped.tar.gz tarball. If you are working with a different tarball, adjust the commands accordingly. {subsection: Unpack the HTCondor binaries (setting up "RELEASE_DIR")} We refer to the place where the HTCondor binaries live as RELEASE_DIR, you can unpack the tarball to create this directory using the following commands. {term} cd /scratch tar xf /condor-8.2.10-x86_64_RedHat6-unstripped.tar.gz # optionally rename the directory to something simpler mv condor-8.4.0-x86_64_RedHat6-unstripped condor84 {endterm} The directory */scratch/condor84* will now contain the HTCondor binaries, it will contain subdirectores bin, sbin, libexec, etc and several others. We call this directory *RELEASE_DIR*. {subsection: Create configuration (setting up "LOCAL_DIR")} HTCondor needs configuration files, and at least 3 runtime directories: log, spool and execute. The condor_install script will create all of these as children of a directory we call *LOCAL_DIR*. LOCAL_DIR can be the same directory as RELEASE_DIR, or it can be different. In this example we will make it different. {term} mkdir /scratch/local/condor84 /scratch/condor84/condor_install --local-dir=/scratch/local/condor84 --make-personal-condor # optional, copy the script that sets up the environment to local_dir cp /scratch/condor84/condor.sh /scratch/local/condor84 {endterm} The directory */scratch/local/condor84* will now contain directories log, spool, and execute. It will also have a condor_config.local file ready for you to begin customizing the configuration. The condor.sh script will contain commands to setup the environment so that you can use the HTCondor you have just created. It puts the HTCondor binaries into your path, and sets the CONDOR_CONFIG variable. If you use the sh or bash shell, you can use it like this {term} source /scratch/condor84/condor.sh {endterm} {subsection: Run your Personal HTCondor} First, make sure that you will run the correct HTCondor. The =which= command can be used to find out if you will run the correct condor_master: {term} > source /scratch/condor84/condor.sh > which condor_master /scratch/condor84/sbin/condor_master {endterm} Before we start Condor we need to make a few changes to the configuration. Open */scratch/condor84/etc/condor_config* in your editor and append the following lines to it. (If NETWORK_INTERFACE is already set to 127.0.0.1 then you don't need to do this) {snip: add to end of condor_config} CONDOR_HOST = 127.0.0.1 NETWORK_INTERFACE = 127.0.0.1 {endsnip} Then run condor_master: {term} condor_master {endterm} Wait a second, and make sure that everything is running: {term} ps -x | grep condor {endterm} After running this command you should see the following daemons running: *: condor_master *: condor_procd *: condor_collector *: condor_negotiator *: condor_startd *: condor_schedd {subsection: Running a Simple Job} To start a job we must first create a submit file, use your editor to create a file sleep.sub, it should have the following statements {file: sleep.sub} # The executable variable specifies the program to be run. In this case we will call the "sleep" function executable = /bin/sleep # Use the following variable to pass arguments to the executable arguments = 10 # Use the following to specify how many of this job to queue and run queue 2 {endfile} After saving your changes, it's now time to submit the submit file you just created {term} condor_submit sleep.sub {endterm} To check the status of your jobs use: {term} condor_q {endterm} {subsection: Closing Condor} If all is well with your personal HTCondor and all jobs have completed successfully, you can kill it with: {term} condor_off -master {endterm} Remember you must use /scratch/condor84/condor.sh or some other method to set the PATH, CONDOR_CONFIG and other environment variables _before you start HTCondor or use any of the tools_. {wiki: CreatingPersonalHtcondorTheHardWay A more complicated recipe that doesn't use condor_install}