When we say "Personal HTCondor" we mean two things which are actually independent, but which normally go together 1: A single node HTCondor pool 1: A HTCondor pool in which all of the daemons and jobs run as a single user These instructions show you how to set up HTCondor to be both of these things, but keep in mind that they are actually independent, and you can easily change the configuration to have a multi-node pool running as a single user, or a single-node pool where the daemons run as root (and jobs run unprivileged). These instructions are for Windows. {wiki: CreatingPersonalHtcondor Instructions for *nix and OSX} {section: Get HTCondor binaries} Downloading the tarball/zip release *: {link: https://research.cs.wisc.edu/htcondor/downloads/ Download Official HTCondor Releases} *: {link: http://submit-3.batlab.org/results/continuous.php Download Daily HTCondor builds} (click on the row you want, and then on the link under the platform you want) Or build HTCondor and tar/zip the release_dir directory from the build *: {wiki: BuildingHtcondorOnWindows Build HTCondor} and then zip it up. (izip is in the msconfig directory of your sources) {term} REM assuming that you build HTCondor V8_4_0-branch in c:\scratch\build\condor cd /d C:\scratch\build\condor\release_dir izip -r ..\condor-8.4.0-339550-Windows-x86.zip * {endterm} For Personal HTCondor we use the tarball/zip release because this lets you control where the binaries will be installed, and will allow you have more than a single version of HTCondor running at once. {section: Setting up your Personal HTCondor} The remainder of this guide will assume that you downloaded or built the condor-8.4.0-339550-Windows-x86.zip archive. If you are working with a different archive, adjust the commands accordingly. {subsection: Unpack the HTCondor binaries (setting up "RELEASE_DIR")} We refer to the place where the HTCondor binaries live as RELEASE_DIR, you can unpack the archive to create this directory using the following commands. {term} cd \scratch mkdir condor84 cd condor84 7z x \condor-8.4.0-339550-Windows-x86.zip {endterm} The directory *\scratch\condor84* will now contain the HTCondor binaries, it will contain subdirectores bin, sbin, libexec, etc and several others. We call this directory *RELEASE_DIR*. {subsection: Create configuration (setting up "LOCAL_DIR")} HTCondor needs configuration files, and at least 3 runtime directories: log, spool and execute. Create configuration files by copying condor_config.generic from the etc directory and then editing it {term} mkdir \scratch\local mkdir \scratch\local\condor84 mkdir \scratch\local\condor84\log mkdir \scratch\local\condor84\spool mkdir \scratch\local\condor84\execute copy \scratch\condor84\etc\condor_config.generic \scratch\condor84\condor_config notepad \scratch\condor84\condor_config {endterm} Open *\scratch\condor84\condor_config* in your editor, find =RELEASE_DIR= and change the value to c:\scratch\condor84, and find =LOCAL_DIR=, remove the # and change the value to c:\scratch\local\condor84 {snip: change lines in condor_config to this} ## Where have you installed the bin, sbin and lib condor directories? RELEASE_DIR = C:\scratch\condor84 ## Where is the local condor directory for each host? This is where the local config file(s), logs and LOCAL_DIR = c:\scratch\local\condor84 {endsnip} Then at the bottom of this file add {snip: add to end of condor_config} use ROLE : Personal NETWORK_INTERFACE = 127.0.0.1 {endsnip} If you plan to run more than a single instance of HTCondor on this machine, you will also need to set unique value for =PROCD_ADDRESS=, and you _may_ want to configure a unique port for the Collector listen on. {snip: [optional] add to condor_config.local} PROCD_ADDRESS = \\.\pipe\condor_procd_pipe8399 #optional COLLECTOR_HOST = $(CONDOR_HOST):8399 {endsnip} {subsection: Setup your environment to run your Personal HTCondor} In order to start this Personal HTCondor or to use any of the HTCondor tools with it, you need to set the PATH and CONDOR_CONFIG environment variables. One way to do this it to create a small batch file and use it setup the environment. create this file in in your \scratch\condor84 directory. call it setenv.bat {file: setenv.bat} set PATH=c:\scratch\condor84\bin;%PATH% set CONDOR_CONFIG=c:\scratch\condor84\condor_config {endfile} {subsection: Run your Personal HTCondor} First, make sure that you will run the correct HTCondor. The =which= command can be used to find out if you will run the correct condor_master: {term} > \scratch\condor84\setenv.bat > where condor_master \scratch\condor84\bin\condor_master.exe {endterm} Then run condor_master: {term} > start \scratch\condor84\bin\condor_master -f {endterm} You can see the daemon list using below command to make sure all the daemons are running: {term} > condor_config_val daemon_list {endterm} Wait a second, and make sure that daemon list is running: {term} > tasklist | findstr /i condor {endterm} Remember you must use \scratch\condor84\setenv.bat or some other method to set the PATH and CONDOR_CONFIG and environment variables _before you start HTCondor or use any of the tools_. {subsection: Run a simple job on your Personal HTCondor} Create a simple script, call it simple.bat, it should print some output and then exit - something like this: {file: simple.bat} echo This is a Simple job echo This is the job's environment env {endfile} Create a HTCondor submit file to run this script, call it simple.sub it should have the following submit statements {file: simple.sub} executable = simple.bat transfer_executable = true should_transfer_files = yes log = simple.log output = simple.out error = simple.err queue 1 {endfile} Run the following command to submit this job to HTCondor {term} > condor_submit simple.sub {endterm} As soon as the job is successfully submitted, it will create a log file called simple.log that will contain something like this: {term} > type simple.log 000 (001.000.000) 09/24 14:03:54 Job submitted from host: <127.0.0.1:50558?addrs=127.0.0.1-50558> ... {endterm} You can use the condor_q command to see the job in the SCHEDD {term} > condor_q -- Schedd: you@your-machine : <127.0.0.1:50558?... ID OWNER SUBMITTED RUN_TIME ST PRI SIZE CMD 1.0 you 9/24 14:08 0+00:00:00 I 0 0.0 simple.bat 1 jobs; 0 completed, 0 removed, 1 idle, 0 running, 0 held, 0 suspended {endterm} If you don't see anything when you run condor_q, the job may have completed already. In that case, try condor_history {term} > condor_history ID OWNER SUBMITTED RUN_TIME ST COMPLETED CMD 1.0 you 9/24 14:03 0+00:01:02 C 9/24 14:06 simple.bat {endterm} If the job does not run within a minute or two, there is most likely a configuration problem with your Personal HTCondor, See {wiki: BasicTroubleshootingPersonalHtcondor Basic Troubleshooting for a Personal HTCondor} for more information. When you are done with your personal HTCondor, you can kill it with: {term} > condor_off -master {endterm}