-----
-MATLAB is an excellent match for Condor. Millions of MATLAB jobs, possibly tens of millions, have been successfully run under Condor over the years. The specifics of working with MATLAB will vary from site to site, but here are some general guidelines.
+MATLAB is an excellent match for HTCondor. Millions of MATLAB jobs, possibly tens of millions, have been successfully run under HTCondor over the years. The specifics of working with MATLAB will vary from site to site, but here are some general guidelines.
-This assumes basic familiarity with MATLAB and Condor. It assumes that you can set up and maintain at least a small Condor pool.
+This assumes basic familiarity with MATLAB and HTCondor. It assumes that you can set up and maintain at least a small HTCondor pool.
{section: Licensing}
-Perhaps the biggest challenge to running MATLAB under Condor is licensing. MATLAB is proprietary software with strict licensing terms. You might have one or two licenses for your desktop computers, but to run hundreds of simultaneous MATLAB jobs you would require hundreds of licenses.
+Perhaps the biggest challenge to running MATLAB under HTCondor is licensing. MATLAB is proprietary software with strict licensing terms. You might have one or two licenses for your desktop computers, but to run hundreds of simultaneous MATLAB jobs you would require hundreds of licenses.
-Your institution may already have acquired suitable licensing for MATLAB. If you have a fixed number of licenses available, Condor's {link:http://www.cs.wisc.edu/condor/manual/v7.6/3_13Setting_Up.html#sec:Concurrency-Limits Concurrency Limit support} can help.
+Your institution may already have acquired suitable licensing for MATLAB. If you have a fixed number of licenses available, HTCondor's {link:http://www.cs.wisc.edu/condor/manual/v7.6/3_13Setting_Up.html#sec:Concurrency-Limits Concurrency Limit support} can help.
Another option uses the MATLAB Compiler to create an executable that can be run with the MATLAB Compiler Runtime (MCR). The MATLAB Compiler is an optional toolbox for MATLAB. In general, executables created with the MATLAB Compiler are not subject to MATLAB's license; you are free to run as many in parallel as possible. _Check your MATLAB licensing to confirm this._
@@ -19,11 +19,11 @@
{section: Executables}
-The MATLAB executable and supporting libraries must be made available on any computers on which the Condor jobs might run.
+The MATLAB executable and supporting libraries must be made available on any computers on which the HTCondor jobs might run.
When using the MATLAB Compiler, instead of the full MATLAB executable and supporting libraries, the MCR is needed. The general techniques for making the MCR available are the same.
Accomplish this by one of
-*: *Install MATLAB on each computer that may run a MATLAB Condor job.*
+*: *Install MATLAB on each computer that may run a MATLAB HTCondor job.*
_:If you install MATLAB locally on each computer, we recommend placing MATLAB in the _same location_ on each computer. That way individual jobs can easily find it. Here is a portion of a user's submit description file:
@@ -67,12 +67,12 @@
_:The batch scripts for Windows platforms are similar.
-*: *Install MATLAB on a shared file system, such that it is accessible by each computer that may run a MATLAB Condor job.*
+*: *Install MATLAB on a shared file system, such that it is accessible by each computer that may run a MATLAB HTCondor job.*
_:Installing on a shared file system is identical to installing it on each local computer, but may be less complicated for administration.
-*: *transfer the MATLAB executable (and supporting libraries) along with the Condor job.*
+*: *transfer the MATLAB executable (and supporting libraries) along with the HTCondor job.*
_:To bring MATLAB along yourself, you would need to package it up, bring it along (specified with =transfer_input_files= in the submit description file), then have a script unpack MATLAB and start it. MATLAB is a large piece of software, and this could be slow. It will also temporarily use a bunch of disk space with a copy of MATLAB. If multiple MATLAB jobs run on the same computer simultaneously, multiple copies of MATLAB will be installed at once. Generally speaking, this is not recommended, although it is more practical for the MCR.
@@ -86,12 +86,12 @@
{section: Invoking}
-MATLAB may try to create a graphical environment. Condor does not support graphical environments; it does not make sense to open up a user interface for a job that will not have a user directly looking at it. You may need some combination of =-nosplash=, =-nodisplay=, and possibly =-nojvm= to stop MATLAB from creating a graphical environment. This should not be necessary for compiled MATLAB Condor jobs.
+MATLAB may try to create a graphical environment. HTCondor does not support graphical environments; it does not make sense to open up a user interface for a job that will not have a user directly looking at it. You may need some combination of =-nosplash=, =-nodisplay=, and possibly =-nojvm= to stop MATLAB from creating a graphical environment. This should not be necessary for compiled MATLAB HTCondor jobs.
Unless you have special arrangements to use multiple CPU cores, you will want =-singleCompThread= so that MATLAB only uses a single core.
-|=-nosplash= | Disable splash screen. (No GUI support in Condor)|
-|=-nodisplay= | Disable GUI. (No GUI support in Condor)|
+|=-nosplash= | Disable splash screen. (No GUI support in HTCondor)|
+|=-nodisplay= | Disable GUI. (No GUI support in HTCondor)|
|=-nojvm= | Disable GUI? Eliminate unnecessary Java (faster?)|
|=-singleCompThread= | Only use one CPU core (play nicely when sharing computer)|
@@ -99,11 +99,11 @@
{section: Parallelism}
-In the most common configuration, Condor does not directly support "parallel" jobs, jobs that might use a system like MPI or multiple threads to take advantage of multiple CPUs at once. Condor can launch jobs that use multiple processes or threads, but by default Condor will offer them a single CPU core to run on. (They may get lucky and be able to use more than one CPU core on a computer, but that should not be relied on.)
+In the most common configuration, HTCondor does not directly support "parallel" jobs, jobs that might use a system like MPI or multiple threads to take advantage of multiple CPUs at once. HTCondor can launch jobs that use multiple processes or threads, but by default HTCondor will offer them a single CPU core to run on. (They may get lucky and be able to use more than one CPU core on a computer, but that should not be relied on.)
A local installation may provide additional options for parallel, usually in the form of offering a job two or more CPUs on a single computer. Local administrators should be able to describe available functionality.
-Given the default configuration, it is usually better to break your work down into multiple independent jobs. For example, if you are processing 10,000 images, instead of a single MATLAB job that processes them, perhaps you could have 10,000 jobs that each process 1 image. Condor is then able to schedule your jobs across multiple computers or at least multiple cores on a single computer, giving you the speed benefits of parallelism.
+Given the default configuration, it is usually better to break your work down into multiple independent jobs. For example, if you are processing 10,000 images, instead of a single MATLAB job that processes them, perhaps you could have 10,000 jobs that each process 1 image. HTCondor is then able to schedule your jobs across multiple computers or at least multiple cores on a single computer, giving you the speed benefits of parallelism.
For older MATLAB (Possibly pre 2011?), to ensure that MATLAB only uses one core, put this in your MATLAB script:
@@ -149,7 +149,7 @@
{section: Compiled MATLAB Example (completely tested)}
-Given a successful mcc run, as described above, and assuming the MATLAB runtime is not pre-installed on any of the execute machines, you can transfer the runtime along with the Condor job. There is an example of this in CHTC in =/home/gthain/CompiledMatlabExample=. First, put the whole runtime into a single tar file, called =m.tgz=. Then, edit the =run_foo.sh= wrapper (which was created by the compiler) to add the lines
+Given a successful mcc run, as described above, and assuming the MATLAB runtime is not pre-installed on any of the execute machines, you can transfer the runtime along with the HTCondor job. There is an example of this in CHTC in =/home/gthain/CompiledMatlabExample=. First, put the whole runtime into a single tar file, called =m.tgz=. Then, edit the =run_foo.sh= wrapper (which was created by the compiler) to add the lines
{code}
tar xzf m.tgz
@@ -158,7 +158,7 @@
export MCR_CACHE_ROOT=`pwd`/cache
{endcode}
-at the beginning of the script, but after the #! line. When MATLAB runs, it wants to create a cache directory under the user's home directory, which may not exist on an execute machine, or may conflict with other concurrently running MATLAB. Then, create a Condor submit description file that looks something like:
+at the beginning of the script, but after the #! line. When MATLAB runs, it wants to create a cache directory under the user's home directory, which may not exist on an execute machine, or may conflict with other concurrently running MATLAB. Then, create a HTCondor submit description file that looks something like:
{code}
universe = vanilla
@@ -181,7 +181,7 @@
{section: Additional Resources}
-Many other sites are using MATLAB under Condor. Here are links to the documentation from just a few.
+Many other sites are using MATLAB under HTCondor. Here are links to the documentation from just a few.
*: {link:http://www.cae.wisc.edu/matlab-condor University of Wisconsin - Madison - CAE}
*: {link:http://www.liv.ac.uk/csd/escience/condor/matlab/old_instructions.htm University of Liverpool}
