HTCondorWiki: Heterogeneous Submit Batch

Page History

Here are two methods for enabling heterogeneous submission of jobs. These are jobs that may run on a variety of platforms, such as both Windows and Linux.

If Perl is available on all execute hosts

A single batch script can be defined to work on all Windows and Linux machines. If Perl is available on all potential execute machines, then depend on it, and write a Perl batch script. Name the script with the extension .pl, and Windows machines will use this extension to identify and run it as a Perl script. The first line of the Perl script will have #!, letting Linux machines know what to do.

TJ's shabang hack

Three parts to this set up make it work:

Create a small Windows console application that returns 0. Name this application #!.exe

Here is C++ source, assuming it is named success.cpp, that can be compiled to produce #!.exe:

// Force the linker to include KERNEL32.LIB
#pragma comment(linker, "/defaultlib:kernel32.lib")
extern "C" void __stdcall ExitProcess(unsigned int uExitCode);
extern "C" void __cdecl begin( void ) { ExitProcess(0); }

This code could be compiled with the Microsoft C++ compiler like this

  cl success.cpp /link /subsystem:console /entry:begin kernel32.lib

Send file #!.exe along with the job. Assuming that this file is in the current working directory at submission, the submit description file will contain
```
  should_transfer_files = IF_NEEDED
  transfer_input_files = #!.exe
```
The following batch script, named with a .bat extension, becomes the executable:
```
#!/bin/bash
#!&& @goto windows_part

echo 'Linux'
ls -l

exit 0
:windows_part
@echo off

@echo Windows
dir
```
On Linux, this works as a normal bash script; the exit 0 stops the script before it gets to the label windows_part. What Windows sees is a .bat file, so it runs in the command shell. The first line is #!/bin/bash, and Windows interprets that line as: run the program #!.exe and pass it /bin/bash as arguments. The second line is #!&& @goto windows_part, which Windows interprets as: run the program #!, and if it succeeds, goto the windows_part label in this script.

So, you have one script, that contains both Linux and Windows commands. It works on Windows as long as you have the program named #!.exe in the current directory (or in the path) that returns success.