Building Htcondor Prior To Cmake On Windows

Not logged in

• Browse
• Help
• Home
• Login
• Reports
• Search
• Timeline
• Wiki

• Contents
• Text

Building HTCondor prior to 7.5.5 on Windows

• Active-State Perl
• Microsoft Visual Studio 2008 Edition + SP1
• Microsoft Windows Server 2003 SP1 Platform SDK

NOTE: The Platform SDK's version is very important. This is the latest version that both supports Windows 2000 and does not play silly tricks with the registry. As for Microsoft Visual Studio Team System 2008, only select the x86 compiler (which is the default); otherwise, some of the environment scripts will not be written correctly (on Vista, anyway; I haven't tested this on other platforms). Not sure why this happens exactly; but if you do select the other options, the Microsoft .NET Platform directory is exported incorrectly to the environment (maybe I'll get around to reporting this to them at some point). The problem is that our command-line build depends on a tool shipped with the Microsoft .NET Platform: msbuild.exe. It's a glorified version of make that uses XML as it's input. What's nice about it is that it will build the project directly, rather than manually exporting Makefiles for consumption, as was previously required.

• Install all the packages listed above to their default destinations, as some of our scripts are reliant on these locations. We do take great efforts to avoid hard-coded paths; however, some things cannot be fully auto-detected at present (sorry, but that is the state-of-the-art right now).
• The order in which you install them is also important; for instance, a Visual Studio Service Pack should be installed after Visual Studio is installed. Also, for proper integration, the Platform SDKs should be installed after Visual Studio has completed its installation, so that it will integrate gracefully with the software.

You can get away with the default installation, but it does have a large disk foot-print. If disk space is at a premium, then the following features can be removed from a Custom installation:

• Visual C#
• Visual Basic
• Visual Web Developer
• Dotfuscator Community Edition
• Unit Testing Tools
• Microsoft SQL Server 2005 Express Edition (x86)
• Crystal Reports Basic for Visual Studio 2008

Do, however, make sure to keep the Redistributable Merge Modules, as the installer (the MSI) will use these to install the correct runtime libraries in the appropriate location.

If you are building HTCondor on an x64 machine then after installing the Platform SDKs, you will either need to move or copy them to

  C:\Program Files

Instead of the default C:\Program Files (x86) destination. This move is required because the configuration scripts that are shipped with the SDKs, which we depend upon, have the C:\Program Files path hard-coded in them, while the installer does not.

Parallel builds should be disabled (these can actually be used now, Ben Burnett [2009-07-10]). As much as we would love to support building projects in parallel, the constraints of current code-base will not allow for it. By default, Visual Studio is configured to build x number of projects in parallel, where x is the number of physical CPUs or cores you have on the machine; you will need to disable this feature before being able to successfully build HTCondor. These changes can be made as follows:

1. From the Tools menu, select Options...
2. In the Options dialog, expand Project and Solutions, and select Build and Run
3. On the right hand side, set the maximum number of parallel project builds to 1.

This setting is stored persistently and not in the user configuration files; therefore, you will only need to be done once per user.

The reason why we don't support building projects in parallel is not because we can't, nor because Visual Studio does not respect inter-project dependencies; in fact, it's quite the opposite: Visual Studio does a great job of observing project dependencies; unfortunately, however, our code currently depends on two pre-compiled headers, which are not fully independent; meaning that one may finish before the other is done and destroy the rest of the build process. Why two, you ask? Well, one is for the C++ code, while the other is for the C code. We could, in theory, support parallel builds, if we went without pre-compiled headers; however, the performance gain from parallelism may not out-way that of the compile-time speed-up that the existing pre-compiled headers already do. The other option would be to force the two projects to build first, before building anything else in parallel--but I'm not sure that this is feasible. Further testing and benchmarking would be required to determine any gains anyway.

In NMI, the test lab we use here, you will also need to create a user account from which we will derive the slot user profiles from. This is a fairly simple process, but for compleatness, the steps are included.

For details on creating the template account HTCondor, please refer to WindowsUserProfiles.

• You must be on the nmi private network to access the nmi cygwin mirror
• get http://cfengine/mirror/nmi_cygwin/setup.exe
• next, install from internet, root is C:\cygwin, packages dir is C:\cygwin\packages
• direct connection, add mirror http://cfengine/mirror/nmi_cygwin
• ensure view is on "Category", click thing next to All to set to "install" (install ALL packages), click next
• answer "no" to any "try again?" messages

• Right click My computer -> remote -> allow users to connect remotely

• Right-click your Command Prompt shortcut, select Properties > Advanced... > Check Run as administrator
• Click OK

• Right-click your Visual Studio shortcut, select Properties > Advanced... > Check Run as administrator
• Click OK

A temporary directory has been hard-coded into gsoap.mak (i.e. C:\Temp). The presence of this directory is required for HTCondor to compile. This will be changed for future releases.

You can build HTCondor one of two ways: from the command-line, or from the IDE. Both require that you be in C:\workspace\source\msbuild directory. See GitingTheCode for further details on getting the source.

1. Run make.bat;
2. Ignore the compiler warnings, they are mostly innocuous and likely to disappear in upcoming releases;
3. If there are any errors, please reffer to the Trouble Shooting sub-section bellow.

1. Run run_msdev.bat;
2. Ensure the Release "Solution Configuration" is selected (Debug is the default);
3. From the Project menu, select Build Solution;
4. Ignore the compiler warnings, they are mostly innocuous and likely to disappear in subsequent releases;
5. If there are any errors, please refer to the Trouble Shooting sub-section bellow.

You can customize how Visual C++ displays classes in the variable view. By default VC++ displays "{...}" and you need to click the small '+' icon to expand the members. You can change this behaviour, and make Visual C++ display whatever data member you want, in whatever order and in whatever format you like.

You need to locate a file called AUTOEXP.DAT in your Visual C++ installation. By default it will be:

VC++ 6.0:

  C:\Program Files\Microsoft Visual Studio\Common\MSDev98\Bin\AUTOEXP.DAT

VC++ 9.0:

  C:\Program Files\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\autoexp.dat

After you have made the changes and saved the file, you will need to restart Visual C++ for the changes to take effect.

In some rare cases the local computer's environment may interfere with building HTCondor: most of the time this is due to an incompatibility in SDK versions. By default, newer versions of Visual Studio install the, at the time, up-to-date SDKs. This is usually found in the %ProgramFiles%\Microsoft SDKs folder. If there is a problem when running the build scripts, this is likely the culprit. It also contains the same configuring scripts that the Platform SDKs do; however, they do not support Windows 2000. A sure sign that this is incorrectly configured is easy to diagnose: the scripts have the name, but take different command-line options, so you will see an error after running set_env.bat. You can remedy this by renaming %ProgramFiles%\Microsoft SDKs to something else while developing or compiling the HTCondor software. This will not be a problem when HTCondor drops support for Windows 2000.

Put shortly, if you do not see the following two lines in the build output:

  Setting environment for using Microsoft Visual Studio 2008 x86 tools.
  Targeting Windows 2000 and IE 5.0 RETAIL

Then there is most certainly a problem.

That said, and paradoxical as it may sound, a build can succeed with this directory intact. It all hinges on the order in which you install the software packages. That's why we stressed the order in which you install the software packages above. If, by chance, it happens to work for you in another order, then consider yourself lucky; having tried several, this seems to be the most dependable procedure.

If you wish to package up HTCondor the way we do on our web page (as a self-contained MSI installer file and a ZIP file), then follow these instructions.

Starting from the source directory, here is an example of the commands you would enter:

1. mkdir c:\temp\condor-7.5.2
2. cd msconfig
3. dorelease.bat c:\temp\condor-7.5.2
4. dopackaging.bat c:\temp\condor-7.5.2 c:\temp
5. After about 15 minutes, condor-7.5.2-winnt50-x86.msi and condor-7.5.2-winnt50-x86.zip will waiting for you in c:\temp

Packaging a build from NMI is very similar to packaging a build from source, except the binaries we want to package will be coming out of tar file from NMI instead of being produced by the dorelease.bat script as shown above. Even though you are not actually building the binaries on the packaging machine, it still needs to be setup as if it were a build machine as documented above (to be certain all the proper tools needed for packaging are installed).

1. Download results.tar from NMI for platform winnt from a given build id.
2. Untar into subdir c:\temp
3. move c:\temp\public c:\temp\condor-v7.5.2
4. rmdir c:\temp\condor-v7.5.2\testbin
5. rmdir c:\temp\condor-v7.5.2\logs
6. git checkout V7_5_2 (note: you need to checkout from git the same version of HTCondor that NMI built, as the packaging scripts and directives themselves are only found in the source code and not in the results.tar file)
7. cd CONDOR_SRC\msconfig
8. dopackaging.bat c:\temp\condor-7.5.2 c:\temp
9. After about 15 minutes, condor-7.5.2-winnt50-x86.msi and condor-7.5.2-winnt50-x86.zip will waiting for you in c:\temp

This work is supported by NSF under Cooperative Agreement OAC-2030508 as part of the PATh Project. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. Site built using CVSTrac.