HTCondorWiki: Build Modernization

Page History

Introduction

This wiki page contains information regarding build modernization and consolidation. For details on progress see Ticket #779

Goals and Objectives

The goal of updating the condor build mechanism is far more reaching then using the latest wiz-bang tool to build code, and is meant to address the following issues:

Build consistency across platforms (Including Windows): Currently Imake does a good job for build consistency across *nix variants, however there is an entirely different set of mechanics for dealing with builds on Windows, which few people really understand. To make matters a bit more confusing, is that the mechanism by which we build & test on Windows is quite non-standard which presents an issue as condor becomes a more open source project. This plurality causes several issues in maintaining two desperate build files, with completely different mechanisms. It would be in the best interests for the group in the long run to employ a single build mechanism which has support on all condor platforms.
Support for unit testing within the build: For the purposes of testing it is advantageous to have a build tool which supports running automated unit tests as part of a "unit" build, with a minimal amount of external overhead.
Packaging: Many build systems provide automated packaging for it's native environment. This can be highly useful as it eliminates and automates custom build steps and scripts and makes it easier to maintain. Having this requirement can reduce the overhead on condor developers and allow them to focus on bug fixes and new features. It also allows flight worthy to get out of the packaging business and simplifies the consumption for downstream folks.

Possible Options

If we use the afore mentioned goals as hard requirements, then it eliminates a lot of possible choices, and only a couple remain:

SCons: SCons is a fairly powerful tool build atop of a python, which allows you to pretty much do anything. The only major disadvantage is the issue of packaging, and size of adoption within the open source community. There are also issues when dealing with exiting auto-tools style builds.
bjam: Created as a replacement/extension to perforce JAM it has it's own syntax and supports numerous platforms, it also does not provide flexible packaging features, and lacks the transition glue to move from an auto-tools style build.
CMake - winner: CMake is widely adopted in the open source community as a x-platform replacement for auto-tools style builds. It has it's own Macro-style syntax, but unlike the afore mentioned tools it is a generator similar to Imake and it works with existing tools (make, visual studio). Another powerful tool in the "CMake warchest" is CPack, which allows for various methods of packaging, across multiple OS distributions.

Getting Started

1.) You will need to obtain CMake 2.8 or > for the target platform you are working on.

2.) Checkout origin/V7_5-cmake-branch, for windows you will need git 1.6.4 or greater.

3.) cmake ./CMakeLists.txt or run cmake-gui

4.) Now you can run make or open the visual studio solution file.

Windows Only:

Prior to building for the 1st time there are some prerequisites that you will need to perform

5.) Install the lastest platform SDK and reopen the solution

6.) Have a recent version of cygwin installed

6.) Open Tools->Options under "Projects and Solutions" click on VC++ Directories and add the following to the Executable directories

$(SolutionDir)\build\bin
C:\cygwin\bin

7.) Open and build.

Contributing

All of the updated cmake files exist under src directories and are open to contribution. The current branch tracks against master(7.5 series).

Conventions

#include <lib>/header.h (no more relative includes)
Compiles fully in C++
Proper use of extern
Minimal linkage, explicitly specify
Platform specific files are suffixed as to allow glob macros to add to target (.windows, .unix)
condor specific macros are located under build/cmake

Packaging

CMake's compliment, CPack, supports a large number of packaging types which can be found here, and it is currently in its infancy in the code base. There are two targets which are created when you include(CPack):

make package
make package_source

The road ahead is to create a stub'd out CPackOption.cmake.in file and use cmakes configure options to fill in the gaps based on the target platform and environment, thereby reducing the cruft in the master CMakeLists.txt and providing the wiz-bang feature of platform specific package generation.

TODO

see Ticket #779

Links

CMake Official Documentation

Google Code Search for open source examples