+{section: OUT OF DATE}
+
+This document is out of date. It describes how things were done in the configure/imake work. In the cmake world things are different. I've done some work to update it (2011-8-28), but there are almost certainly errors.
+
{section: Adding a new external package to condor}
This document describes how to add a new external package to the
@@ -14,50 +18,33 @@
*::: Build script example
*:: Handling patches (optional)
*: Telling the build system about the new package
-*:: Changing autoconf-related stuff
-*:: Changing imake-related stuff
+*:: Telling CMake about your package
*: Dealing with externals caches
Before getting into the specifics, here's the 10,000 foot view of
how the Condor build system deals with externals:
-
*:The actual source tarball is hosted on a public web server.
-*:autoconf-generated =configure= script decides what versions
- of what externals we're trying to use for a given version of
- Condor
-*:=configure= script puts its results in
- =config/externals.cf= and =config/config.sh=
-*:=imake= uses =config/externals.cf= to define make
- variables that point to the necessary parts of each external
- (usually the installed output of the external build)
-
-*:=src/Imakefile= contains rules that specify what externals
- actually need to be built (and optionally, what other externals
- they depend on). These make rules are fired off by =make externals=,
- which is a rule that the top-level =all=
- make rule depends on, early in the build.
-*:Each external package that's being built is managed via
+*:cmake decides which externals we need. The version is part of the cmake information for the branch.
+*:(?? May be outdated ??) Each external package that's being built is managed via
=externals/build_external= which is the main script that
handles common tasks for all externals. This script is
responsible for setting up some directories, downloading and
unpacking the source tarball,
maintaining a build log, and a few other things.
-*:The =externals/build_external= script invokes a
+*:(?? May be outdated ??) The =externals/build_external= script invokes a
build script that is specific to a given version of a given
external. This build script handles the details of building a
specific external and installing the libraries, header files,
and/or anything else we need from the package into a location
where the Condor build can use it.
-
-*:If the build script is successful,
+*:(?? May be outdated ??)If the build script is successful,
=externals/build_external= touches a file that says the
external was happily built and installed. The Condor build
Makefiles can depend on this "trigger" file to ensure that
anything that needs the external is built after the external was
successfully built.
-
Understanding the above overview will help make sense of the
specific steps needed for modifying the build system to know about and
use a new external.
@@ -88,10 +75,7 @@
Regarding the name: it should be descriptive, but not too
long/verbose. "z" is not good. "zlib-compression-library" isn't
-either. usually it's pretty obvious. if you have any questions
-*ASK* someone else *BEFORE* you start adding, since it'll
-eventually require adding a new directory to our repository, and
-that's impossible to undo...
+either. usually it's pretty obvious.
Regarding the version: ideally, this should be the official version
number of the external package as decided by its authors. If we patch
@@ -105,7 +89,7 @@
When adding new patches to an existing external, add a =-p1= to the =[version]= to get your new version. If the =[version]= already ends in =-p<number>=, increment the number for your new version. You should continue to use the original tarball (with it's original name).
-Again, if you have any questions or are uncertain, just ask.
+If you have any questions or are uncertain, just ask. Good people to ask are *TODO*.
Finally, the combination of =[name]-[version]= is often
referred to in the build system (and this document) as the *package
@@ -116,38 +100,21 @@
{section: Packaging the external so it can be built by the build system}
-
Each external package used to build Condor lives in a unique
directory as part of the =externals= directory tree. The basic
directory layout of =externals= is as follows:
{code}
externals/
- /build
- /install
- /triggers
/bundles/
- /[name]
+ /[name]/
/[version]
{endcode}
-The =build= directory is a temporary build sandbox, which
-we'll talk about more later. The =install= directory is where
-each external package installs its build output so it can be used by
-the Condor build. The =triggers= directory holds the trigger
-files (mentioned in the overview) that determine if a given external
-was successfully built. Want to have your Condor workspace rebuild the
-zlib external? Just remove the
-
-=externals/triggers/zlib-[version]= file and the
-=src/Imakefile= will ensure that a top-level "make" will
-rebuild zlib...
-
-The =build=, =install=, and =triggers= directories can optionally live in a separate location, where they can be shared by multiple build workspaces. On unix, this location is given by the =-with-externals= option to configure. On Windows, this location is given by the =EXTERN_DIR= environment variable. At UW-CS, /p/condor/workspaces/externals will be used if it exists.
The =bundles= directory contains subdirectories for each
-kind of package, and each package directory has subdirectories for
-each version of that package. For example, at the time of this
+kind of package, and each package directory has a subdirectory for
+the version of that package. Some packages may have multiple versions *TODO Why?* For example, at the time of this
writing, we've got 5 different versions of the glibc external, each
living in their own subdirectory of =externals/bundles/glibc=.
@@ -155,18 +122,16 @@
externals/
/bundles/
/glibc/
- /2.1.1
- /2.2.2
- /2.2.93
- /2.3.2
- /2.3.2.27.9.7
+ /2.3.2.27.9.7/
+ /2.5-20061008T1257-p0/
+ /2.5-20061008T1257-x86_64-p0/
+ /2.7-18/
+ /2.7-18-x86_64/
{endcode}
-Inside each version-specific subdirectory, are 2 main things:
-
-*:=URLS= file, which contains URLs to the source tarball(s)
-(ideally, exactly what you'd download
- from the authors distribution, unmodified)
+Inside each version-specific subdirectory, are several things:
+
+*:=CMakeLists.txt= which describes how to build the external. It will include the URL to download the source tarball(s) from. (ideally, exactly what you'd download from the authors distribution, unmodified)
*:Build script
Optionally, there may be patch file(s) which include modifications
@@ -176,7 +141,6 @@
{subsection: Setting up the source tarball}
-
Ideally, there's nothing you should need to do to modify the source
tarball of the external package. We want the original, unmodified
source whenever possible. However, the name of the tarball is
@@ -714,3 +678,9 @@
If you do decide to use the AFS cache, you must make sure it has the final revision of your external once you're ready to check it in. You can do so by simply removing the appropriate trigger file in =/p/condor/workspaces/externals/triggers=.
The externals caches in NMI are trickier. Each machine has a local cache and some platforms have multiple machines. Manually clearing the caches on all of the machines is cumbersome and error-prone. Better to never use the externals cache when developing a new external.
+
+{section: FAQS}
+
+*How do I rebuild a specific external*
+
+TODO
