-This is an attempt to enumerate all the steps necessary to release a new version of Condor, given a valid nightly build ID. It's currently a work-in-progress, but will hopefully converge on an authoritative list in the near future.
+This is a living document that enumerates all the steps necessary to release a new version of Condor.
-There are now two major parts of wrangling a release:
+In parallel with this effort the wrangler should {link: PrepareVersionHistory prepare the version history}
-*: Making sure the version history is right
-*: Everything else
-
-Each task is described in its own section. In general, the version history is a ton of work, so it's best to select two separate people, one to do the version history, one for wrangling everything else.
-
-{section: Making Sure the Version History Is Right}
-
-First, be sure to read the {link: http://www.cs.wisc.edu/condor/developers/version-history.html Version History Policy and FAQ}. This spells out the policy for the version history and explains a big part of your task at release time: finding missing entries.
-
-Of course, you can't finalize the version history until the code-freeze for the release and the source is tagged. So, in practice, you'll have to coordinate with the real release wrangler to make sure you're looking at the right diff.
-
-Once you've got the official release tag (or the build tag that's going to become the official tag), here's what you do:
-
-1: Check out a copy of the version history to verify:
-{code}
-git clone /p/condor/repository/CONDOR_DOC.git
-cd CONDOR_DOC
-git checkout -b V7_0_2-branch origin/V7_0_2-branch
-{endcode}
-Inside doc, you'll need to edit =version-history/X-Y.history.tex= where =X-Y= matches the release series you're working from (e.g. =7-0=).
-1: Run git log between the last release and the current release from the same series:
-{code}
-git log -p V7_0_1..V7_0_2 > 7.0.2-diff
-{endcode}
-If the =V7_0_2= tag doesn't exist yet, use the appropriate build tag, e.g. =BUILD-V7_0_2-branch-2008-6-9=.
-1: Go through the saved output (e.g. =7.0.2-diff=) and make sure everything in there that should be documented has a corresponding entry in the version history.
-1: If you find parts of the diff that should be documented which are missing, make a list of the missing entries and who committed the changes.
-1: A development release will normally have several fixes from the concurrent stable branch, so it should have a version history item that indicates this. Something like "This release contains all bug fixes from Condor version 7.6.2." in the 7.7.0 version history.
-1: Once you've gone through the entire diff, email your list of missing entries to condor-team@cs.wisc.edu for public humiliation and so that people know how much they owe to the version history pizza fund (see the {link: http://www.cs.wisc.edu/condor/developers/version-history.html policy/FAQ}). You can either add the missing entries yourself, or get the guilty parties to do it.
-
-That's the whole task. However, give yourself at least a day to do it, since the diff can be huge and this is a time consuming process. Once you're done, notify the "real" release wrangler so they know when it's safe to publicly release the manual.
-
-{section: Everything Else}
-
-There are two main phases of the actual release:
+There are a few phases of the actual release:
1: Preliminary work to freeze the code, make a release branch, etc. (this can take a few days, so start early)
-1: Release process given a valid build candidate
+1: Upgrading the pool(s) and watching for bugs
+1: Release the binaries to the public and announce the release
-{subsection: Preliminary Work}
+{section: Preliminary Work}
1: Coordinate a development code-freeze. This involves chasing down people who are trying to finish bug fixes, new features (if it's a development release), porting, or any other core development that is slated for a given release.
1: Ensure that the parent branch (e.g. =V7_0-branch=) is building properly on all platforms. If any platforms aren't building, you need to either solve it yourself or track down the person who broke the builds and get them to fix it immediately. This can take days, so _start early_!
-1: Notify the CS department. Send email to the =uwcs-condor@cs.wisc.edu= list announcing our plans for the pool upgrade. We like to give users at least a few days notice so that if there are important paper deadlines, etc, they have time to finish their jobs, or to let us know they need us to delay the upgrade.
+1: Notify the appropriate people about the update.
+*:: For a *stable release* we update the CS pool. Send email to the =uwcs-condor@cs.wisc.edu= list announcing our plans for the pool upgrade. We like to give users at least a few days notice so that if there are important paper deadlines, etc, they have time to finish their jobs, or to let us know they need us to delay the upgrade.
+*:: For a *developer release* we update the NMI pool and CHTC pool. Let the admins of these pools know about the pending release (currently the Condor infrastructure team for NMI, and infrastructure or Dan Bradley for CHTC)
1: Make a release branch. The main thing to know is that the branch should have the name =VX_Y_Z-branch=. Here's an example of the steps for a 7.0.3 release, from a known good NMI tag (assuming your working directory is inside of a clone of CONDOR_SRC.git):
*:: The preferred method for making a release branch is from a known good build tag on NMI.
@@ -105,13 +74,12 @@
1: If you'd like to fire off a build immediately, then read the {wiki: NmiBuild Building Condor with NMI} page. You'll need to start the build as cndrauto so follow the directions.
-That's it... you're done with the prep-work. Now, the real work begins...
-{subsection: Final Release Process}
+{section: Upgrade the Pools}
-1: Settle on a build ID candidate that can build on all platforms
+1: Settle on a build ID candidate that successfully built on all platforms. Ensure that all tests passed.
-1: Perform the ManualRegressionTests, possibly assigning some to various staff members. (Question -- has anyone done this? If so, where to run the Xen VMU tests?)
+1: Perform the ManualRegressionTests, possibly assigning some to various staff members.
1: On nmi-s006, pin the build for 365 days using =/nmi/bin/nmi_pin=:
{code}
@@ -139,8 +107,6 @@
*:: Rename the zip and MSI file to include the platform string
*:: Test the MSI installer.
-1: Definitive pass-thru of test results, by platform. Need a list by platform saying "good" or "bad".
-
1: Deploy binaries onto production pool, start the "N-day" counter.
Stable releases go onto the CS Dept. pool. Upgrading this pool is a complicated process in itself, which is documented on the {link: http://www.cs.wisc.edu/condor/developers/pool-update.html Upgrading the UW CS Pool} page. Developement releases go onto the {quote:BaTLab} and CHTC pools. Talk to the infrastructure team to arrange upgrading these pools.
*:: N >= 1 for a development series release
@@ -149,9 +115,9 @@
1: RUST-watcher must be diligent about looking for "bad things", bug reports from our users, etc.
-1: The Version History et all in the manual needs to be verified that it is all up to date. See the section on updating the version history above for details.
+1: Verify that the Version History in the manual is up to date. {link PreparingVersionHistory}
-1: The list of supported platforms in the manual needs to be verified that it is all up to date. This generally means talking to Pete Keller. The list is in =doc/overview/overview.tex= under 'Availability'. Furthermore, the platform-specific notes in the manual =doc/platforms/*= should be verified and updated if there have been any changes.
+1: Verify the list of supported platforms in the manual. This generally means talking to Pete Keller. The list is in =doc/overview/overview.tex= under 'Availability'. Furthermore, the platform-specific notes in the manual =doc/platforms/*= should be verified and updated if there have been any changes.
1: Tag Build ID w/ release tag. For example, to create "V7_0_3" as opposed to "BUILD-V7_0_3-branch-2008-6-15":
*:: (Assume clones of CONDOR_SRC.git, and CONDOR_DOC.git are present.)
@@ -176,8 +142,10 @@
*:: =git tag V7_0_3 BUILD-V7_0_3-branch-2008-6-15=
*:: =git push origin V7_0_3= (This puts the V7_0_3 tag in the central CONDOR_DOC.git repo.)
-1: Pre-release binary tarballs to the web.
-The binaries are not available to the public until they've been pushed to the mirrors (see step 14). However, they still need to be placed into AFS for the download system to find. If the version of Condor is =X.Y.Z=, then place the binary releases of Condor for the various available architectures into =/p/condor/public/binaries/vX.Y/X.Y.Z=. Note that for any particular architecture, =unstripped= binaries should not be released to the web if there is a corresponding =condordebugsyms= tarball. And with 7.5.x and above, static binaries should not be released.
+{section: Release Day}
+
+1: Release binary tarballs to the web.
+Releasing the binaries to the public is a two step process. First, we put the binaries into AFS at =/p/condor/public/binaries/vX.Y/X.Y.Z=. Then, we invoke the download system scripts to push these binaries from AFS out to the mirrors (this is a later step in these instructions).
*:: WINDOWS: With <Current version +1> hopefully Windows packages will be automatic. If you have only the tarballs and not the Windows MSI/ZIP, you can build them from the tarballs. People who know how to make Windows packages: TJ, Cathrin, Z, Todd T.
1::: Unzip/untar the windows tarball into a working directory, c:\scratch\temp for instance. You will end up with all of the files in c:\scratch\temp\public
1::: Create a directory for the packagages, c:\scratch\temp\package for instance.
@@ -190,51 +158,12 @@
*:: UNIX: Mostly this involves copying everything off of nmi-s006 and into /p/condor/public/binaries/...
-1:::: For each condordebugsyms-* file, delete any corresponding * -unstripped.tar.gz". That is, for any given platform there should be a * -unstripped.tar.gz _or_ a condordebugsyms-*.
-1:::: Move everything into place. This could take thirty minutes or so. We
-use cp here to make it easier on you when you inevitably fill up the AFS
-partition onto which this release goes.
-{code}
-cp * /p/condor/public/binaries/stable/v7.4/7.4.0/
-{endcode}
+FILL THIS IN
-*::: Option 2: Use Nick's shiny =condor_nmi_extract= Python script, which lives in =CONDOR_SRC/nmi_tools=.
-{linebreak}
-The script writes to ./cache and ./public -- and will create these directories if they don't already exist. The cache is to maintain a list of things that it's already extracted from, so if you run it again, it won't do unnecessary work.
-{linebreak}
-Run the python in /prereq/Python-2.6.2/bin (put it in the front of your path). Use "-h" to get help.
-*:::: login to nmi-s006
-*:::: Get the GID or the (RunID and date of the build) of the build you want.
-*:::: Run the extractor, passing it either a GID or run ID -- if it sees a "_" in the ID, it'll assume it's a GID.
-*::::: Provide a GID:
-{code}
-$ nmi-extract-results <GID>
-$ nmi-extract-results 1276577428_21704
-{endcode}
-*::::: Provide a runid and look for builds within +/- 24 hours of today that match the passed in RID
-{code}
-$ nmi-extract-results <RunID>
-$ nmi-extract-results 246724
-{endcode}
-*::::: Provide a runid and look for builds with +/- 24 hours of the specified date
-{code}
-$ nmi-extract-results --date <date> <RunID>
-$ nmi-extract-results --date 2010/06/16 246724
-{endcode}
-*::::: Provide a runid and extract only specific platform(s).
-{code}
-$ nmi-extract-results <RunID> <platform> ...
-$ nmi-extract-results 246724 x86_64_rhas_3 x86_rhap_5
-{endcode}
-*:::: The --runid search modes aren't all that painful at all, it generally finds the matching RunID in well less than a minute, so it's much better than the "run find by hand" method, which is painful.
-*:::: If it succeeds in finding the build, it will then extract all of the relevant binaries into ./public/v7.1.2 (well, actually, the correct name). When it's done, you should have everything you need in that directory, including the condor_src and windows tarballs (note, however, that we, as of yet, still need to have Todd, Cathrin, or Z build the ZIP and MSI files).
-*:::: Copy the files back to one of the CSL machines; I try to use nation or one of the less used machines (nation is a good choice), especially if I'm copying directly into AFS, otherwise it loads them down badly (AFS just doesn't handle it well at all).
-{code}
-$ cd public/v7.1.4
$ rsync -av --rsh=ssh * nation:/p/condor/public/binaries/v7.1/7.1.4
{endcode}
-1: Pre-release source tarball. The source tarball used to be created every night as part of the nightly builds. After the conversion to cmake, it is no longer created. While releasing 7.5.5, I created it by hand by tarring the contents of =userdir/common= in the =rundir= of the NMI build. I put these files in a top-level directory named =condor-X.Y.Z= and I removed the =soar= directory. Nick's =condor_nmi_extract= script will also create the source tarball.
+1: Pre-release source tarball. The source tarball used to be created every night as part of the nightly builds. After the conversion to cmake, it is no longer created. While releasing 7.5.5, I created it by hand by tarring the contents of =userdir/common= in the =rundir= of the NMI build. I put these files in a top-level directory named =condor-X.Y.Z= and I removed the =soar= directory.
The source should be placed in =/p/condor/public/binaries/vX.Y/X.Y.Z= using the name "=condor_src-X.Y.Z-all-all.tar.gz=".
@@ -252,13 +181,6 @@
./generate_html src
{endcode}
-
-*::Update the download system's information. Note that the warnings above download-versions being incompatible with the standard Python install stands; make sure you use a more recent Python. (Note: you can't do this until you do step #2 below, namely, defining a new release series, and adding the X.X.0 point release into the download system)
-{code}
-download-versions update --lic=http://parrot.cs.wisc.edu/v7.4.license.html \
- --inst=http://www.cs.wisc.edu/condor/manual/v7.4/3_2Installation.html 7.4.0
-{endcode}
-
*::The download log files must exist; the download scripts won't create them if they don't.
{code}
cd /p/condor/public/license
@@ -313,7 +235,20 @@
Python 2.5
{endcode}
*:: For more complete documentation on the download tools, look at /p/condor/downloads/doc/operation.txt
+
+*:: *If this is a new series*:
+
+*::: Edit /p/condor/downloads/etc/downloads.conf and add a few lines. Search for the previous series and follow its example. In particular, as a "[release 7.4] section, using a previous release as a template. Also, for each "[mirror SOMETHING]" section, add a release_7_4 entry, using previous releases as a template.
+
+*::: Update the download system's information.
+{code}
+download-versions update --lic=http://parrot.cs.wisc.edu/v7.4.license.html \
+ --inst=http://www.cs.wisc.edu/condor/manual/v7.4/3_2Installation.html 7.4.0
+{endcode}
+
+
*:: Add a version to the new download "database". In this case, we'll install version 7.0.2, and we'll copy the release information from 7.0.1, except for the release date. Then, we'll edit the 7.0.1 description to reflect that it's no longer the most recent. Finally, we'll blow away 7.0.0 entirely. These steps are required. For all of the below commands, I'm going to assume that you've added =/p/condor/downloads/bin= to your PATH.
+
*::: List the current versions:
{code}
$ download-versions print
@@ -327,24 +262,16 @@
$ download-versions update --description "Recent Stable Release" 7.0.1
{endcode}
-*:: If this is a new series, you will need to edit /p/condor/downloads/etc/downloads.conf and add a few lines. Search for the previous series and follow its example. In particular, as a "[release 7.4] section, using a previous release as a template. Also, for each "[mirror SOMETHING]" section, add a release_7_4 entry, using previous releases as a template.
-
*:: Install the binaries into the new download system. This step is required.
{code}
$ download-install -v --mode symlink 7.0.2 /p/condor/public/binaries/v7.0/7.0.2
{endcode}
-*:: Here's how you would create a disabled 7.1.3 version, install binaries for it, and then enable it later, changing the date to match the actual release
-{code}
-$ download-versions add 7.1.3 --copy 7.1.2 --date +3 --descr "Latest Developer Release" --disable
-...
-$ download-install -v --mode symlink 7.1.3 /p/condor/public/binaries/v7.1/7.1.3
-...
-$ download-versions up 7.1.3 --enable --today
-{endcode}
+
*:: Push the new binaries out to the local mirror (parrot). If we didn't specify the "--mirror parrot", it'd push to all mirrors, including INFN, and that takes quite a while. A push to parrot usually takes on the order of 50 minutes or so. The binaries will be automatically pushed to all mirrors every AM, but this allows us to get it to one mirror at least. If you're prompted for a password, enter the password of the =condor= account on CSL machines.
{code}
$ download-push -v --mode real --mirror parrot 7.0.2
{endcode}
+
*:: Generate symlinks to the binaries & push them out to the local mirror. This is also done by cron, twice per hour, so, if you didn't do this step, it'd get done for you. However, it only takes a few minutes, so I included it here.
{code}
$ download-gen-symlinks -v --mirror parrot
