{subsection: Final Release Process}
-Yet to be imported...
+1: Settle on a build ID candidate that can build on all platforms
+
+1: On nmi-s006, pin the build for 120 days using =/nmi/bin/nmi_pin=:
+{code}
+$ nmi-pin --days=120 --force <runid>
+{endcode}
+
+1: Make sure we've got a win32 build and releasable package (at this point, the nightly windows builds still don't provide packaging, so this remains a manual step).
+
+1: Definitive pass-thru of test results, by platform. Need a list by platform saying "good" or "bad".
+
+1: Deploy binaries onto production pool, start the "N-day" counter.
+This is a complicated process in itself, which is documented on the {link: http://www.cs.wisc.edu/condor/developers/pool-update.html Upgrading the UW CS Pool} page.
+*:: N >= 1 for a development series release
+*:: N >= 3 for a bug fix release in a stable series (X.Y.>0)
+*:: N >= 28 for a new stable series (X.Y.0)
+
+1: RUST-watcher must be diligent about looking for "bad things", bug reports from our users, etc.
+
+1: The Version History et all in the manual needs to be verified that it is all up to date. See the section on updating the version history above for details.
+
+1: The list of supported platforms in the manual needs to be verified that it is all up to date. This generally means talking to Pete Keller. The list is in =doc/overview/overview.tex= under 'Availability'. Furthermore, the platform-specific notes in the manual =doc/platforms/*= should be verified and updated if there have been any changes.
+
+1: Tag Build ID w/ release tag. For example, to create "V7_0_3" as opposed to "BUILD-V7_0_3-branch-2008-6-15":
+*:: (Assume clones of CONDOR_SRC.git, and CONDOR_DOC.git are present.)
+*:: =cd CONDOR_SRC=
+*:: =git tag V7_0_3 BUILD-V7_0_3-branch-2008-6-15=
+*:: =git push origin V7_0_3= (This puts the V7_0_3 tag in the central CONDOR_SRC.git repo.)
+*:: *Note:* Push the TAG name, not the branch name.
+*:: =git checkout V7_0_3-branch=
+*:: modify =src/CVS_Tags= on V7_0_3-branch to document the new tag. This must include the following information:
+*::: Name of the tag and date the tag was created.
+*::: The branch the tag was created on.
+*::: The official version string included in that release.
+*::: The original build tag that the release tag was created from. This last point is very important, since it contains the actual date the release's codebase came from, not just the date the release tag "alias" was created. This is important for date-based comparisons in git, for example.
+*::For example, here's the 6.9.5 tag's entry:
+{code}
+11/28/07 V6_9_5
+ The offical tag for 6.9.5, from the V6_9_5-branch.
+ Version string: $CondorVersion: 6.9.5 Nov 28 2007 BuildID: 65347 $
+ Original build tag: BUILD-V6_9_5-branch-2007-11-28
+{endcode}
+Note that in this case, everything was actually from the same date, but that's an exception. Usually, the date in the version string and original build tag will differ from the date the release tag itself was added.
+*:: =git commit -a=
+*:: =git push origin V7_0_3-branch= (This pushes the change to src/CVS_Tags.)
+*:: =cd ../CONDOR_DOC=
+*:: =git tag V7_0_3 BUILD-V7_0_3-branch-2008-6-15=
+*:: =git push origin V7_0_3= (This puts the V7_0_3 tag in the central CONDOR_DOC.git repo.)
+
+1: Pre-release binary tarballs to the web.
+The binaries are not available to the public until they've been pushed to the mirrors (see step 14). However, they still need to be placed into AFS for the download system to find. If the version of Condor is X.Y.Z, then place the binary releases of Condor for the various available architectures into /p/condor/public/binaries/vX.Y/X.Y.Z
+*:: UNIX: Mostly this involves copying everything off of nmi-s006 and into /p/condor/public/binaries/...
+*::: Option 1: Use Nick's shiny "nmi-extract-results" script, which lives in tools/nmi-interaction. It's also in ~nleroy/bin on nmi-s006.
+{linebreak}
+The script writes to ./cache and ./public -- and will create these directories if they don't already exist. The cache is to maintain a list of things that it's already extracted from, so if you run it again, it won't do unnessesary work.
+{linebreak}
+I included help below because the Python on nmi-s006 is old & crusty, and --help doesn't work properly.
+{code}
+nmi-extract-results -h
+Usage: nmi-extract-results: [options]
+
+Options:
+ --version show program's version number and exit
+ -h, --help show this help message and exit
+ -d DATE, --date=DATE Set date yyyy/mm/dd|yy/mm/dd|mm/dd <2008/11/12>
+ -r RUNID, --runid=RUNID
+ Specify runid (slow); example: 113473
+ -g GID, --gid=GID Specify GID; example: 1225823210_6802
+ --condor-version=VERSION
+ Specify condor version (i.e. '7.1.4')
+ --find Exit after locating the run directory
+ -n, --no-exec Disable execution
+{endcode}
+*:::: login to nmi-s006
+*:::: Get the GID or the (RID and date of the build) of the build you want.
+*:::: Run the extractor in one of the following forms:
+*::::: This version takes a GID
+{code}
+$ nmi-extract-results --gid <GID>
+{endcode}
+*::::: This version takes an RID and looks for builds within +/- 24 hours of today that match the passed in RID
+{code}
+$ nmi-extract-results --rid <RID>
+{endcode}
+*::::: This version takes an RID and looks for builds with +/- 24 hours of the specified date
+{code}
+$ nmi-extract-results --rid <RID> --date <date>
+{endcode}
+*:::: The --rid search modes aren't all that painful at all, it generally finds the matching RID in well less than a minute, so it's much better than the "run find by hand" method, which is painful.
+*:::: If it succeeds in finding the build, it will then extract all of the relevant binaries into ./public/v7.1.2 (well, actually, the correct name). When it's done, you should have everything you need in that directory, including the condor_src and windows tarballs (note, however, that we, as of yet, still need to have Ben build the ZIP and MSI files).
+*:::: Copy the files back to one of the CSL machines; I try to use nation or one of the less used machines, especially if I'm copying directly into AFS, otherwise it loads them down badly (AFS just doesn't handle it well at all).
+{code}
+$ cd public/v7.1.4
+$ rsync -av --rsh=ssh * nation:/p/condor/public/binaries/v7.1/7.1.4
+{endcode}
+
+*::: Option 2: Do it by hand:
+*:::: for a given build's GID, run the following on nmi-s006:
+{code}
+GID=cndrauto_nmi-s006.cs.wisc.edu_1167884128_28240
+RELEASE=6.8.3
+cd /space/tmp
+mkdir release-$RELEASE
+cd release-$RELEASE
+find /nmi/run/cndrauto/2008/*/$GID/userdir -type f -name results.tar.gz | grep -v '/common/' | grep -v '/nmi:x86_winnt_5.1/' | xargs -n1 tar xvzf
+{endcode}
+*:::: Get some coffee, it'll be a while.
+*:::: On a CSL machine:
+{code}
+RELEASE_MAJOR=6.8
+RELEASE=6.8.3
+cd /p/condor/public/binaries/v$RELEASE_MAJOR
+mkdir $RELEASE
+cd $RELEASE
+rsync -v --progress --rsh=ssh nmi-s006:/space/tmp/release-$RELEASE/public/v$RELEASE_MAJOR/'*' ./
+ls | grep -v unstripped | xargs --replace mv '{}' ../
+{endcode}
+*:::: (You can also use "=scp nmi-s006:/space/tmp/release-$RELEASE/public/v$RELEASE_MAJOR/'*' ./=" to copy the binaries over. rsync provides the advantage that you can re-run it over and over again if your transfer is interrupted and you won't re-copy what you already have.)
+*::: Option 3: Derek has a script to search through a given NWO rundir and move everything we need. The script lives in the nmi_tools directory in CVS, and is called =move_to_afs=. Beware that this script will eat up around 8 gigabytes while it runs.
+*:: WINDOWS: With 6.7.18 hopefully Windows packages will be automatic. Until that's working, Greg Quinn is manually creating the packages and will probably just copy it into AFS directly.
+
+1: XXX This step needs a refinement in the script intellect. XXX
+{linebreak}{linebreak}
+Create matching glidein tarballs, and install in /p/condor/public/html/glidein/binaries. This process is handled by running the following script, replacing 7.1.1 with the version you are releasing:
+{code}
+make_glidein_tarballs 7.1.1
+{endcode}
+This script is pretty crude, and it'll only work with verion numbers (and directory names) of the form x.y.z, not vx.y.z. It puts the output files in /p/condor/public/binaries/glidein .
+The script expects to find the condor build tarballs either in the normal release directory in AFS or in a subdirectory of it named with the version numbers of the release. If anything goes wrong, ask Dan for help.
+
+1: Release source tarball. The source tarball is created every night as part of the nightly builds. Nick's =nmi-extract-results= script will extract it along will all of the binary tarballs. Otherwise, you can extract it yourself. If you look in the =rundir= of a given NMI runid, you should see a tarball in =userdir/common= named =results.tar.gz=, which contains the source tarball. Extract it like so:
+{code}
+tar zxvf results.tar.gz condor-X.Y.Z.tar.gz
+{endcode}
+The source tarball should be placed in =/p/condor/public/cgi-bin/source/tarballs=, and it should be named "=condor_src.VX_Y_Z.tar.gz=". You must run =/p/condor/public/cgi-bin/source/make_links.pl= after copying the the source tarball into place. The source should also be placed in =/p/condor/public/binaries/vX.Y/X.Y.Z= using the name "=condor_src-X.Y.Z-all-all.tar.gz=".
+
+1: Tell the new download system about the new binaries.
+*:: All of the programs listed here take --help options, and the are all discussed to some extent in /p/condor/downloads/doc/operation.txt .
+*:: These tools require Python version 2.5. The /s/std/bin/python on Centos 4.5 is 2.4.1, and these scripts won't run with it. If that's the case, add /s/python-2.5/bin/ to the front of your PATH. You can detect the version of the python that you're running by:
+{code}
+$ which python
+/s/std/bin/python
+$ python -V # Note that this is a capital "V".
+Python 2.4.1
+$ /s/std/bin/python -V
+Python 2.4.1
+$ /s/python-2.5/bin/python -V
+Python 2.5
+$ export PATH=/s/python-2.5/bin:$PATH
+$ rehash
+$ which python
+/s/python-2.5/bin/python
+$ python -V
+Python 2.5
+{endcode}
+*:: For more complete documentation on the download tools, look at /p/condor/downloads/doc/operations.txt
+*:: Add a version to the new download "database". In this case, we'll install version 7.0.2, and we'll copy the release information from 7.0.1, except for the release date. Then, we'll edit the 7.0.1 description to reflect that it's no longer the most recent. Finally, we'll blow away 7.0.0 entirely. These steps are required. For all of the below commands, I'm going to assume that you've added =/p/condor/downloads/bin= to your PATH.
+*::: List the current versions:
+{code}
+$ download-versions print
+{endcode}
+*::: Create the 7.0.2 version:
+{code}
+$ download-versions --copy 7.0.1 --date 2008/06/10 add 7.0.2
+{endcode}
+*::: Update the 7.0.1 description:
+{code}
+$ download-versions update --description "Recent Stable Release" 7.0.1
+{endcode}
+*::: Delete the 7.0.0 release. The first step "download-delete" will delete the files maintained by the download subsystem. The second step "download-versions delete" will delete the version from the version database. So, do the following:
+{code}
+$ download-delete 7.0.0
+$ download-versions delete 7.0.0
+{endcode}
+
+*:: If this is a new series, you will need to edit /p/condor/downloads/etc/downloads.conf and add a few lines. Search for the previous series and follow its example.
+*:: Install the binaries into the new download system. This step is required.
+{code}
+$ download-install -v --mode symlink 7.0.2 /p/condor/public/binaries/v7.0/7.0.2
+{endcode}
+*:: Here's how you would create a disabled 7.1.3 version, install binaries for it, and then enable it later, changing the date to match the actual release
+{code}
+$ download-versions add 7.1.3 --copy 7.1.2 --date +3 --descr "Latest Developer Release" --disable
+...
+$ download-install -v --mode symlink 7.1.3 /p/condor/public/binaries/v7.1/7.1.3
+...
+$ download-versions up 7.1.3 --enable --today
+{endcode}
+*:: Push the new binaries out to the local mirror (parrot). If we didn't specify the "--mirror parrot", it'd push to all mirrors, including INFN, and that takes quite a while. A push to parrot usually takes on the order of 15 minutes or so. The binaries will be automatically pushed to all mirrors every AM, but this allows us to get it to one mirror at least.
+{code}
+$ download-push -v --mode real --mirror parrot 7.0.2
+{endcode}
+*:: Generate symlinks to the binaries & push them out to the local mirror. This is also done by cron, twice per hour, so, if you didn't do this step, it'd get done for you. However, it only takes a few minutes, so I included it here.
+{code}
+$ download-gen-symlinks -v --mirror parrot
+{endcode}
+
+1: Release the manual. Continuing the V7_0_3 example...
+*: =git clone /p/condor/repository/CONDOR_DOC.git=
+*: =cd CONDOR_DOC=
+*: =git checkout -b V7_0_3-branch origin/V7_0_3-branch=
+*: =cd doc=
+*: =make release=
+*: move the symlink in /p/condor/public/html/manual for whatever release series you're changing to point at the new version.
+*:: =cd /p/condor/public/html/manual=
+*:: =rm v7.0=
+*:: =ln -s v7.0.3 v7.0=
+-----
+ 15. Update the web HTML regarding this release
+ * cvs -d /p/condor/repository/HTML co condor-web
+ * cd condor-web
+ * edit src/lib/news.mas (to add an announcement 'Whats New?')
+ o Here is a sample announcement:
+
+ { date => 'January 22, 2008',
+ title => 'Condor 7.0.0 released!',
+ news => <<ENDNEWS
+ The Condor Team is pleased to announce the release of
+ Condor 7.0.0. This first release in a new stable series
+ includes all new features added in the 6.9 development
+ series. See the
+ <a href="http://www.cs.wisc.edu/condor/manual/v7.0/8_3Stable_Release.html#sec:New-7-0-0">Version
+ History</a> for a complete list of changes. Condor 7.0.0
+ binaries and source code are available from
+ <a href="http://www.cs.wisc.edu/condor/downloads/">our
+ Downloads page.</a>
+ ENDNEWS
+ },
+
+ * edit src/downloads/index.html (to fix the release-specific text on the main downloads page)
+ * touch src/index.html (so the main page updates with the version information from the downloads/index.html page)
+ * touch src/manual/index.html (so the manual links update with the version information from the downloads/index.html page)
+ * NOTE: if you're doing the first release of a new release series, you'll also need to edit:
+ o src/index.html to:
+ + change the "The Condor Manual:" list of links (chances are good you'll want to remove something older when you add the new one -- we'll never want that list to be longer than 3 links).
+ + Change where the links under Current Releases goes to. The text label for the link automatically updates, but the link doesn't.
+ o src/developers/index.html to add the new series to the "Manuals and Other Documentation" list.
+ o edit src/manual/index.html (to update which versions of each branch/release series are the current manual)
+ * cvs commit (all of your changes in condor-web)
+ * ./generate_html src (to update the HTML on the web with your changes).
+
+ 16. Archive release products to DVD. Get recordable DVDs and DVD slipcovers from Ken. Drop off the finished DVD with Pete, who will store it in a box in his filing cabinet. Include the following (the archiving directions include these):
+ * Binary tarballs. This includes archiving the unstripped binaries.
+ * Matching glidein tarballs. Get these from /p/condor/public/binaries/glidein .
+
+ $ ssh tonic "cd /p/condor/public/binaries; tar cf - glidein/7.1.3" | tar xvfpS -
+
+
+ * Source tarball.
+ * Manual: tarball of HTML tree, and ref.pdf . The HTML files are in /p/condor/public/html/manual . Do something like this:
+
+ $ ssh tonic "cd /p/condor/public; tar cf - html/manual/v7.1.3" | tar xvfpS -
+ $ ssh tonic "cd ; tar cf - doc" | tar xvfpS -
+
+
+ 17. Send email to condor-world and condor-users. Here is a sample announcement:
+
+ Hello,
+
+ The Condor Team is pleased to announce the release of Condor 7.0.0. This first
+ release in a new stable series includes all new features added in the 6.9
+ development series. See the Version History for a complete list of changes.
+ Condor 7.0.0 binaries and source code are available from our Downloads page.
+
+ Version History:
+ http://www.cs.wisc.edu/condor/manual/v7.0/8_3Stable_Release.html#sec:New-7-0-0
+
+ Downloads Page:
+ http://www.cs.wisc.edu/condor/downloads/
+
+
+ Thank you for your interest in Condor!
+
+
+ - The Condor Team
+
+ 18. Close out all Gnats problem reports fixed by this release.
+ 19. If this is a stable series, update the Condor wikipedia page to indicate the current stable release.
+ 20. Stop building the branch nightly:
+ * cd /p/condor/home/tools/cvs-tools
+ * Edit create_build_tag-input
+ * Delete the section that defined the nightly build tag for this branch.
+ * cvs diff create_build_tag-input (to verify the change before you commit it)...
+ * cvs commit -m "added nightly build tags for V6_X_Y-branch" create_build_tag-input
