This document gives a high level overview of the software engineering process of HTCondor. This HTCondor project employs many of the best practices in software engineering. Many of the aspects of the process resemble to Agile Methodologies.
As an open source project, the source code repository and issue tracking database is accessible online by the public. Anyone can read the source and development documentation.
The code for HTCondor is kept in a git repository that is mirrored on github. Developers clone the repository and work locally. Much wisdom can be found in ManagingCondorSourceTreesWithGit.
Every commit to the source code must have the ticket number in the commit comment. The form is #1234. Do not start a line with #. It will be interpreted as a comment.
The HTCondor groups maintains two series of releases. The (odd numbered, e.g. 7.9.x) development releases and the (even numbered, e.g. 7.8.x) stable releases. The development releases have new features that may destablize and installation. The stable series contains bug fixes. These are release on a monthly cadence. Security updates may be release outside of the monthly to address urgent problems.
One can look at the ReleaseHistory for recent release dates and tickets.
The master branch of the git repository contains work for the next development release. A branch is created close to release time of the developer series. Any changes made to this branch in preparation for release is merged to the master branch.
There is also a branch for the stable series. Just prior to release a branch is created for the release. This branch is stablized prior to release. Changes to this release branch are merged up the branch for the stable series and eventually merge up to the master branch.
On rare occasions, a fix in the master or stable series are cherry picked (not merged) down to a release branch.
The master branch contains work toward new features. New features and significant defect fixes reguire a design document to be reviewed and approved. Please see the DesignDoc section for more information.
The stable branch contains defect fixes and urgent security updates. For reference, look at the StablePolicy. All updates to the stable branch must be code reviewed. Take a look at PerformingCodeReviews.
The central hub of information is handled by GitTrac (previously known as CVSTrac). This amazing piece of software contains houses the wiki and ticket tracking system. This tool is powered by a SQLite database and complex queries can be constructed to extract information. Take a look at some of the reports in ReportList
The ticket tracking system tracks all the work related to HTCondor.
The FixedVersion of the ticket is the target release for this ticket. It is predicitive. Tickets that do not complete before a release are automatically moved to the next release. Design docs are attached to tickets. Also, stable series tickets must have a CODE REVIEW remark.
The tickets are prioritized according to the following scheme.
- Critical: fix in the current release
- Major: fix within the next few releases
- Minor: fix be the end of the development series
- Unclassified: Not yet triaged and ranked
- Future: ideas that may be implemented in the future
Some tickets are marked private (the ticket number ends with 'p'). Tickets for security related issues that may undermine installed sites are marked private.
Every ticket that affects the code base is supposed to have an accompanying test. Fixed defects have a test that ensures that the defect is no longer present. Enhancements have test(s) to verify that the new feature is working properly.
The test coverage for the automated test is unknown at this time.
Continuous Integration Builds
After each check-in, the HTCondor software is built and testcases are run against the build on the BaTLab (Build and Test Lab). The results on the build can be found at submit-3.batlab.org/results/. The condor group is the exclusive user of submit-3.batlab.org. All other users of the BaTLab use submit-1.
The continuous integration builds help determine which check-in broke some functionality.
The release wrangler checks-in updates to set the proper version number, which causes a build to be kicked off. Then s/he chooses a recent Continuous Integration build that passes the tests. The output of the build is used for the release. So, the releases are binary releases.
The release process is covered by the Release Punch-List
The steps and timing for updated the CHTC pool are outlined in the following flow chart HTCondor Upgrade
The users of the CS and CHTC pools are expected to test the software through normal usage. There is potential weakness in depending on the users to run the software through its paces. The users may be test new features or stress the system.
The condor system is documented in the doc subtree of the source tree. Changes to the documentation are required to be committed when the source changes are committed. The documentation tree contains the LaTeX source for user and administration guides are well as all the man pages.
The current documentation for HTCondor can be found online at http://research.cs.wisc.edu/htcondor/manual/