{section: Overview }
HTCondor currently provide native packages in Debian and RPM format. Location of the repositories are listed below: {linebreak}

http://www.cs.wisc.edu/condor/yum {linebreak}
http://www.cs.wisc.edu/condor/debian

{section: Background }
Providing package via repository offers many advantages. Firstly, many OS distributions automatically detect package change and notify users to update. Secondly, package management programs such as yum or apt-get automatically resolve dependency

{section: Repository Layout}
There are 3 separate repositories: stable, development and testing (pre-release testing repo).

{subsubsection: Yum }
{code}
/p/condor/public/html/yum/stable/rhel5/*.rpm
/p/condor/public/html/yum/stable/rhel4/*.rpm
/p/condor/public/html/yum/development/rhel5/*.rpm
/p/condor/public/html/yum/development/rhel4/*.rpm
{endcode}

{subsubsection: Debian }

{code}
/p/condor/public/html/debian/stable/dists/lenny/contrib/binary-i386/*.deb
/p/condor/public/html/debian/stable/dists/lenny/contrib/binary-amd64/*.deb
...
/p/condor/public/html/debian/development/dists/etch/contrib/binary-i386/*.deb
/p/condor/public/html/debian/development/dists/etch/contrib/binary-amd64/*.deb
...
{endcode}

{section: Manually Managing Repositories }
{subsection: Yum }

{subsubsection: Add/remove packages }
1: Go to a specific branch such as /p/condor/public/html/yum/stable/rhel5/ and add/remove package from the directory.
2: Invoke createrepo to update the repo index.
{code}createrepo /p/condor/public/html/yum/stable/rhel5/{endcode}

{subsubsection:  Add a new OS distribution }
1: Create new branch such as /p/condor/public/html/yum/stable/rhel6/ and follows the same step as add/remove package.

{subsection: Debian }

{subsubsection: Add/remove packages }
1: Add/remove package from its branch such as /p/condor/public/html/debian/stable/dists/lenny/contrib/binary-i386
2: Set appropriate environment variable for invoking repository utility
{code}export PATH=$PATH:/p/condor/public/html/debian/stable/dists/lenny/contrib/binary-i386/
export PERL5LIB=$PERL5LIB:/p/condor/workspaces/vdt/debian-build-tools/lib/perl5/vendor_perl/5.8.8 {endcode}
3: Go to the target folder and invoke dpkg-scanpackages to generate index file
{code}dpkg-scanpackages --arch i386 --multiversion . "/dev/null" > Packages {endcode}
4: Create compressed version of index file (apt-get or synaptic will download only the compressed version)
{code}gzip -9c Packages > Packages.gz
bzip2 -9k Packages {endcode}

{subsubsection: Add a new OS distribution }
1: Copy entire distribution for existing one
{code}cp -r /p/condor/public/html/debian/stable/dists/lenny /p/condor/public/html/debian/stable/dists/squeeze {endcode}
2: Remove all packages (.deb) and index files (Packages.gz, Packages.bz)
3: Edit archive tag in the Release file. Possible values are: stable, oldstable, testing or unstable. Release file is used for APT pinning.
4: Follows Step 2 of add/remove packages

{section: Automate Script for Managing Repositories }
{subsection: Related Files }
|*File*|*Description*|
|/p/condor/public/html/yum/files/manage_repo.pl	|Script |
|/p/condor/public/html/yum/files/mapping.txt	|Config file|

{subsection: Overview }
The main goal of this script is to facilitate the release process. The script accepts NMI RUNID as a parameter. It uses the same logic as nmi_crowbar to query URLs from NMI database of a given RUNID. Then it will read the mapping.txt which provides mapping between NMI platform and platform name on each repo in order to download packages and put them into the correct location. Finally, it will update the repositories' index files.

In normal mode of operation 1-2 GB should be enough for scratch disk requirement because it only download and extract one package at time.

{subsection: mapping.txt Format }

|*NMI_PLATFORM*	        |*TYPE*	        |*ARCH*         |*TARGET_OS*|
|ia64_rhas_3		|RPM		|ia64		|rhel4|
|x86_deb_5.0		|DEB		|i386		|lenny|

*: The first column specifies a NMI platform name.
*: Type must be DEB or RPM
*: ARCH is the architecture string.For RPM, it should match the arch string in RPM's name. For Debian, it must match Debian-style arch string (i386, ia64, alpha, amd64, .. )

Note that we use packages built on RHEL3 on RHEL4.  The script will only download packages from platforms listed in the mapping.

_The script is not capable of add a new OS distribution via mapping.txt. Please use the manual method to add a new OS distribution into a repository before add new mapping in mapping.txt_

{subsection: Script Usage}
1: To download packages from a given RUNID, use /scratch/kooburat as scratch folder and push them into the stable repos. It will also update the repos when finishes.

{code}manage_repo.pl -i 24009 -s /scratch/kooburat -b stable{endcode}

2: To download packages from a given RUNID into scratch folder.

{code}manage_repo.pl -i 24009 -s /scratch/kooburat  {endcode}

3: To remove a specific version of package from a given set of repos.

{code}manage_repo.pl -b development -r 7.3.1-1 {endcode}

{section: Using Repositories }
{subsection: Stable and Development Repositories }
Please see below for detailed instructions {linebreak}
http://www.cs.wisc.edu/condor/yum {linebreak}
http://www.cs.wisc.edu/condor/debian

{subsection: Internal Testing Repositories }
To test pre-release packages. Testing repositories are created for both Debian and Yum repository. Please use the instruction below to access these repositories. {linebreak}
_Note: Access to these repositories is limited only to wisc.edu network._ {linebreak}

{subsubsection: Yum }
Use the following repo files (Depends on platform) {linebreak}
{code}
http://www.cs.wisc.edu/condor/yum/repo.d/condor-testing-rhel4.repo
http://www.cs.wisc.edu/condor/yum/repo.d/condor-testing-rhel5.repo {endcode}

{subsubsection: Debian }
Add the following lines to _/etc/apt/sources.list_ (Depends on platform)
{code}deb http://www.cs.wisc.edu/condor/debian/testing/ etch contrib
deb http://www.cs.wisc.edu/condor/debian/testing/ lenny contrib{endcode}

{section: Download Statistics }
We use CSL's facility to capture the download statistic on these repositories. A weekly statistic email will be sent to the addresses listed in these files.

/p/condor/public/html/yum/.statinfo {linebreak}
/p/condor/public/html/debian/.statinfo {linebreak}

Please see below documentation for more detail. {linebreak}
http://www.cs.wisc.edu/twiki/bin/view/CSDocs/WebServerFAQs#How%20Do%20I%20Get%20Statistics%20For%20My%20W