Page History

Turn Off History

Read the Docs Overview

In 2019 we migrated the HTCondor Manual from our old LaTeX format into the Sphinx documentation generator using the reStructuredText format. We will also be hosting it on Read the Docs. The manual can be found here:

http://htcondor.readthedocs.io

Prerequisites

  1. Install Python. You must install Python 3+. We no longer support Python 2.
    • https://www.python.org/downloads
    • Make sure your $PYTHONPATH environment variable is set to include the Condor Python bindings. In the following example, replace the <htcondor-release-dir> tag with the correct path:
      export PYTHONPATH=<htcondor-release-dir>/lib/python3:$PYTHONPATH
      

  2. Install Sphinx and related support packages. The recommended way to install Sphinx is via `pip`:
    pip3 install --user -r docs/requirements.txt
    
    On RHEL and CentOS we do not support installing Sphinx from yum or RPM. The available package is an old version which does not support all our extensions. For other Linux distributions, Windows and macOS, you can find instructions here:

  3. Install pandoc. On RHEL, for example:
    sudo yum install pandoc
    
    Windows and Mac installers can be found at https://github.com/jgm/pandoc/releases/

  4. Install the HTCondor Python bindings. This could be via pip (make sure you get the right version, e.g. pip3 install htcondor==8.8.1), or by editing your PYTHONPATH environment variable to point at the .so files inside your HTCondor install. Note: on RTD itself, the docs are built using the htcondor version read out of CMakeLists.txt

  5. Install the Python pathlib module if you're on Python <= 3.4:
    pip3 install --user pathlib
    

  6. [Optional] Install https://pypi.org/project/sphinx-autobuild/. This is useful if you're making lots of quick edits to the manual and expect to build it many times.
    pip3 install --user sphinx-autobuild
    
    See below for instructions on using sphinx-autobuild.

Building

Building and previewing the manual locally

Using sphinx-autobuild to build and preview

Instead of the instructions in the previous section, go to /docs and run

sphinx-autobuild . _build/html
You will see a log of the Sphinx build running, and eventually will be provided a link to a localhost webserver hosting the docs. Leave this program running: sphinx-autobuild will watch the docs source tree for changes and rebuild when it detects changes (you may still need to manually reload the page inside your web browser, but it will usually force a refresh by itself).

Different versions of the manual

Editing

The manual now uses the reStructuredText (rST) format, which is similar to Markdown markup but considerably more powerful. A helpful reference to reStructuredText is available here:

http://docutils.sourceforge.net/docs/user/rst/quickref.html

In addition, the new manual also uses the Sphinx documentation generator. Sphinx does many useful things such as:

A full Sphinx reference is available here:

http://www.sphinx-doc.org/en/master/contents.html

This section provides some markup style guidelines, as well as information about how we use both built-in and custom tools.

Section Titles

Section titles are very fluid in rST and there are many different ways to make them. To keep things as consistent as possible, try to use the following:

Page titles get underlined with the = symbol
============================================

Section titles get underlined with the - symbol
-----------------------------------------------

Subsection titles get underlined with the ' symbol
''''''''''''''''''''''''''''''''''''''''''''''''''

Indentation

The rST format is very sensitive to indentation. Paragraphs and other blocks of text are expected to be left-aligned. Indenting a block by any amount of whitespace (compared to the preceding block) causes it to get indented.

This is a top-level block of text. It will appear aligned to the left-most side of the page.

 This paragraph is indented by one space. Even though it's only a single space,
 it will render as a full first-level indent.

  This paragraph is indented by one more space than the one above it. As a
  result it will render as a second-level indent.

    This time I've indented a block by two more spaces the one above it. It
    doesn't matter that this is inconsistent with the single-space indents
    above. This block will render as a third-level indent.

Back to the top level!

            This block is indented by 12 spaces. However, as with the previous
            examples, the amount of whitespace doesn't matter. Because it's the
            first indented block compared to the preceding block, it will only
            render as a first-level indent.

Linking to gittrac tickets

Use the following syntax to automatically link to a gittrac ticket, where #### is the number of the ticket:

:ticket:`####`

Adding index entries

To add a basic index entry, use the following syntax:

:index:`Name of index entry`

If you want your index entry to appear under a parent entry, the syntax is a little more complicated:

:index:`Name of index entry <single: Name of index entry; Name of parent entry>`

Linking to internal documents

To add a link to an internal document, the syntax looks like :doc:`/path/to/page-title`. For example, to link to the Overview > Exceptional Features section, add the following:

:doc:`/overview/exceptional-features`

By default, the link text will be the name of the page. If you want to add custom text, it looks something like the following:

:doc:`Here is my custom text link </overview/exceptional-features>`

Linking to subsections within documents

You can also easily add links to subsections within documents. Whenever a section or subsection is defined using the correct section title syntax (see "Section Titles" above), Sphinx will automatically make the an HTML anchor so you can link directly to it. Use the following syntax:

:ref:`path/to/document:section title`

For example, to link to the "Start an Annex" subsection on the Annex User's Guide page, use the following:

:ref:`cloud-computing/annex-users-guide:start an annex`

By default, Sphinx will use the name of the section as the link text. You can override this with the following syntax:

:ref:`My custom link text goes here<path/to/document:section title>`

Documenting Python Objects

Python "objects" (classes, methods, free functions, enums, anything) are documented via sphinx-autodoc (https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html). "Docstrings" for these objects are written directly into the Python bindings C++ source code, are embedded into the Python library during the HTCondor build process, and are then read by Sphinx during the manual build.

As an example, a method declaration (adding a method to a class) looks like this in the C++ source code:

.def("queue_with_itemdata", &Submit::queue_from_iter,
            R"C0ND0R(
            Submit the current object to a remote queue.

            :param txn: An active transaction object (see :meth:`Schedd.transaction`).
            :type txn: :class:`Transaction`
            :param int count: A queue count for each item from the iterator, defaults to 1.
            :param from: an iterator of strings or dictionaries containing the itemdata
                for each job as in ``queue in`` or ``queue from``.
            :return: a :class:`SubmitResult`, containing the cluster ID, cluster ClassAd and
                range of Job ids Cluster ID of the submitted job(s).
            :rtype: :class:`SubmitResult`
            :raises RuntimeError: if the submission fails.
            )C0ND0R",
            (boost::python::arg("self"), boost::python::arg("txn"), boost::python::arg("count")=1, boost::python::arg("itemdata")=boost::python::object())
            )
Note the use of a raw string delimited by C0ND0R for the docstring itself. The syntax is described here: https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html

The corresponding .rst source that would embed this docstring, prettily-formatted, in the manual looks like

.. autoclass:: Submit

   .. automethod:: queue
   .. automethod:: queue_with_itemdata
   .. automethod:: expand
   .. automethod:: jobs
   .. automethod:: procs
   .. automethod:: itemdata
   .. automethod:: getQArgs
   .. automethod:: setQArgs
   .. automethod:: from_dag
The .. auto<something>:: are Sphinx directives provided by sphinx-autodoc which are replaced by the autodoc-formatted descriptions. So this block produces all of the documentation for the Submit object.

When documenting a new thing in the bindings, you must also add an appropriate .. auto<something>:: in the appropriate .rst.

Publishing

Publishing the manual onto Read the Docs

Different versions of the manual

Generating man pages