Building Documentation

Introduction

Two types of Mitto documentation can be built: unversioned and versioned. Unversioned documentation is built from whatever files are present in the repo at the time the documentation is built (i.e., the currently checked out branch +/- any changes present).

Versioned documentation is a set of one or more bodies of documentation, each associated with a release number. During the build process, each version of the documentation is checked out from git, the HTML is created, and it is added to the set of versioned documentation. The overall set of documentation has a pulldown that allows the viewer to choose the documentation version that is displayed.

Generally, when creating new content or revising existing content, one should work with single-version documentation as it is much faster than multi-version. Only after all changes have been reviewed and one is ready to publish the changes is it necessary to build multi-version documentation.

Setup

Zuar’s zsphinx docker image is used to build these documents. Two software packages must be installed on your local system to work with these documents:

  1. Docker - The easiest path to this is to install Docker Desktop:

  2. Python3 - Any Python3 is fine; no external dependencies need to be installed and a virtual environment is not required.

  3. Clone this git repository locally: git clone git@github.com:zuarbase/mitto-docs.git

To confirm that everything is working, the following commands will pull, build, and serve a single version of the documentation:

cd mitto-docs
git checkout master
git pull
make html
make serve

View documentation in your browser at http://0.0.0.0:8000/

At this point, you should be able to successfully build and view documentation on your local system.

Publishing of the documentation that is built is done via the rsync command. If you wish to publish documentation, you will need to place credentials for licensing.zuar.com in your ~/.netrc file. The credentials will occupy a sigle line in the file and will look something like this:

machine licensing.zuar.com login admin password password-goes-here-but-this-isnt-it

Building Unversioned Documentation

Important

This process builds the current contents of the mitto-docs directory, whether or not it is checked in to the git repository or not.

  1. Ensure the repo is up to date:

    cd mitto-docs
git checkout master
git pull

  2. Create a branch for your changes:

    git checkout -b changes-for-3.4.1

  3. Cleanup any stale documentation:

    make clean

  4. Make your changes.

  5. Build your documentation:

    make html

    The HTML and other files create by building the documentation is placed in build/html. E.g.:

    $ tree -L 1 build/html
build/html
├── _images
├── _sources
├── _static
├── api
├── architecture
├── build-log.html
├── connectors
├── databases
├── genindex.html
├── index.html
├── jobs
├── json_schemas
├── objects.inv
├── reference
├── release_notes
├── search.html
├── searchindex.js
├── sitemap.xml
├── todo-links.html
├── todo-list.html
├── todo.html
├── user-guides
└── user-interface

  6. View your changes:

    make serve

    View content at http://0.0.0.0:8000

  7. Iterate over the previous three steps until the changes are as desired.

  8. Commit your changes to git via git commit.

    Important: Do not check the contents of build/ into the repository. The HTML, etc., is always rebuilt; only documentation source is controlled.

  9. Merge your branch into master

  10. Optional - tag the merge with a release number, e.g.:

    git tag -m 3.4.1 3.4.1

    If you create a tag locally, be sure to push it to github via, e.g., git push origin 3.4.1.

    See more about version tagging in the next section.

Building Versioned Documentation

Important

Versioned documentation is built solely from the contents of one or more git commits. The contents of the current directory is ignored during the build process, with two exceptions – settings.toml and conf.py in the current directory are used to control the build of the documentation (regardless of whether they are checked-in to the repo or not).

Defining What to Build

A single file controls which documentation is built: settings.toml. Example contents:

# This file controls documentation builds, particularly versioned documentation.
#
# NOTE: If you make changes here, you likely also need to change
#       version_info in conf.py as well.
#
# TODO:
# 1. One of MITTO_VERSION and MITTO_REL should be eliminated.

# The "current" product version.  This will almost always be the most recent version.
# This value is used as the version number in the generated documentation.
MITTO_VERSION="3.4.0"

# The remaining values in this file control the behavior of sphinx-multiversion
# See the documenation elsewhere that describes what each variable does and how
# to set it.

# Tags for which documentation will be built (regex)
MITTO_TAGS='(3.3.[045]|3.4.0)'

# Branches for which documentation will be built (regex)
# If you only wish to build documentation for tags and omit all branches,
# use a value that won't match any branches.
MITTO_BRANCHES='matches-no-branches'

# Defines branches and tags that will be marked as "released" (regex).
# Anything that is built, but does not match, will be marked "development".
MITTO_RELEASED='^refs/tags/(3.3.[045]|3.4.0)$'

# String (not regex) that specifies the release to which the "latest" link will redirect to.
MITTO_LATEST='3.4.0'

# String (not regex) is a Jira release number for which to create release notes.
# Only used when building release notes.
MITTO_REL='3.4.0'

# Controls contents of version pulldown in top right of page.
# Contents is a multiline string that defiles which versions appear
# in the pulldown as well as what they link to.  Ths string must be
# valid JSON.  The Python value of the string is passed to the
# sphinx_material theme.
MITTO_VERSION_INFO='''{
   "2.11.2":     "https://www.zuar.com/api/mitto/2.11.2/index.html",
   "3.2.18":     "https://www.zuar.com/api/mitto/3.2.18.post1/index.html",
   "3.3.0":      "/docs/runner/3.3.0/index.html",
   "3.3.4":      "/docs/runner/3.3.4/index.html",
   "3.3.5":      "/docs/runner/3.3.5/index.html",
   "3.4.0":      "/docs/runner/3.4.0/index.html"
}'''

Output of Build

When multiple versions of documentation are built, they are built into docs/build/html. Each version is contained in its own subdirectory. E.g.:

$ tree -L 1 build/html
build/html
├── 3.3.0
├── 3.3.4
├── 3.3.5
├── 3.4.0
├── index.html
└── latest -> 3.4.0

A special index.html is built an placed in docs/build/html; it automatically redirects browsers to the “current” version of the documentation. Links in the sidebar of each page display the version currently being viewed and allow the user to switch documentation versions.

Walkthrough - Update Documentation for 3.4.1

The example settings.toml file will build documentation for the following versions: 3.3.0, 3.3.4, 3.3.5, 3.4.0. Let’s walk through an example of adding documentation for 3.4.1.

This example assumes that you have already followed the steps in “Building Unversioned Documentation” to create your content.

Important

This also assumes that you have created a 3.4.1 tag for your changes.

  1. Create a branch for the new documentation:

    git checkout master
git pull
git checkout -b pickup-3.4.1

  2. Confirm versions currently being built:

    make versions-show
Documentation will be built for the following versions:
["3.3.0","3.3.4","3.3.5","3.4.0"]

  3. Update the settings.toml file to contain the following:

    MITTO_VERSION="3.4.1"
MITTO_TAGS='(3.3.[045]|3.4.[01])'
MITTO_BRANCHES='matches-no-branches'
MITTO_RELEASED='^refs/tags/(3.3.[045]|3.4.[01])$'
MITTO_LATEST='3.4.1'
MITTO_REL='3.4.1'
MITTO_VERSION_INFO='''{
   "2.11.2":     "https://www.zuar.com/api/mitto/2.11.2/index.html",
   "3.2.18":     "https://www.zuar.com/api/mitto/3.2.18.post1/index.html",
   "3.3.0":      "/docs/runner/3.3.0/index.html",
   "3.3.4":      "/docs/runner/3.3.4/index.html",
   "3.3.5":      "/docs/runner/3.3.5/index.html",
   "3.4.0":      "/docs/runner/3.4.0/index.html",
   "3.4.1":      "/docs/runner/3.4.1/index.html"
}'''

  4. Confirm the versions that will be built:

    make versions-show
Documentation will be built for the following versions:
["3.3.0","3.3.4","3.3.5","3.4.0","3.4.1"]

    If the correct versions aren’t shown, you almost certainly have one or both of these problems:

    1. One or more regexes in settings.toml are incorrect.

    2. One or more tags aren’t present (e.g., did you create a 3.4.1 tag?).

  5. Build the documentation for all versions:

    make versions

  6. Check that the build contents look reasonable:

    tree -L 1 build/html
build/html
├── 3.3.0
├── 3.3.4
├── 3.3.5
├── 3.4.0
├── 3.4.1
├── index.html
└── latest -> 3.4.1

    Compared to the earlier version, note the presence of a 3.4.1 directory and the fact that latest points to that directory.

  7. Review the changes in your browser:

    make serve

    View at http://0.0.0.0:8000

  8. Commit the changes you made to settings.toml.

    git add settings.toml
git commit -m'pickup 3.4.1'
git push

  9. Optional - publish your changes.

    You can publish to either of two locations. The first is considered an unofficial “draft” location and the second is the “official” public location.

    1. DRAFT: https://www.zuarbase.net/docs/runner/

    2. OFFICIAL: https://www.zuar.com/docs/runner/

    To publish “draft” documentation:

    make publish

    To publish “official” public documentation:

    PUBLIC=y make publish

  10. Either locally (followed by a push) or on github, merge your changes into master.

Post-publication Changes

After publishing documents for a release (e.g., 3.4.1), if you have a need to update those documents, you must reuse the 3.4.1 tag for the new changes. Tag re-use is generally a bad idea with git, but we really have no other option.

Important

If you make changes to tags (e.g. to 3.4.1) locally, be sure to push those changes to github; if you don’t nobody else will see those changes and confusion will ensue.

JSON Schema Documentation

Portions of Runner are implemented using Pydantic classes which contain embedded documentation. One must use a local Runner development environment to create JSON Schema documentation from these classes. The development environment specifies a precise directory structure that must be in place. See the README.

On your local development system:

  1. Ensure your mitto repository has the desired product version checked out. Note that the mitto/docs/src/json_schema_classes.json file controls which classes are processed. You may need to update it from time to time.

  2. Build JSON Schema documents using the docs container.

    cd mitto-dev-runtime
docker compose run -it --rm docs bash -c 'cd /app/mitto/docs; make json_schemas'

    On the local host, these files will be available in the mitto repo in this directory: mitto/docs/src/reference/json_schemas/. You may wish to commit these, or you may not.

  3. Import the JSON Schema documents into the mitto-docs repo.

    cd mitto-docs
make refresh-json-schemas

    This will copy the JSON Schema documents out of the mitto repo and into mitto-docs/docs/src/reference/json_schemas.

  4. Build and serve local documentation as usual.

    make html
make serve

    Commit and publish changes as desired.