# Building Documentation

## Introduction

Two types of Mitto documentation can be built: `unversioned` and
`versioned`.  Unversioned documentation is built from whatever files
are present in the repo at the time the documentation is built (i.e.,
the currently checked out branch +/- any changes present).

Versioned documentation is a set of one or more bodies of
documentation, each associated with a release number.  During the
build process, each version of the documentation is checked out from
`git`, the HTML is created, and it is added to the set of versioned
documentation.  The overall set of documentation has a pulldown that
allows the viewer to choose the documentation version that is
displayed.

Generally, when creating new content or revising existing content, one
should work with single-version documentation as it is much faster
than multi-version.  Only after all changes have been reviewed and one
is ready to publish the changes is it necessary to build multi-version
documentation.

## Setup

Zuar's `zsphinx` [docker image](https://github.com/zuarbase/zsphinx)
is used to build these documents.  Two software packages must be
installed on your local system to work with these documents:

1. Docker - The easiest path to this is to install Docker Desktop:
   * [mac](https://docs.docker.com/desktop/install/mac-install/)
   * [windows](https://docs.docker.com/desktop/install/windows-install/)
   * [linux](https://docs.docker.com/desktop/install/linux-install/)

2. Python3 - Any Python3 is fine; no external dependencies need to be
   installed and a virtual environment is not required.

3. Clone this git repository locally:
   `git clone git@github.com:zuarbase/mitto-docs.git`

To confirm that everything is working, the following commands will
pull, build, and serve a single version of the documentation:

.. prompt:: bash $,# auto

   $ cd mitto-docs
   $ git checkout master
   $ git pull
   $ make html
   $ make serve

View documentation in your browser at http://0.0.0.0:8000/

At this point, you should be able to successfully build and view
documentation on your local system.

Publishing of the documentation that is built is done via the `rsync`
command.  If you wish to publish documentation, you will need to place
credentials for `licensing.zuar.com` in your `~/.netrc` file.
The credentials will occupy a sigle line in the file and will look
something like this:
```
machine licensing.zuar.com login admin password password-goes-here-but-this-isnt-it
```

## Building Unversioned Documentation

.. IMPORTANT::

   This process builds the current contents of the `mitto-docs`
   directory, whether or not it is checked in to the `git` repository
   or not.

1. Ensure the repo is up to date:

   .. prompt:: bash $,# auto

      $ cd mitto-docs
      $ git checkout master
      $ git pull

1. Create a branch for your changes:

   .. prompt:: bash $,# auto

      $ git checkout -b changes-for-3.4.1

1. Cleanup any stale documentation:

   .. prompt:: bash $,# auto

      $ make clean

1. Make your changes.

1. Build your documentation:

   .. prompt:: bash $,# auto

      $ make html


   The HTML and other files create by building the documentation is
   placed in `build/html`.  E.g.:

   .. code-block::

      $ tree -L 1 build/html
      build/html
      ├── _images
      ├── _sources
      ├── _static
      ├── api
      ├── architecture
      ├── build-log.html
      ├── connectors
      ├── databases
      ├── genindex.html
      ├── index.html
      ├── jobs
      ├── json_schemas
      ├── objects.inv
      ├── reference
      ├── release_notes
      ├── search.html
      ├── searchindex.js
      ├── sitemap.xml
      ├── todo-links.html
      ├── todo-list.html
      ├── todo.html
      ├── user-guides
      └── user-interface

1. View your changes:

   .. prompt:: bash $,# auto

      $ make serve

   View content at http://0.0.0.0:8000

1. Iterate over the previous three steps until the changes are as
   desired.

1. Commit your changes to git via `git commit`.

   **Important:** Do not check the contents of `build/` into the
   repository. The HTML, etc., is always rebuilt; only documentation
   source is controlled.

1. Merge your branch into `master`

1. Optional - tag the merge with a release number, e.g.:

   .. prompt:: bash $,# auto

      $ git tag -m 3.4.1 3.4.1

   If you create a tag locally, be sure to push it to github
   via, e.g., `git push origin 3.4.1`.

   See more about version tagging in the next section.

## Building Versioned Documentation

.. IMPORTANT::

   Versioned documentation is built **solely** from the contents of one or
   more `git` commits.  The contents of the current directory is
   ignored during the build process, with two exceptions -- `settings.toml`
   and `conf.py` in the current directory are used to control the build of the
   documentation (regardless of whether they are checked-in to the repo
   or not).
   
### Defining What to Build

A single file controls which documentation is built: `settings.toml`.
Example contents:
```
# This file controls documentation builds, particularly versioned documentation.
#
# NOTE: If you make changes here, you likely also need to change
#       version_info in conf.py as well.
#
# TODO:
# 1. One of MITTO_VERSION and MITTO_REL should be eliminated.

# The "current" product version.  This will almost always be the most recent version.
# This value is used as the version number in the generated documentation.
MITTO_VERSION="3.4.0"

# The remaining values in this file control the behavior of sphinx-multiversion
# See the documenation elsewhere that describes what each variable does and how
# to set it.

# Tags for which documentation will be built (regex)
MITTO_TAGS='(3.3.[045]|3.4.0)'

# Branches for which documentation will be built (regex)
# If you only wish to build documentation for tags and omit all branches,
# use a value that won't match any branches.
MITTO_BRANCHES='matches-no-branches'

# Defines branches and tags that will be marked as "released" (regex).
# Anything that is built, but does not match, will be marked "development".
MITTO_RELEASED='^refs/tags/(3.3.[045]|3.4.0)$'

# String (not regex) that specifies the release to which the "latest" link will redirect to.
MITTO_LATEST='3.4.0'

# String (not regex) is a Jira release number for which to create release notes.
# Only used when building release notes.
MITTO_REL='3.4.0'

# Controls contents of version pulldown in top right of page.
# Contents is a multiline string that defiles which versions appear
# in the pulldown as well as what they link to.  Ths string must be
# valid JSON.  The Python value of the string is passed to the
# sphinx_material theme.
MITTO_VERSION_INFO='''{
   "2.11.2":     "https://www.zuar.com/api/mitto/2.11.2/index.html",
   "3.2.18":     "https://www.zuar.com/api/mitto/3.2.18.post1/index.html",
   "3.3.0":      "/docs/runner/3.3.0/index.html",
   "3.3.4":      "/docs/runner/3.3.4/index.html",
   "3.3.5":      "/docs/runner/3.3.5/index.html",
   "3.4.0":      "/docs/runner/3.4.0/index.html"
}'''
```

### Output of Build

When multiple versions of documentation are built, they are 
built into `docs/build/html`. Each version is contained in its own
subdirectory.  E.g.:
```
$ tree -L 1 build/html
build/html
├── 3.3.0
├── 3.3.4
├── 3.3.5
├── 3.4.0
├── index.html
└── latest -> 3.4.0
```

A special `index.html` is built an placed in `docs/build/html`; it
automatically redirects browsers to the "current" version of the
documentation.  Links in the sidebar of each page display the version
currently being viewed and allow the user to switch documentation
versions.

### Walkthrough - Update Documentation for `3.4.1`

The example `settings.toml` file will build documentation for the
following versions: `3.3.0`, `3.3.4`, `3.3.5`, `3.4.0`.  Let's walk
through an example of adding documentation for `3.4.1`.

This example assumes that you have already followed the steps in 
"Building Unversioned Documentation" to create your content.

.. important::

   This also assumes that you have created a `3.4.1` tag
   for your changes.

1. Create a branch for the new documentation:

   .. prompt:: bash $,# auto

      $ git checkout master
      $ git pull
      $ git checkout -b pickup-3.4.1

1. Confirm versions currently being built:

   .. prompt:: bash $,# auto

      $ make versions-show
      Documentation will be built for the following versions:
      ["3.3.0","3.3.4","3.3.5","3.4.0"]

1. Update the `settings.toml` file to contain the following:
   ```
   MITTO_VERSION="3.4.1"
   MITTO_TAGS='(3.3.[045]|3.4.[01])'
   MITTO_BRANCHES='matches-no-branches'
   MITTO_RELEASED='^refs/tags/(3.3.[045]|3.4.[01])$'
   MITTO_LATEST='3.4.1'
   MITTO_REL='3.4.1'
   MITTO_VERSION_INFO='''{
      "2.11.2":     "https://www.zuar.com/api/mitto/2.11.2/index.html",
      "3.2.18":     "https://www.zuar.com/api/mitto/3.2.18.post1/index.html",
      "3.3.0":      "/docs/runner/3.3.0/index.html",
      "3.3.4":      "/docs/runner/3.3.4/index.html",
      "3.3.5":      "/docs/runner/3.3.5/index.html",
      "3.4.0":      "/docs/runner/3.4.0/index.html",
      "3.4.1":      "/docs/runner/3.4.1/index.html"
   }'''
   ```

1. Confirm the versions that will be built:

   .. prompt:: bash $,# auto

      $ make versions-show
      Documentation will be built for the following versions:
      ["3.3.0","3.3.4","3.3.5","3.4.0","3.4.1"]

   If the correct versions aren't shown, you almost certainly have one
   or both of these problems:

   1. One or more regexes in `settings.toml` are incorrect.
   2. One or more tags aren't present (e.g., did you create a `3.4.1` tag?).

1. Build the documentation for all versions:

   .. prompt:: bash $,# auto

      $ make versions

1. Check that the build contents look reasonable:
      
   .. prompt:: bash $,# auto

      $ tree -L 1 build/html
      build/html
      ├── 3.3.0
      ├── 3.3.4
      ├── 3.3.5
      ├── 3.4.0
      ├── 3.4.1
      ├── index.html
      └── latest -> 3.4.1

   Compared to the earlier version, note the presence of a `3.4.1`
   directory and the fact that `latest` points to that directory.

1. Review the changes in your browser:

   .. prompt:: bash $,# auto

      $ make serve

   View at http://0.0.0.0:8000

1. Commit the changes you made to `settings.toml`.

   .. prompt:: bash $,# auto

      $ git add settings.toml
      $ git commit -m'pickup 3.4.1'
      $ git push

1. Optional - publish your changes.

   .. warning::

      There is nothing wrong with publishing your documentation now
      and this is the most convenient time to do so.  However, if you
      do, it is imperative that you follow the remaining steps in this
      section afterwards.  If you don't, you run the risk of not being
      able to recreate what you've just published.

   You can publish to either of two locations.  The first is
   considered an unofficial "draft" location and the second is the
   "official" public location.
   1. *DRAFT:* https://www.zuarbase.net/docs/runner/
   2. *OFFICIAL:* https://www.zuar.com/docs/runner/

   To publish "draft" documentation:

   .. prompt:: bash $,# auto
   
      $ make publish

   To publish "official" public documentation:

   .. prompt:: bash $,# auto
   
      $ PUBLIC=y make publish
      
1. Either locally (followed by a push) or on github,
   merge your changes into `master`.

### Post-publication Changes

After publishing documents for a release (e.g., `3.4.1`), if you have
a need to update those documents, you must reuse the `3.4.1` tag for
the new changes.  Tag re-use is generally a bad idea with `git`, but we
really have no other option.

.. important::

   If you make changes to tags (e.g. to `3.4.1`) locally, be sure to
   push those changes to github; if you don't nobody else will see
   those changes and confusion will ensue.

## JSON Schema Documentation

Portions of Runner are implemented using Pydantic classes which contain embedded
documentation.  One must use a local Runner development environment to create  JSON
Schema documentation from these classes.  The development environment specifies a
precise directory structure that must be in place.  See the [README](
https://github.com/zuarbase/mitto-dev-runtime).

On your local development system:

1. Ensure your `mitto` repository has the desired product version checked out.
   Note that the `mitto/docs/src/json_schema_classes.json` file controls which classes
   are processed.  You may need to update it from time to time.

1. Build JSON Schema documents using the `docs` container.
   ```
   cd mitto-dev-runtime
   docker compose run -it --rm docs bash -c 'cd /app/mitto/docs; make json_schemas'
   ```

   On the local host, these files will be available in the `mitto` repo 
   in this directory: `mitto/docs/src/reference/json_schemas/`.
   You may wish to commit these, or you may not.

1. Import the JSON Schema documents into the `mitto-docs` repo.
   ```
   cd mitto-docs
   make refresh-json-schemas
   ```
   This will copy the JSON Schema documents out of the `mitto` repo and into
   `mitto-docs/docs/src/reference/json_schemas`.

1. Build and serve local documentation as usual.
   ```
   make html
   make serve
   ```

   Commit and publish changes as desired.