Read the Docs: Host the docs

Read the Docs (RTD) uses Sphinx to build our docs and host the HTML output. RTD allows you to “activate” branches in a GitHub repository and anytime a commit is made to an activated branch, a Sphinx documentation build is kicked off. Hosted documentation can be public or private (requiring an RTD account in the IDM organization). Private documentation can also be shared temporarily with a sharing link or a temporary password (not an RTD account password).

RTD also hosts multiple versions of the documentation, improves upon the Sphinx out-of-the-box search functionality, and provides integrated search and page view analytics. We have configured the builds to email the repo owner and a member of the content team whenever a documentation build fails.

Our hosted documentation sets on RTD are all public (even for private tools), so you do not need an RTD account to view the documentation. You do, however, need an RTD account to view build logs or active versions of the documentation. If you are the repo owner, you may receive build failure emails with a link to view build logs; this link will produce a 404 error unless you are logged in with an RTD account.

You must have an admin account to set up a new project on Read the Docs as described below. If you do not, contact the documentation team to set up a new project on RTD for you.

  1. If the repo is private, request admin access so you can create webhooks from the RTD dashboard when the project is ready to be imported. This webhook is necessary to automatically trigger rebuilds of the documentation when changes are committed to the repo.

    • If this is not acceptable to the repo owner, work with them to manually create the SSH keys to add the webhooks:

      1. First, manually import a project at https://readthedocs.com/dashboard/import/manual/

      2. Go to Admin > SSH keys and click on the displayed key. Copy the public key that appears (beginning with ssh-rsa) and send to the repository owner.

      3. In the code repo, ask them to go to Settings > Deploy keys and add the public key with read-only access.

      4. Then have them create the webhook for the GitHub repo using the following instructions: https://docs.readthedocs.io/en/latest/webhooks.html#webhook-creation

  2. Create a working branch directly from your personal fork to use for creating a “test” project on Read the Docs.

    Although this requires you to import two separate projects in Read the Docs twice (a “test” project from your fork and then the “production” project from the main repo), it is the safer method and allows you to test changes from a fork before submitting future pull requests.

    If you just update the path to the repo from the project to transition it from your fork to the IDM repo, this creates problems with the SSH keys, web hook, and breadcrumbs in the built docs.

    • When the test project is building successfully, submit a pull request from your fork to the main IDM repo. Then you can go through the steps below again to import and configure the production project from the main repo.

  3. In addition to the files usually needed for the Sphinx build (_template, _static, etc.) create the following files:

    • A docs/requirements.txt file that lists all Sphinx extensions and other dependencies needed for the documentation build

    • .readthedocs.yml config file

      • The RTD build does not use the make.bat or Makefile files for the -W or other build flags. There is a minor workaround in docs that contain API reference to point to code in the makefile for API generation. But other areas of the makefiles are not utilized by RTD. We are maintaining them only for testing builds locally.

      • The YAML file must be at the root of the repository.

      • Set fail_on_warning: true, which is the equivalent of the -W flag in the makefiles. It treats warnings as errors so we don’t miss problems in the build.

      • Set the installation packages

        • At minimum, this includes the docs/requirements.txt file to install Sphinx and other doc requirements.

        • For docsets that include API reference, point to all files used to install the code packages themselves, and all dependencies. This can be either with pip (pointing to a requirements.txt file) or setuptools (pointing to a setup.py folder–it must use this specific filename, there is no way to specify a different Python setup file).

    • In conf.py, add tags.add('product-name') to indicate the conditional tag for the only directive. This is the equivalent of the -t tag in Makefile/make.bat for the local build.

  4. Test the doc build locally. See Sphinx: Build the HTML output.

  5. Commit and push your changes to the remote working branch.

  6. Import the project in Read the Docs:

    1. On the RTD, dashboard, click Import a Project and select the relevant repo.

    2. On the Teams menu, select Institute for Disease Modeling - IDM Content and then click Next.

    3. Open the project and then click Versions and select the branches you want to build documentation from.

      • For the test project, use the name of your working branch and set the Privacy level to Private.

      • For the production project, the “latest” version will generally be the master/main or dev branch. Check with the repo owner if you are unsure. Set the Privacy level to Public (even for code repos that are private). Don’t forget to set both the doc project and the latest branch to Public.

    4. In Admin > Advanced Settings, set the Google Analytics code to UA-87656833-3. Do this only for production projects.

    5. If the repo is private, in Admin > Environment Variables set PIP_EXTRA_INDEX_URL to https://packages.idmod.org/api/pypi/pypi-production/simple.

    6. Work with the code repo owner to set automation rules (generally you will want to trigger a new build for tagged versions). Set this under Admin > Automation Rules. Set one rule to activate the branch and another to make it public.

    7. In Admin > Notifications add a doc owner and code owner to the email notifications for build breaks (do this once you verify that the build is passing).

  7. Add the project as an IDM subproject (to configure a custom domain).

    1. Click the Read the Docs logo.

    2. Click the institute-for-disease-modeling/idm project to open it.

    3. In Admin > Subprojects, click Add subproject.

    4. Select the project from the Child drop-down list.

    5. Enter the alias. This should be the lowercase name of the software and will be inserted into the custom domain URL in the format https://docs.idmod.org/projects/alias/en/latest/

  8. If you want to keep docs private for private repositories, set up sharing. Note that IDM LT has given approval to share documentation publicly for private repos to enable intersphinx linking. See Grammar and style guide for more information on intersphinx. However, providing a sharing link to reviewers from the RTD test project build is a good practice.

    1. In Admin > Sharing, from the Access type dropdown, select Secret link.

    2. Provide an explanation for the sharing link.

    3. Paste the sharing link into your GitHub PR for reviewers to use (note this will expire in 7 days).

    4. Verify that the default branch for the test project is your working branch so that reviewers see the correct version of the documentation.

  9. Once the project is set up and building correctly, add it to the list of docsets on the IDM wiki here: https://wiki.idmod.org/display/SP/IDM+Docsets.

Note

The RTD build is on Linux and is therefore case-sensitive. There were errors in some migrated docsets with casing mismatches in glossary entries and the CSV files and paths to those files (be aware that the local Windows build will not catch these issues because it is not case-sensitive).