Merge pull request #467 from PNNL-CompBio/documentation-staging

Documentation staging

Author: sgosline

.github/workflows/sphinx.yml

@@ -0,0 +1,43 @@
+name: Sphinx-docs
+
+on:
+  push:
+    branches:
+      - main
+      - documentation-staging
+  workflow_dispatch:
+
+jobs:
+  pages:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Install Pandoc
+      uses: pandoc/actions/setup@v1
+      with:
+        version: 3.5
+
+    - name: Install dependencies
+      run: pip install -r docs/requirements.txt
+
+    - name: Build HTML
+      run: python -m sphinx.cmd.build -M html docs/source docs/build
+
+    - name: Upload artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: html-docs
+        path: docs/build/html/
+
+    # - name: Deploy
+    #   uses: sphinx-notes/pages@v3
+    #   with:
+    #     publish: true
+    
+    - name: Deploy
+      uses: peaceiris/actions-gh-pages@v3
+      with:
+        github_token: ${{ secrets.DOCSITE_TOKEN }}
+        publish_branch: gh-pages
+        publish_dir: docs/build/html

Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

coderdata/dataset.yml

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.dockerignore renamed to docs/.dockerignore

@@ -0,0 +1,141 @@
+
+# Documentation Site Using Sphinx
+
+This document includes:
+- [The Sphinx local directory structure](#local-directory-structure)
+- [Sphinx commands](#sphinx-commands)
+- [Guide to updating the documentation site](#updating-the-documentation-site)
+- [Helpful sphinx features](#helpful-sphinx-features)
+- [Site Deployment](#deploying-the-documentation-site-using-github-actions)
+
+
+## Local Directory Structure
+
+![Directory Structure](local_struct.jpg?raw=true)
+
+## Sphinx Commands
+
+### Dependencies
+
+Included in `coderdata/docs/requirements.txt`. Example installations that enable various sphinx extensions.
+
+- sphinx 
+- myst-parser   - adds support for markdown files
+- sphinx-design - provides extra design functions (i.e buttons and grids)
+- nbsphinx      - enables rendering of .ipynb files
+- sphinx_tabs   - adds tabbed content components
+
+### Build
+
+To build the Sphinx documentation locally, navigate to the `docs/` directory and run the `sphinx-build` command. This command processes the content files from the `docs/source/` directory and generates the rendered HTML output in the `docs/build/` directory.
+
+```sh
+$ cd ../coderdata/docs
+
+$ python -m sphinx.cmd.build -M html <SOURCEDIR/> <OUTPUTDIR/>
+
+$ python -m sphinx.cmd.build -M html source/ build/
+```
+To view the rendered site open `coderdata/docs/build/html/index.html`.
+
+### Quickstart
+
+To create a basic sphinx set-up in a local repository use sphinx quickstart command.
+
+```sh
+$ cd ../repo/docs
+$ python -m sphinx.cmd.quickstart
+```
+Questions will follow this command and when it prompts if you'd like to separate the source and build directories select yes. 
+
+## Updating the Documentation Site 
+
+The `conf.py` file is the central configuration file for Sphinx and it determines the behavior and appearance of the generated documentation. Specifically it configures extensions, defines metadata, controls output settings, and customizes sphinx behavior. The `index.rst` file is the starting page for the Sphinx documentation build process. It acts as the root file, linking all other pages and defining the document hierarchy. All other .md and .rst files are added and internally referenced within the index page and/or within the sidebar. 
+
+### Pages
+**Note:** All updates will be made in the `documentation-staging` branch.\
+To update pages navigate to `docs/source/`, all .rst and .md files live here. Here is a reference of the main pages used for the documentation site. The tutorial page consists if the docsite_tutorial.ipynb converted to an html. The notebook lives in `docs/source/_notebooks/` and is converted to an html during the build process based on code within `conf.py`.
+
+| Page             | File Name                                                               | Content                        |
+| -----------------| ----------------------------------------------------------------------- | ------------------------------ |
+| Home Page        | index.rst                                                               | Introduction and Install       |
+| Usage            | usage.md                                                                | API and CLI                    |
+| API Reference    | APIreference.rst                                                        | Autogenerated documentation from docstrings|
+| Datasets         | datasets_included.rst                                                   | Datasets overview              |
+| Tutorial         | docsite_tutorial.ipynb                                                  | Deep Learning Tutorial         |
+| Contributing     | contribution_guide.rst *includes contribution.md and add_code_guide.md* | Contribution Guide             |
+
+**Note:** 
+- When adding pages, reStructuredText files are the most sphinx friendly and allow for use of helpful directives. However, Markdown files can be used and will render properly when the myst-parser extension is used. 
+- New pages must be added to the `.. toctree::` in the `index.rst` to be recognized by Sphinx. 
+- Currently the API reference does not include documentation for `plot_2D_respones_metric`, `format`, `split_train_test_validate`, and `split_train_other`. It seems that the most updated `coderdata` directory currently does not include docstrings for these functions.
+
+### Tutorial
+
+The tutorial is sourced in `source/_notebooks/` and when built pushed to `_notebooks/` in `gh-pages` branch. It uses nbsphinx extension to convert from .ipynb to an html and is referenced in the `index.rst` and the `my_custom_sidebar.html`. The dependencies for this should only be needed in your local environment and not in the `requirements.txt`. 
+
+### Images and Graphics
+
+To add any new images they will live in `docs/source/_static/` and when built are pushed to `_images` in `gh-pages` branch. The coderdata_header.jpg was made using [canva](https://www.canva.com/) and coderdata_1.jpg was made using biorender.
+
+### Site Theme and Design
+
+The original sphinx theme used is 'sphinxdoc', however, to change certain aspects I created `custom.css` to override the sphinxdoc theme. To update the custom theme navigate to `docs/source/_static/custom.css`. Changes to the sidebar can be made in `docs/source/_templates/my_custom_sidebar.html`. Any other custom templates will be added to `docs/source/_templates/`.
+
+**Note:** The sphinxdoc theme is auto-generated during the build command into `docs/build/html/_static/sphinxdoc.css`. It is regenerated everytime sphinx build runs so changes must be made in custom.css. 
+
+
+## Helpful Sphinx Features
+
+### Site Links
+
+- [sphinx-doc.org](https://www.sphinx-doc.org/en/master/index.html)
+- [sphinx.org/sample-sites](https://sphinx-themes.org/sample-sites/default-sphinxdoc/)
+- [sphinx.org/directives](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#rst-directives)
+
+
+### Directives and Examples Used in CoderData Documentation
+
+- .. automodule::
+This is used to generate documentation from the package. It is used in API reference.
+
+```sh
+.. automodule:: <pkg>.module
+   :members: <function>
+   :undoc-members:
+   :show-inheritance:
+```
+
+- .. include::
+To include content from another file. Example shows how to render a markdown file when using the include call.
+
+```sh
+.. include:: guide.md
+   :parser: myst_parser.sphinx_ 
+```
+
+- .. image::
+
+```sh
+.. image:: _static/image.png   
+		:align: center
+		:scale: 100%
+```
+## Deploying the Documentation Site Using Github Actions
+
+### Directory Structure for Deployment with Relevant Branches
+
+![Directory Structure](deploy_struct.jpg?raw=true)
+
+### Site Deployment
+
+- We added `sphinx.yml` to the `.github/workflows/` directory in the `documentation-staging` branch.
+
+- Then in `documentation-staging` we included all the source directory files that are needed to build the documentation. (These are all the same files that are needed to build the documentation locally, the only addition is `requirements.txt`). 
+**Note:** If you have been building the documentation locally and then are moving to commit to the staging branch there is no need to include any files from `docs/build/` directory.
+
+- Once a push is made from `documentation-staging` the workflow is triggered.
+
+- The built html files are then deployed to the root of the `gh-pages` branch. The included files in the branch are shown in the graphic above.
+
+**Note:** When updates to the python files in `coderdata` directory occur in `main`, those changes must be transfered to the `documentation-staging` branch in order for Sphinx to see the updated docstrings with the functions and correctly autogenerate the documentation for the API reference page.