Documentation overhaul - Fixing formatting issues, autodoc implementation, and added variables for versions. (#3734)

Closes #3639. 

This is a bit of an overhaul of the docs to make them more
easy to utilize. This changes include:
- Removal of overhanging public aspects of the documentation
- Reformatting of terminal commands to allow for easy copy & pasting
- Updating points of contact
- Screenshot Updates
- Automated code API documentation. [Google's docstring
style](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings)
should be used.
- Added variables for things like software versions & packages.
- Unless python & node version are manually specified, when docs are
built these will be pulled from
[`.nvmrc`](https://github.com/HyphaApp/hypha/blob/main/.nvmrc) &
[`.python-version`](https://github.com/HyphaApp/hypha/blob/main/.python-version)
- Consolidated the production and development guides into one location
(both under `Setup` rather than one under `Getting Started` and one
under `Setup`), and updated them to be similar to one another.
- Added different steps for getting Hypha development deployed under
multiple OSs

Author: wes-otf

.github/workflows/deploy-docs.yml

@@ -24,6 +24,7 @@ jobs:
       - name: Deploy docs
         uses: mhausenblas/mkdocs-deploy-gh-pages@master
         env:
+          GEN_REF_PAGES: true
           GITHUB_TOKEN: ${{ github.token }}
           CUSTOM_DOMAIN: docs.hypha.app
           CONFIG_FILE: mkdocs.yml

CODE_OF_CONDUCT.md

@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at [email protected] and/or [email protected]. All
+reported by contacting the project team at [email protected]. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.

README.md

@@ -57,7 +57,7 @@ We are grateful to organizations that have chosen to implement Hypha and appreci
 - [Remote Inning](https://www.remoteinning.com/)
 - [Maxwell Pearl](https://maxwellpearl.com/)
 - [Psycle](https://psycle.com/)                                                    
-- [Simply Secure](https://simplysecure.org/)
+- [Superbloom](https://superbloom.design/)
 - [Throneless Technology](https://throneless.tech/)
 - [Torchbox](https://www.torchbox.com/)
 
@@ -75,4 +75,4 @@ We are grateful to organizations that have chosen to implement Hypha and appreci
 * Community forum (Discourse): https://we.hypha.app/
 * Chat (Zulip): https://chat.hypha.app/
 * Why Hypha? [Hyphae](https://en.wikipedia.org/wiki/Mycorrhizal_network):** long, branching ecosystem enriching organisms that form interconnected networks to collectively exchange resources. Hopefully, Hypha helps it's users do the same.
-* Copyright (C) 2021 - Open Technology Fund
+* Copyright (C) 2024 - Open Technology Fund

SECURITY.md

@@ -4,7 +4,7 @@ We take security very seriously. We welcome any peer review of our 100% open sou
 
 ## Where should I report security issues?
 
-In order to give the community time to respond and upgrade we strongly urge you report all security issues privately. Please email [email protected] and/or [email protected] with details and reproduction steps. Security issues *always* take precedence over bug fixes and feature work. We can and do mark releases as "urgent" if they contain serious security fixes.
+In order to give the community time to respond and upgrade we strongly urge you report all security issues privately. Please email [email protected] with details and reproduction steps. Security issues *always* take precedence over bug fixes and feature work. We can and do mark releases as "urgent" if they contain serious security fixes.
 
 For a list of recent security commits, check [our GitHub commits prefixed with SECURITY](https://github.com/HyphaApp/hypha/search?utf8=%E2%9C%93&q=SECURITY&type=Commits).
 

docker/entrypoint.dev.sh

@@ -11,6 +11,6 @@ python3 manage.py migrate
 python3 manage.py wagtailsiteupdate hypha.test 8090
 
 # Start gunicorn server.
-gunicorn hypha.wsgi:application --env DJANGO_SETTINGS_MODULE=hypha.settings.dev --reload  --bind 0.0.0.0:9001
+gunicorn hypha.wsgi:application --env DJANGO_SETTINGS_MODULE=hypha.settings.dev --threads 3 --reload  --bind 0.0.0.0:9001
 
 exec "$@"

docs/assets/how-to-login-login-page.png

@@ -1,9 +1,5 @@
-Hypha consists of three distinctive components:
-
-1. First is the **Apply Site**, where people seeking for funds can apply by submitting their applications, the submission goes through different stages before it being approved for funding. After the applications are approved, they can be converted to a project. Apply site allows to manage the lifecycle of a project starting from a PAF to contracting to invoicing.
-
-2. Then the **Admin or Wagtail Admin**, is used to create custom forms, setup funds/labs and workflow around them. Think of it as the back-office of for your submissions and projects.
 
+## Diagram
 
 ```
                             Integrations                           
@@ -16,16 +12,33 @@ Hypha consists of three distinctive components:
                                                                     
                                                                     
 
-        ┌───────────┐                                            
-        │APPLY SITE │◀───┐    ┌───────────┐                      
-        └───────────┘    │    │  DJANGO   │         ┌───────────┐
-                         ├────│  BACKEND  │◀────────│PostgresSQL│
-        ┌───────────┐    │    └───────────┘         └───────────┘
-        │   ADMIN   │◀───┘                                       
-        └───────────┘                                            
+    ┌────────────┐                                  Databases
+    │ APPLY SITE │◀────┐                            ┌ ─ ─ ─ ─ ─ ─ ─ ─┐
+    └────────────┘     │      ┌────────────┐        ╎ ┌────────────┐ ╎
+                       ├──────│  Django /  │◀───────╎ │ PostgreSQL │ ╎
+    ┌────────────┐     │      │  Wagtail   │        ╎ └────────────┘ ╎
+    │   WAGTAIL  │◀────┘      └────────────┘        └─ ─ ─ ─ ─ ─ ─ ─ ┘
+    │   ADMIN    │                                  
+    └────────────┘
 ```
 
-## Django
+-----------
+
+## Main Components
+
+### Apply Site
+
+Where people seeking for funds can apply by submitting their applications, the submission goes through different stages before it being approved for funding. After the applications are approved, they can be converted to a project. Apply site allows to manage the lifecycle of a project starting from a PAF to contracting to invoicing.
+
+### Wagtail Admin
+
+Used to create custom forms, setup funds/labs and workflow around them. Think of it as the back-office of for your submissions and projects.
+
+-----------
+
+## Under The Hood
+
+### Django
 
 Hypha is built on top of the Django Web Framework. All the pages are rendered server side. It uses wagtail CMS for creating and managing custom application forms, public pages and settings.
 

docs/assets/how-to-login-nav.png

@@ -2,7 +2,7 @@
 
 ## Creating Pull Requests
 
-Found a bug and know how to fix it? Have some design wireframes to improve some usability issues? Traslated Hypha into another language? Built a new feature you want to add? Please submit an issue or create a pull request on [GitHub](https://github.com/HyphaApp/hypha/issues).
+Found a bug and know how to fix it? Have some design wireframes to improve some usability issues? Translated Hypha into another language? Built a new feature you want to add? Please submit an issue or create a pull request on [GitHub](https://github.com/HyphaApp/hypha/issues).
 
 Guidance for contributing code:
 

docs/assets/how-to-login-w-pass-btn.png

@@ -10,7 +10,7 @@ We use pull requests for all changes. No commits are done directly in to the mai
 
 4. **Link the PR to the corresponding issue** - A common way is to add "Fixes \#1234" at the top of the PR description. See [Linking a pull request to an issue](https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)
 
-5. **Add a reviewer** - If noting else has been agreed upon add Fredrik Jonsson [@frjo](https://github.com/frjo).
+5. **Add a reviewer** - If nothing else has been agreed upon add Fredrik Jonsson [@frjo](https://github.com/frjo).
 
 ## Git command examples
 
@@ -20,17 +20,17 @@ First check out master and do a git pull ot get all the latest updates.
 
 Then create a new branch and do a checkout of it.
 
-```text
-$ git checkout master
-$ git pull
-$ git checkout -b fix/gh-1234-fixing-thing-a
+```shell
+git checkout master
+git pull
+git checkout -b fix/gh-1234-fixing-thing-a
 ```
 
 ### Adding commits
 
-```text
-$ git add
-$ git commit -m "A good commit message."
+```shell
+git add
+git commit -m "A good commit message."
 ```
 
 ### Pushing branch first time to GitHub
@@ -39,9 +39,9 @@ First make sure we are in the correct branch. Then push the branch to origin, i.
 
 (Pushing to `HEAD` is equivalent to pushing to a remote branch having the same name as your current branch.)
 
-```text
-$ git checkout fix/gh-1234-fixing-thing-a
-$ git push -u origin HEAD
+```shell
+git checkout fix/gh-1234-fixing-thing-a
+git push -u origin HEAD
 ```
 
 The message in the Terminal will contain the URL to create an PR. On most systems you can Command/CTRL click that to open it directly in your default browser.
@@ -50,12 +50,12 @@ The message in the Terminal will contain the URL to create an PR. On most system
 
 Checkout master and update it. Checkout the branch you are working on and issue the command to rebase it from master. If that resulted in any changes you will then need to do a force push to GitHub.
 
-```text
-$ git checkout master
-$ git pull
-$ git checkout fix/gh-1234-fixing-thing-a
-$ git rebase master
-$ git push --force
+```shell
+git checkout master
+git pull
+git checkout fix/gh-1234-fixing-thing-a
+git rebase master
+git push --force
 ```
 
 Read more about [Git rebase](https://www.atlassian.com/git/tutorials/rewriting-history/git-rebase).

docs/assets/how-to-login-w-pass-prompt.png

@@ -0,0 +1,3 @@
+## Deploying Hypha
+
+Navigating to the setup guide [production](../setup/deployment/production/stand-alone.md) or [development](../setup/deployment/development/stand-alone.md)

docs/assets/how-to-login-wo-pw-page.png

@@ -31,4 +31,4 @@ View our [Roadmap](https://github.com/HyphaApp/hypha/wiki/Roadmap) for upcoming
 ## Technology
 
 * Built with [Django](https://www.djangoproject.com/), PostgreSQL and [Wagtail](https://wagtail.io/)
-* Deploy with [Heroku](./setup/deployment/heroku.md), [Docker](./setup/deployment/docker.md), or [your own server](./setup/deployment/stand-alone.md).
+* Deploy with [Heroku](./setup/deployment/production/heroku.md), [Docker](./setup/deployment/production/docker.md), or [your own server](./setup/deployment/production/stand-alone.md).

docs/getting-started/architecture.md

@@ -0,0 +1,101 @@
+import re
+
+# The versioning file for python
+PYTHON_VERSION_FILE = ".python-version"
+
+# The versioning file for node (used by nvm)
+NVM_VERSION_FILE = ".nvmrc"
+
+# RegEx string recommended for semver:
+# https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
+SEMVER_REGEX = r"^(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$"
+
+
+# Exception classes
+class InvalidVersionException(Exception):
+    """A version specified in `PYTHON_VERSION_FILE` or `NVM_VERSION_FILE` are of invalid format."""
+
+
+class VersionFileNotFoundError(FileNotFoundError):
+    """A version file specified in `PYTHON_VERSION_FILE` or `NVM_VERSION_FILE` was not found."""
+
+
+def define_env(env):
+    """
+    Where mkdocs-macros are defined. For more info, see:
+    https://mkdocs-macros-plugin.readthedocs.io/en/latest/macros/
+    """
+
+    # If python & node versions are left blank in mkdocs.yml, attempt to automatically populate them
+
+    if env.variables.versions["python"]["version"] is None:
+        py_ver = get_python_version()
+        # Remove the patch number from the docs versioning
+        env.variables.versions["python"]["version"] = ".".join(py_ver.split(".")[:-1])
+
+        if env.variables.versions["python"]["packages"]["macos"] is None:
+            env.variables.versions["python"]["packages"][
+                "macos"
+            ] = f"python@{'.'.join(py_ver.split('.')[:-1])}"
+
+    if env.variables.versions["node"]["version"] is None:
+        node_ver = get_node_version()
+
+        # Remove the patch number from the docs versioning
+        env.variables.versions["node"]["version"] = ".".join(node_ver.split(".")[:-1])
+
+
+def get_python_version() -> str:
+    """
+    Returns the semantic version of the current Python runtime based off the specified `PYTHON_VERSION_FILE`
+    """
+    version: str
+
+    try:
+        with open(PYTHON_VERSION_FILE, "r") as py_ver:
+            version = py_ver.read()
+
+        if not valid_semver(version):
+            raise InvalidVersionException(f'Unrecognized Python version: "{version}"!')
+
+        return version
+    except FileNotFoundError as err:
+        raise VersionFileNotFoundError(
+            f'Failed to find Python version file at "{PYTHON_VERSION_FILE}"'
+        ) from err
+
+
+def get_node_version() -> str:
+    """
+    Returns the semantic version of the current Node runtime based off the specified `NVM_VERSION_FILE`
+    """
+    version: str
+
+    try:
+        with open(NVM_VERSION_FILE, "r") as nvm_ver:
+            version = nvm_ver.read()
+
+        # Node version is usually in the format of "vX.X.X"
+        if version.startswith("v"):
+            version = version[1:]
+
+        if not valid_semver(version):
+            raise InvalidVersionException(f'Unrecognized Node version: "{version}"!')
+
+        return version
+    except FileNotFoundError as err:
+        raise VersionFileNotFoundError(
+            f'Failed to find Node version file at "{NVM_VERSION_FILE}"'
+        ) from err
+
+
+def valid_semver(version: str) -> bool:
+    """
+    Uses the suggested semver regex pattern to validate that the given version string is valid.
+    """
+    regex_match = re.match(SEMVER_REGEX, version)
+
+    if regex_match is None:
+        return False
+
+    return True

docs/getting-started/contributing/code-contributions.md

@@ -0,0 +1,52 @@
+"""
+Generate reference pages for the API. Uses concurrency to speed things up, and
+ensures that generation of references pages in enabled in the environment via
+the `GEN_REF_PAGES` var.
+
+For more info on how this is done, see:
+https://mkdocstrings.github.io/recipes/#automatic-code-reference-pages
+"""
+
+import concurrent.futures
+import os
+from pathlib import Path
+
+import mkdocs_gen_files
+
+# The code source directory
+src = Path(__file__).parent.parent.parent / "hypha"
+skip_folders = ["public", "tests", "migrations"]
+
+
+def process_path(path):
+    module_path = path.relative_to(src).with_suffix("")
+    doc_path = path.relative_to(src).with_suffix(".md")
+    full_doc_path = Path("references/API", doc_path)
+    parts = tuple(module_path.parts)
+    # Don't document any code in public/ or tests/ directories
+    for skip_folder in skip_folders:
+        if skip_folder in parts:
+            return
+    if parts[-1] == "__init__":
+        parts = parts[:-1]
+        doc_path = doc_path.with_name("index.md")
+        full_doc_path = full_doc_path.with_name("index.md")
+    elif parts[-1] in ["__main__"]:
+        return
+    # Create the required files on build with mkdocs_gen_files, see:
+    # https://oprypin.github.io/mkdocs-gen_files/
+    if len(parts) > 0:
+        with mkdocs_gen_files.open(full_doc_path, "w") as fd:
+            identifier = f"::: hypha.{'.'.join(parts)}"
+            fd.write(identifier)
+    mkdocs_gen_files.set_edit_path(full_doc_path, path)
+
+
+def generate_pages():
+    with concurrent.futures.ThreadPoolExecutor() as executor:
+        list(executor.map(process_path, sorted(src.rglob("*.py"))))
+
+
+# Confirm `GEN_REF_PAGES` is enabled in the env
+if os.getenv("GEN_REF_PAGES", False):
+    generate_pages()