Add JSON for each Script and a metadata.json file

[newzealandpaul]

This is an updated pull request. The original is here. The changes are:

This incorporates feedback from maintainers on JSON structure. The changes/improvements are:

• Use null for values rather than ""
• Has a metadata.json file with categories (IDs and a sort order). This file can be used for tags in the future as well.
• Script JSON files now have IDs for categories.
• Alerts has been renamed notes.
• Notes can be type warning (the default, we can manually update these over time). Frontend devs can specify other note types.
• Includes latest scripts as of the time this pull request has been submitted, using the database as at 11:40 pm Tuesday, 5 November UTC.

Acknowledgments

I want to acknowledge @tteck who put this all together in the first pace, this just builds upon their work.

I also want to acknowledge the amazing work @BramSuurdje put the database together on the database and suggested the json schema. Having this in the first place made this job a lot easier.

Also @havardthom who consulted during this process and helped move things along.

Description

This merge adds a json/ directory with a .json file for each script.

The json structure is in the format proposed by @havardthom with some minor changes due to discovery of information as I wrangled the data.

The data is extracted from the pocketbase DB in consultation with @BramSuurdje. It also parses the scripts (ct, vm and misc) for details.

Here is an example JSON file (not a real script json file):

{
    "name": "A Test App",
    "slug": "atestapp",
    "categories": [
        45
    ],
    "date_created": "2024-05-02",
    "type": "ct",
    "updateable": true,
    "privileged": false,
    "interface_port": "8082",
    "documentation": null,
    "website": "https://www.example.com/",
    "logo": "https://raw.githubusercontent.com/Athou/commafeed/master/example/public/app-icon-144.png",
    "description": "A Test App is a Google Reader inspired self-hosted RSS reader.",
    "install_methods": [
        {
            "type": "default",
            "script": "ct/atestapp.sh",
            "resources": {
                "cpu": "2",
                "ram": "2048",
                "hdd": "4",
                "os": "debian",
                "version": "12"
            }
        }
    ],
    "default_credentials": {
        "username": "admin",
        "password": "admin"
    },
    "notes": [
        {
            "text": "Primary and Worker Private Keys Must Match.",
            "type": "warning"
        },
        {
            "text": "Configuration Path: `/opt/cronicle/conf/config.json`",
            "type": "warning"
        }
    ]
}

As of now metadata.json looks like this:

{
    "categories":
    [
        {"name": "Miscellaneous", "id": 0, "sort_order": 99.0},
        {"name": "Proxmox VE Tools", "id": 1, "sort_order": 1.0},
        {"name": "Home Assistant", "id": 2, "sort_order": 2.0},
        {"name": "Automation", "id": 3, "sort_order": 3.0},
        {"name": "MQTT", "id": 4, "sort_order": 4.0},
        {"name": "Database", "id": 5, "sort_order": 5.0},
        {"name": "Zigbee - Zwave", "id": 6, "sort_order": 6.0},
        {"name": "Monitoring - Analytics", "id": 7, "sort_order": 7.0},
        {"name": "Docker - Kubernetes", "id": 8, "sort_order": 8.0},
        {"name": "Operating System", "id": 9, "sort_order": 9.0},
        {"name": "TurnKey", "id": 10, "sort_order": 10.0},
        {"name": "Server - Networking", "id": 11, "sort_order": 11.0},
        {"name": "Media - Photo", "id": 12, "sort_order": 12.0},
        {"name": "AdBlocker - DNS", "id": 13, "sort_order": 13.0},
        {"name": "Document - Notes", "id": 14, "sort_order": 14.0},
        {"name": "Dashboards", "id": 15, "sort_order": 15.0},
        {"name": "File - Code", "id": 16, "sort_order": 16.0},
        {"name": "NVR - DVR", "id": 17, "sort_order": 17.0}
    ]
}

Notes

I found a couple of scripts that are in the database but no longer exist as scripts:

1. scrutiny
2. collabora-online

So I think keeping the database in the repo as static files that can be committed or changed in lock-step with the scripts is a good idea. In the future we can automate checking that new scripts have a correct json file.

The updated script that generated these json files can be found here.

This wrangling job took quite a long time dealing with edge cases etc.

Type of change

Please check the relevant option(s):

• Bug fix (non-breaking change that resolves an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (a fix or feature that would cause existing functionality to change unexpectedly)
• New script (a fully functional and thoroughly tested script or set of scripts.)
• Self-review performed (I have reviewed my code, ensuring it follows established patterns and conventions)
• Documentation update required (this change requires an update to the documentation)

Related Pull Requests / Discussions

• Outdated: Add JSON Documents for Each Script #70

[newzealandpaul]

@MickLesk this contains the requests and suggestions made by the team. Can you please squash and merge. The sooner this is in the system, the less chance it will fall out of sync. It includes the latest scripts.

[havardthom]

I don't see any approvals on this PR, has anyone fully reviewed it or was it just merged on trust? Are we in a rush? I was atleast going to let copilot have a scan first

[newzealandpaul]

@havardthom this is a modification of the previous PR, we had all discussed it and it modifies no files. It only adds json.

[remz1337]

I agree with @havardthom , we should aim to always have 2 approvals, no matter the PR. Security and trust from the community is top priority.

[remz1337]

@newzealandpaul I wrote both Scrutiny and Collabora scripts. Tteck initially merged them, but after some time decided to remove as they need more advanced installation instructions. For example, Scrutiny requires a collector to be installed on the host, and collabora does not provide a web UI, so the only way to test is to setup the integration with Nextcloud.

If we could find a way to show additional installation notes (on the website and upon starting the installation script), I think these could make their way back into the official repo. Otherwise we can remove the files for now