Implemented Containerization of the Lyra Webapp

.dockerignore

@@ -0,0 +1,11 @@
+webapp/.next
+webapp/node_modules
+webapp/coverage
+webapp/store.json
+
+**/.DS_Store
+**/npm-debug.log*
+**/.env*.local
+**/.vercel
+**/*.tsbuildinfo
+**/next-env.d.ts

.github/workflows/build-and-push-image.yaml

@@ -0,0 +1,39 @@
+name: Build and Push Docker Image
+
+on:
+  push:
+    tags:
+      - '*.*.*'
+      - '*.*.*-rc.*'
+
+jobs:
+  build-and-push:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Log in to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract version from tag
+        id: extract_version
+        run: echo "VERSION=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV
+
+      - name: Validate version
+        run: |
+          if [[ ! "${{ env.VERSION }}" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-rc.[0-9]+)?$ ]]; then
+            echo "Invalid tag version: ${{ env.VERSION }}. Ensure to respect semantic versioning."
+            exit 1
+          fi
+
+      - name: Build and tag image
+        run: docker build -t ghcr.io/zetkin/lyra:${{ env.VERSION }} .
+
+      - name: Push image
+        run: docker push ghcr.io/zetkin/lyra:${{ env.VERSION }}

Dockerfile

@@ -0,0 +1,49 @@
+##################################################
+# 1) BUILD STAGE
+##################################################
+FROM node:22-alpine AS builder
+
+WORKDIR /app
+
+# Copy only the necessary files for install
+COPY package.json package-lock.json ./
+COPY webapp/package.json webapp/
+
+# Install dependencies in a single layer to leverage caching
+# npm ci is better for reproducible builds than npm install
+RUN npm ci
+
+# Copy the rest of the source code
+COPY webapp webapp
+
+RUN npm --workspace webapp run build
+
+##################################################
+# 2) RUNTIME STAGE
+##################################################
+FROM node:22-alpine AS runner
+
+RUN apk add --no-cache git openssh && \
+    addgroup -g 1001 nodejs && \
+    adduser -G nodejs -u 1001 -D nodeuser
+
+WORKDIR /app
+
+# Copy over the production build from builder stage
+COPY --from=builder /app/webapp/.next/standalone ./
+COPY --from=builder /app/webapp/.next/static ./webapp/.next/static
+
+RUN mkdir -p /home/nodeuser/.ssh && \
+    mkdir -p /lyra-projects && \
+    chown -R nodeuser:nodejs /home/nodeuser/.ssh && \
+    chown -R nodeuser:nodejs /app && \
+    chown -R nodeuser:nodejs /lyra-projects
+
+COPY known_hosts /home/nodeuser/.ssh/known_hosts
+
+# Switch to non-root user
+USER nodeuser
+
+
+EXPOSE 3000
+CMD ["sh", "-c", "git config --global user.email \"$GIT_USER_EMAIL\" && git config --global user.name \"$GIT_USER_NAME\" && node webapp/server.js"]

README.md

@@ -0,0 +1,249 @@
+# Lyra web app
+
+Lyra is a translation management system that integrates with
+source code repositories of internationalized applications.
+
+- Lyra can extract messages and translations
+  from source code repositories.
+- Lyra is an editor for translating messages.
+- Lyra can create pull requests to merge updated
+  translations into the source code repository.
+
+> [!WARNING]
+> A party who controls a source code repository branch
+> which Lyra is set up to translate will also have
+> a lot of control over the Lyra process. That control
+> probably includes reading files from the operating system
+> and possibly includes arbitrary code exection.
+
+For each repository Lyra is set up to translate,
+Lyra needs control over a file system directory.
+
+> [!WARNING]
+> A party who controls the contents of
+> a local repository directory will also have
+> a lot of control over the Lyra process, probably
+> including arbitrary code execution.
+
+## Setup
+
+1. Install npm
+2. Install dependencies: `npm install`
+
+### Visual Studio Code
+
+If you are using Visual Studio Code, there are a few more optional steps
+you can take to setup your develop environment.
+
+#### Prettier
+
+Install the recommended extension `Prettier - Code formatter`.
+
+#### PlantUML
+
+1. Install the extension `PlantUML` (Id: `jebbs.plantuml`).
+2. Follow its documentation for setting up its requirments.
+
+## Running in development
+
+In the root folder (outside webapp) create file `./config/projects.yaml`
+with example content:
+
+```yaml
+projects:
+  - name: example-unique-name
+    base_branch: main
+    project_path: . # relative path of project from repository root
+    owner: amerharb
+    repo: zetkin.app.zetkin.org
+    host: github.com
+    github_token: << github token >>
+```
+
+⚠️  Note that `repo` must not have the same value for two different projects.
+
+Multiple projects are supported, and they're all stored within the `lyra-projects` folder on the same level as the lyra repository itself.
+
+The project repository (client repository) will be cloned locally (if it does not exist yet) and needs to have a lyra configuration file
+`lyra.yml` or `lyra.yaml` in the root of the repository.
+This lyra configuration file looks like this:
+
+```yaml
+projects:
+  - path: . # relative path to project in repo
+    messages:
+      format: ts
+      path: src # relative path of messages folder relative from above project path
+    translations:
+      path: src/locale # relative path of translations folder relative from above project path
+    languages: # list of language codes supported in the project
+      - sv
+      - de
+    base_branch: main # optional default to 'main'
+```
+
+Start the server with `npm run dev` in `webapp`.
+
+Open the URL provided in the terminal to view projects.
+
+Click on projects to view project pages
+or click on languages to view translation pages.
+
+## Dependencies
+
+Lyra runs on Node and likely uses its HTTP-server to serve requests
+from Internet. This HTTP server will process data before any
+authentication or validation: it is fully exposed. A vulnerability
+in Node can have a severe impact of Lyra, including compromise of
+GitHub authentication tokens, and through those, compromise of at
+least the targeted repositories.
+
+https://nodejs.org/
+
+Under Node, Lyra runs on the framework Next.js, while not as exposed,
+or as privileged as Node, it still processes data before authentication
+and a lot of that data has not been validated by any other program before
+being processed by Next.js. The framework does not have quite as much
+privileges as Node but once we are confident in Node being fully patched
+and known vulnerabilities being mitigated, we should do the same for Next.js
+
+https://nextjs.org/
+
+We have several other direct dependencies that are used by Lyra but
+a vulnerability in any of these will most likely be less severe than
+a vulnerability in Node or in Next.
+
+Our direct dependencies pull in more transivite dependencies. Some of
+these could be more sensitive than our direct dependencies but we will
+likely never have enough resources to analyze them all. With some luck,
+the projects producing our direct dependencies takes some responsibility
+for their dependencies. Keeping our direct dependencies patched reduces
+the risk somewhat of us being affected by public vulnerabilities in our
+transitive dependencies.
+
+Lyra's build and test programs have even more dependencies. These will
+not typically have access to production data or production credentials,
+but they will very likely have access to very powerful developer
+credentials. Tracking published vulnerabilities in all these is beyond
+all hope and feasibility but we can try to keep them somewhat up to date.
+
+## Standalone build (monorepo)
+
+We enable Next.js **standalone output** to keep Docker images tiny.
+Because this package lives in a workspace, we must widen dependency tracing.
+
+### `next.config.js`
+
+```js
+const path = require('path');
+
+/** @type {import('next').NextConfig} */
+module.exports = {
+  output: 'standalone',
+  experimental: {
+    // trace files from the repository root
+    outputFileTracingRoot: path.join(__dirname, '../..'),
+  },
+};
+```
+
+### Gotchas
+
+* Place every module that is **imported at runtime** in this package’s `dependencies` (not `devDependencies`).
+  Example: `typescript` required by `src/utils/readTypedMessages.ts`.
+
+* After upgrading **Next.js** or adding workspace packages, check the bundle:
+
+  ```bash
+  npm run build
+  node --check webapp/.next/standalone/lyra/webapp/server.js
+  ```
+
+### References
+
+* Next.js docs – “output › Caveats”
+  [https://nextjs.org/docs/13/app/api-reference/next-config-js/output#caveats](https://nextjs.org/docs/13/app/api-reference/next-config-js/output#caveats)
+
+* GitHub discussion – deep `webapp/server.js` path (#72436)
+  [https://github.com/vercel/next.js/discussions/72436](https://github.com/vercel/next.js/discussions/72436)
+
+* GitHub issue – public assets missing in monorepo builds (#33895)
+  [https://github.com/vercel/next.js/issues/33895](https://github.com/vercel/next.js/issues/33895)
+
+
+## Docker setup
+
+To run Lyra in a docker container, you need to build the Docker image using the [`Dockerfile`](./Dockerfile) in the root of this repository.
+The [`docker-compose.yaml`](./docker-compose.yaml) file in the root of this repository can be used to build the image and run the image as a container in one command:
+```shell
+$ docker compose up
+```
+
+or in a detached mode:
+```shell
+$ docker compose up -d
+```
+
+### Docker volume mounts
+
+#### SSH key
+Note that in order for the running docker container to be able to interact with the client repository,
+you need to mount a private SSH key of a user with access to the repository into the Docker container.
+Currently, this is achieved by mounting the private SSH key at `~/.ssh/id_rsa` into the container at
+`/home/nodeuser/.ssh/id_rsa`.
+If your SSH key is located elsewhere on your local machine, you will need to adjust the path in the
+[docker-compose.yaml](./docker-compose.yaml) file accordingly.
+⚠️ Note that encrypted private keys are not supported for now.
+
+When mounting the SSH key into the container, the file ownership and permissions from your local system are preserved. 
+Since the container runs as the nodeuser user (UID 1001), but the mounted key is owned by your local user, 
+it’s important to ensure that the SSH key has the correct permissions. 
+SSH requires that private keys are not accessible by others.
+To avoid permission issues, you should set the permissions of your private key to 600 on your local machine:
+
+```bash
+$ chmod 600 ~/.ssh/id_rsa
+```
+
+This ensures that the private key is only readable by the owner, which is sufficient for SSH to accept it inside the 
+container.
+
+File permissions for the private key and lyra-store.json on your local machine also need to allow access for a user ID of 1001.
+
+#### Lyra Store
+
+Before running the container, ensure the file `lyra-store.json` exists on the host system.
+This file is the store for lyra projects and is mounted into the container via docker volume mounts. 
+You can copy this via `cp ./webapp/store.json ~/lyra-store.json` to this location or just change it to use [`webapp/store.json`](./webapp/store.json).
+
+### Release a new container image
+
+The GitHub Actions workflow [`build-and-push-image.yaml`](.github/workflows/build-and-push-image.yaml) is designed to
+automate the process of building, tagging, and pushing a Docker image to the GitHub Container Registry (ghcr.io)
+whenever a new tag is pushed to the repository.
+The tags must follow semantic versioning, while release candidates are supported as well.
+
+Do not forget to document your changes within the [`CHANGELOG.md`](./webapp/CHANGELOG.md) file and adjusting the version within the [`./webapp/package.json`](./webapp/package.json) file.
+
+### Use built image from the container registry
+
+In case you want to use the already built image that is pushed to the GitHub Container Registry, you can adjust the [
+`docker-compose.yaml`](docker-compose.yaml) file as follows (replace `latest` with the version of your preference):
+
+```diff
+services:
+  lyra:
+    container_name: lyra
+-   build:
+-     context: .
++   image: ghcr.io/zetkin/lyra:latest
+    ports:
+      - "3000:3000"
+    environment:
+      - [email protected]
+      - GIT_USER_NAME="Lyra Translator Bot"
+    volumes:
+      - ~/.ssh/id_github:/home/nodeuser/.ssh/id_rsa:ro
+      - ~/lyra-store.json:/app/webapp/store.json
+      - ./config:/app/config
+```

docker-compose.yaml

@@ -0,0 +1,16 @@
+services:
+  lyra:
+    container_name: lyra
+    build:
+      context: .
+    ports:
+      - "3000:3000"
+    environment:
+      - [email protected]
+      - GIT_USER_NAME="Lyra Translator Bot"
+    volumes:
+      # note: adjust the path to the ssh key you want to use to access the git repositories
+      - ~/.ssh/id_rsa:/home/nodeuser/.ssh/id_rsa:ro
+      # note: adjust the path to the lyra-store-file you want to use
+      - ~/lyra-store.json:/app/webapp/store.json
+      - ./config:/app/config

known_hosts

@@ -0,0 +1,3 @@
+github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl
+github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=
+github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=