CI / CD Updates / Project Code Organization -- Multi-Repo attempt #2

[DanVanAtta]

I'm trying to overhaul/fix the CI/CD a little bit. Wanted to open this discussion for suggestions and to kindly please call out the bad ideas

The overall approach is multi-repo. Perhaps this is best explained by noting the repositories, their themes, and what happens as part of their "main build output" (ie: which artifacts and outcomes happen after 'master' branch is updated).

Infrastructure

This repo sets up the servers. That means:

• admin accounts
• all the linux configs
• application install
• applications started & running (and no application restarts)

When master is updated:

• runs 'ansible', which essentially does all of the above in an idempotent way over SSH via python scripts

triplea-game/triplea

• game client with AIs
• http (REST) client libraries to all triplea servers, benefits:
  • Having the REST client SDK be a shared lib between client and server is very convenient and helps keep them in sync.
  • triplea-game/triplea no longer has any dependency on any of the server or the database stuff
    • faster tests
    • faster compile and run times
    • less dependencies
    • docker is no longer a requirement
    • less build outputs & runnables within the project, a simpler project

In this config, we can potentially turn on parallel junit test execution & get some good speedups. The tests with database were a big reason why parallel junit tests did not work, and those tests will no longer be in this repo and therefore not an issue.

Lobby-server

Java code base for lobby server. Has JAR dependency on triplea-game/triplea. Requires developers to set up a github access token and a ~/.gradle/gradle.properties file with the access token.

When master is updated:

• run all the lobby tests that we can
• set up integ tests to start a server & database via docker. Integ tests then become a simpler as they can assume a server is running and not worry about starting or stopping one.
• builds a docker image, stores it in docker registry
• UPDATES THE RUNNING LOBBY to pick up the latest version. Though, until we have zero downtime deployment (a somewhat urgent goal), we need to keep any disruptive restarts of the lobby as an infrequent and/or manual-only process.

Other Repositories

After the above 3 are well established, we'll be at a pretty good milestone. From there, I'm thinking a few more repositories...

Game Support Server

The 'support-server' would have the following endpoints:

• get latest game version
• submit error report (currently exists as part of lobby, we would remove the 'error-report' from the lobby to

Forums

Just be the docker stuff for forums. When master is updated, ideally we publish a docker image & update forums to start running that update (ideally the latter part will be a simply service restart, or invoke an action in the infrastructure repository to do a docker-pull & docker run)

Maps Server

• Endpoints to replace triplea-maps.yaml, game-clients use this information for map downloads

Dice Server

We have one written with node-js, though never launched. I did not realize how much is there. Either the node.js code we have is resurrected or we write something else up.

When master is updated, ideally we instruct the dice-server to fetch the latest updates and then a zero-downtime deployment for the new code to be live.

---

That was kinda a lot to sketch out... Any input/suggestions & help are all welcome!

[asvitkine]

I like the proposal!