Support multiple jitsu apps in the same package.json

[jwietelmann]

I imagine there's some closed-source work that would be required to make this happen, but this looked like the best place to document it.

What I want is to be able to jitsu deploy production where production maps to a different app. This better reflects the git workflow of many organizations and the next logical step would be for the webhooks to allow me to have one branch continuously deploy to one app and another to continuously deploy to another app. All without constantly having to change and resolve conflicts in package.json.

Consider the following setup: I have master branch (myapp), staging branch (myapp_staging), and feature branches (don't get deployed anywhere). When I merge feature -> staging, it could deploy to myapp_staging, and when I merge staging -> master it could deploy myapp.

Even if you want to manually use this kind of git workflow, right now you need to constantly change the package.json back and forth or remove it with .gitignore altogether, in which case you lose dependency versioning.

So anyway, I propose that jitsu's extensions to package.json be refactored to allow multiple, labeled apps so that we can jitsu deploy <app label> explicitly.

Here's just a rough example package.json I threw together:

{
  "version": "0.0.1-37",
  "apps": {
    "default": {
      "name": "myapp_staging",
      "subdomain": "myapp_staging",
      "private": true
    },
    "production": {
      "name": "myapp_production",
      "subdomain": "myapp"
    }
  },
  "scripts": {
    "start": "node app.js"
  },
  "engines": {
    "node": "0.8.x"
  },
  "dependencies": {
    "express": "3.0.3",
    "jade": "*",
    "less": "*",
    "mongoose": "*"
  }
}

[coen-hyde]

This is attempting to tackle the same issue as #292. While I prefer issue's #292 approach I definitely would like some support from nodejitsu built into jitsu to help dealing with multiple environment deploys.

[jwietelmann]

Yeah honestly it might be better to get jitsu's config out of package.json altogether. But at a minimum there needs to be some sort of support for deploying to different apps without constantly changing the package.json file. It just makes business sense, too. If Nodejitsu wants enterprise customers, jitsu has to support your typical production/staging/QA/etc. corporate setup in a way that doesn't interfere with that workflow. Making devs change and git add the package.json file in order to deploy to a different endpoint seriously screws with version control and workflow.

[coen-hyde]

I've got a build process that builds the correct package.json, which is then git ignored. But it feels rather dirty. And it was more effort to setup than i'd like.

[constantx]

+1 (same as #352)

[sbabigian]

+1 for multiple app targets. I agree all around. This is really making me consider Heroku instead for enterprise code where we have dev, prod, testing environments each on its own git branch.

[blakmatrix]

This is blocked pending business plan milestones, so we're looking at a month or two before we can make such a large closed sourced feature available.

[dscape]

+1

[indexzero]

@jwietelmann What does private: true do in app: { default: { ... ??

[Southern]

@indexzero I believe he's talking about making it where you can also use this in NPM. It wouldn't be compatible with NPM anyway, because it would ignore the apps field... and that's a change that would have to be made with NPM, and probably not that well received. I also noticed this package.json doesn't have a name field in the root of the object. They're all under apps, so NPM would have to conform to this.. otherwise we get:

 ~ ❱ echo "{}" > package.json && cat package.json
{}
 ~ ❱ npm pack
npm ERR! addLocal Could not install .
npm ERR! Error: No 'name' field
npm ERR!     at validName (/usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:557:40)
npm ERR!     at final (/usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:340:27)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:140:33
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/slide/lib/async-map.js:48:11)
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/read-package-json/read-json.js:320:48
npm ERR!     at fs.js:117:20
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:53:5
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:62:5
npm ERR!     at Object.oncomplete (fs.js:297:15)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>

Ignoring the local error, name is the first thing NPM looks for.

@coen-hyde We could still use the fields in #292 in the apps field. @blakmatrix offered a jitsu.conf file rather than package.json.

I would suggest something like:

{
  "name": "test",
  "version": "0.0.1",
  "description": "",
  "main": "server.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node server.js"
  },
  "repository": "git://github.com/random/repo",
  "author": "Your Name",
  "license": "Your License",
  "engines": {
    "node": "0.8.16"
  },
  "dependencies": {
    "our-dep": "0.0.1"
  },
  "environments": {
    "default": "development",
    "development": {
      "name": "test-development",
      "subdomain": "test.dev",
      "version": "0.0.2",
      "dependencies": {
        "our-dep": "0.0.2",
        "our-dep2": "0.0.1"
      },
      "domains": [
        "dev.yourdomain.com"
      ]
    },
    "production": {
      "name": "test-production",
      "subdomain": "test.pro",
      "domains": [
        "yourdomain.com",
        "www.yourdomain.com"
      ]
    }
  }
}

Then make jitsu deploy <environment> and just overwrite any keys in the root that it's given. That uses the same type of layout as #292, and wouldn't cause problems if npm pack is called by itself. Calling jitsu deploy without an environment would use environments.default, so that we always have an environment to append changes to. That way we can leave the original NPM version alone when bumping during deploys.

Using the above package.json (minus the fake dependencies):

 ~/dev/node/projects/test ❱ tar -tf $(npm pack)
npm WARN package.json test@0.0.1 No README.md file found!
package/package.json
package/server.js
package/test-0.0.1.tgz

[coen-hyde]

@Southern I like it.

[indexzero]

There is value here, but everything I've seen it way over complicated. Lets talk about this at a higher level than what the package.json should look like and I believe the changes to the package.json will become obvious.

Users want to run the same application at multiple stages of the development process:
e.g. "I have this app running in production on Nodejitsu, but I want to test the new version before it goes live"

Given this fact the following properties seem totally unnecessary when considering a given application:

  {
    name: "myapp"
    subdomain: "myapp",
    domains: ["www.custom-domain.com"],
    version: "1.0.0",
    dependencies: {
      "our-dep": "0.0.2",
      "our-dep2": "0.0.1"
    },
  }

1. name, version, subdomain, domains: I don't see any reason why the name, version, subdomains and domains of these non-production copies of an application cannot just be derived from the name of the copy itself e.g.
   • name: myapp-test is the "test" copy of myapp
   • version: 1.0.0-test is the "test" copy of myapp@1.0.0
   • subdomain: test.myapp.jitsu.com is the test copy of myapp.jitsu.com
   • domain: test.www.custom-domain.com is the test copy of the custom domain.
2. Copies will only be allowed for the latest snapshot of an app. Once you deploy to the main copy, everything is blown away. Why would you want multiple things out of sync?

So the only actual chance to the package.json format I see is somewhere to specify dependencies for a given environment.

[Southern]

@indexzero That's only one environment, though. I've had at least one person question how to run 3 environments at once. I want to say they were like.. staging, q&a and production.

[indexzero]

@Southern simple:

  {
    name: "myapp"
    subdomain: "myapp",
    domains: ["www.custom-domain.com"],
    version: "1.0.0",
    environments: ['qa', 'staging', 'production']
    dependencies: {
      "our-dep": "0.0.2",
      "our-dep2": "0.0.1"
    },
  }

[coen-hyde]

@indexzero Previously you mentioned having only copies for the latest snapshot then blowing them away once you deploy the main copy. How would you deal with hosting feature branches then? They'd be blown away each time the main trunk is deployed. How about we do this:

$ jitsu deploy [environment]

Where environment can be whatever you want. There would be no need for listing environments in the package.json. Then use all your predetermined attributes you mentioned above. eg
[environment].myapp.jitsu.com

It would then be up to the user to decide their workflow.

[coen-hyde]

Something else to consider is environment variables. Each environment would need it's own environment variable space for things like database credentials.