V2 #68

asilvas · 2015-04-12T21:10:15Z

_Status_: v2.0.0-alpha2 published

So far...

Features:

Proxy support with dynamic app versioning
- Central proxy config file, promotions persisted to disk to survive service restarts
- Isolated worker processes per version
- On demand versioning via x-version header
- Version promotion brings up worker count for desired version and brings down worker count of old version
Worker process downgrade from root via workerGid & workerUid
Backward compatibility with v1.x! Who does that with a major release?!

Enhancements:

Refactored communication between processes
Worker processes may now trigger commands (same) and wait for response (new). The new message bus enables RPC and other flows like this between processes

TODO:

I'd still like to get argument scoping in place to avoid cluster-service arguments not playing nice with the apps arguments. Example: --cs.workerCount 1 instead of --workerCount 1. But if no cs arguments are detected, it'll fall back to legacy mode to avoid breaking existing apps
Tests needed to cover all new proxy features
All tests working again, but more needed

Example Proxy Usage:

cservice:  cluster-service v2.0.0-alpha2 starting...
cservice:  LOCAL ONLY MODE. Run with 'accessKey' option to enable communication channel.
cservice:  CLI is now available. Enter 'help [enter]' for instructions.
cservice:  No workers running. Try 'start server.js'.
cservice> proxy start ./examples/proxy/proxy.json
cservice:  Worker proxy-worker.js(4940) start, reason: Unknown
cservice:  Worker proxy-worker.js(5604) start, reason: Unknown
cservice:  Proxy running on ports 1000,1001
cservice:  Starting workers... timeout: 60000
cservice:  Worker worker.js(2336) start, reason: start
cservice:  Worker worker.js(8380) start, reason: start
cservice:  Worker worker.js(7520) start, reason: start
cservice:  Worker worker.js(8428) start, reason: start
cservice:  Worker worker.js(9592) start, reason: start
cservice:  Worker worker.js(8372) start, reason: start
cservice:  Worker worker.js(10112) start, reason: start
cservice:  Worker worker.js(8652) start, reason: start
cservice> proxy info
cservice:
{ versionPath: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy',
  workerFilename: 'worker.js',
  portRange: { min: 11000, max: 12000 },
  options:
   { defaultVersion: 'v1',
     bindings:
      [ { port: 1000, workerCount: 2 },
        { port: 1001,
          workerCount: 2,
          tlsOptions:
           { key: './examples/certs/test1-key.pem',
             cert: './examples/certs/test1-cert.pem' } } ],
     nonDefaultWorkerCount: 1,
     nonDefaultWorkerIdleTime: 3600 },
  proxyWorkers: [ { port: 1000, ssl: false }, { port: 1001, ssl: true } ],
  versionWorkers:
   [ { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' },
     { worker: 'C:\\_save\\Google Drive\\_save\\dev\\node-cluster-service\\examples\\proxy\\v1\\worker.js',
       version: 'v1' } ] }
cservice> proxy promote v2
cservice:  Starting workers... timeout: 60000
cservice:  Worker worker.js(1892) start, reason: start
cservice:  Worker worker.js(4852) start, reason: start
cservice:  Worker worker.js(10172) start, reason: start
cservice:  Worker worker.js(10004) start, reason: start
cservice:  Worker worker.js(4896) start, reason: start
cservice:  Worker worker.js(7200) start, reason: start
cservice:  Worker worker.js(5464) start, reason: start
cservice:  Worker worker.js(5408) start, reason: start
cservice:  Proxy promoted version v2 successfully
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Worker worker.js(2336) exited, reason: proxy demote
cservice:  Worker worker.js(8380) exited, reason: proxy demote
cservice:  Worker worker.js(8428) exited, reason: proxy demote
cservice:  Worker worker.js(7520) exited, reason: proxy demote
cservice:  Worker worker.js(9592) exited, reason: proxy demote
cservice:  Worker worker.js(8372) exited, reason: proxy demote
cservice:  Proxy demoted old version v1 successfully
cservice:  Worker worker.js(10112) exited, reason: proxy demote
proxy promote v1
cservice:  Starting workers... timeout: 60000
cservice:  Worker worker.js(7948) start, reason: start
cservice:  Worker worker.js(8188) start, reason: start
cservice:  Worker worker.js(11176) start, reason: start
cservice:  Worker worker.js(5308) start, reason: start
cservice:  Worker worker.js(9096) start, reason: start
cservice:  Worker worker.js(10884) start, reason: start
cservice:  Worker worker.js(10732) start, reason: start
cservice:  Proxy promoted version v1 successfully
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Killing workers... timeout: 60000
cservice:  Worker worker.js(4852) exited, reason: proxy demote
cservice:  Worker worker.js(1892) exited, reason: proxy demote
cservice:  Worker worker.js(7200) exited, reason: proxy demote
cservice:  Worker worker.js(10004) exited, reason: proxy demote
cservice:  Worker worker.js(4896) exited, reason: proxy demote
cservice:  Worker worker.js(10172) exited, reason: proxy demote
cservice:  Proxy demoted old version v2 successfully
cservice:  Worker worker.js(5464) exited, reason: proxy demote
shutdown all
cservice:  Killing workers... timeout: 60000
cservice:  Worker worker.js(10732) exited, reason: shutdown
cservice:  Worker worker.js(8188) exited, reason: shutdown
cservice:  Worker proxy-worker.js(5604) exited, reason: shutdown
cservice:  Worker worker.js(10884) exited, reason: shutdown
cservice:  Worker worker.js(9096) exited, reason: shutdown
cservice:  Worker proxy-worker.js(4940) exited, reason: shutdown
cservice:  Worker worker.js(5308) exited, reason: shutdown
cservice:  Worker worker.js(7948) exited, reason: shutdown
cservice:  Worker worker.js(8652) exited, reason: shutdown
cservice:  Worker worker.js(5408) exited, reason: shutdown
cservice:  All workers shutdown. Exiting...
cservice:  'Shutting down...'

…pgrades

scommisso · 2015-11-16T16:48:05Z

lib/commands/restart.js

+        Math.floor(
+            tasks.length * cservice.options.restartConcurrencyRatio
+        );
+    async.parallelLimit(tasks, limit, function(err) {


scommisso · 2015-11-16T16:48:20Z

LGTM 👍

asilvas and others added 13 commits April 12, 2015 13:50

v2-alpha1 in progress

6bd9eeb

merge from master to fix tests

7bd2e80

update changelog

85819e2

update changelog

e1287b7

merge from master

4699cd1

support for proxy.nonDefaultWorkerIdleTime. Proxy crash under load

ed65950

formatting test fixes

53d242f

#72. Fix crash on process.send using fail safe

4a23deb

version bump

49c8a4e

push to public npm, always

f08be02

#71. Add restartConcurrencyRatio option of 33% to expedite restarts/u…

8635789

…pgrades

#71. update readme to include restartConcurrencyRatio

6ca4069

fix style rules

cfec415

scommisso reviewed Nov 16, 2015
View reviewed changes

asilvas-godaddy and others added 5 commits November 18, 2015 09:14

version bump

8ca11b2

latest changes

316de2b

typo

da80fe0

fix upgrade usage of extend. more than 2 args not supported

79af70e

Merge branch 'v2' of github.com:godaddy/node-cluster-service into v2

5d2a734

asilvas merged commit 9a44604 into master Sep 1, 2016

asilvas deleted the v2 branch September 1, 2016 23:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V2 #68

V2 #68

asilvas commented Apr 12, 2015

scommisso Nov 16, 2015

scommisso commented Nov 16, 2015

V2 #68

V2 #68

Conversation

asilvas commented Apr 12, 2015

Features:

Enhancements:

TODO:

Example Proxy Usage:

scommisso Nov 16, 2015

Choose a reason for hiding this comment

scommisso commented Nov 16, 2015