Shell operator skip tasks from queue

[vasili439]

Hi! I have handler for namespace create event. It just copy 4 secret resources from default namespace to the new one. But when I try to deploy same service (with different vars) to 4 different namespaces I have only 3 new namespaces with secrets and 1 namespace left without them. In logs I have:

{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-sock created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:16Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-ops created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:16Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-sock-ops created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:16Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"Hook executed successfully","queue":"main","task":"HookRun","time":"2020-07-29T05:59:16Z"}
{"binding":"schedule","event.id":"588ae024-14cf-4cea-b845-8e118116b4d6","level":"info","msg":"queue task HookRun:main:kubernetes:namespace-handler.sh:OnCreateDeleteNamespace","operator.component":"handleEvents","queue":"main","task.id":"66160f40-a527-4422-900a-f0c7a2c6fa20","time":"2020-07-29T05:59:17Z"}
{"binding":"schedule","event.id":"5c6e0927-0f23-45ea-a69f-0145a73e8fd8","level":"info","msg":"queue task HookRun:main:kubernetes:namespace-handler.sh:OnCreateDeleteNamespace","operator.component":"handleEvents","queue":"main","task.id":"f58efe1f-5425-416d-8f88-8e3cd0d5a8f3","time":"2020-07-29T05:59:17Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"Execute hook","queue":"main","task":"HookRun","time":"2020-07-29T05:59:19Z"}
{"level":"info","msg":"Will delete 1 tasks from queue 'main'","time":"2020-07-29T05:59:19Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"Namespace customer-authentication-leonru was created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-sock created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-ops created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"secret/docker-registry-sock-ops created","output":"stdout","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}
{"binding":"OnCreateDeleteNamespace","event":"kubernetes","hook":"namespace-handler.sh","level":"info","msg":"Hook executed successfully","queue":"main","task":"HookRun","time":"2020-07-29T05:59:20Z"}

and looks like task for the last namespace was deleted according: {"level":"info","msg":"Will delete 1 tasks from queue 'main'","time":"2020-07-29T05:59:19Z"}

[diafour]

Hello!
The task is not skipped. It is deleted from the queue, but binding contexts are combined. https://github.com/flant/shell-operator/blob/master/HOOKS.md#shell-operator-lifecycle.
Binding contexts from adjacent tasks for the same hook are combined to run hook once with an array of binding contexts. So you need to iterate over the array when handling the binding context. jq '.[0]' $BINDING_CONTEXT_PATH is used in examples for demonstration purposes.

You can use frameworks/shell to properly handle combined contexts. The framework is not documented yet. In short, the hook should look like this:

#!/bin/bash

source /frameworks/shell/context.sh
source /frameworks/shell/hook.sh

function __config__() {
cat <<EOF
  configVersion: v1
  kubernetes:
  - name: OnCreateDeleteNamespace
  ...
  schedule:
  - name: onTime
  ...
EOF
}

# define handlers for each binding:

function __on_kubernetes::OnCreateDeleteNamespace {
 context::get of context::jq to access binding context for this binding
}

function __on_schedule::onTime {
 ...
}

# or use one handler:

function __main__() {
  ...
}

hook::run "$@"

[vasili439]

ok. Thank you for your reply. Will try to iterate through bindings.

[vasili439]

@diafour I have a fix for described situation. Can I commit it to new branch?

[diafour]

Great! Create a new pull request, please.

[vasili439]

#195