New Scheduler with CouchDB backend

[hunhoffe]

Environment details:

• Local ansible deployment
• Ubuntu 20.04, Docker 20.10.12

Steps to reproduce the issue:

1. Follow Ansible deployment instructions for new Scheduler
2. Modify them to deploy with couchdb backend and not Elastic Search
3. Will hit error at the following command: ansible-playbook -i environments/$ENVIRONMENT openwhisk.yml -e db_activation_backend=CouchDB

Provide the expected results and outputs:

I'd expected it to deploy correctly.

Provide the actual results and outputs:

The scheduler will fail to deploy. The error ansible gives is:

fatal: [scheduler0]: FAILED! => {"attempts": 12, "changed": false, "elapsed": 0, "msg": "Status code was -1 and not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://172.17.0.1:14001/ping"}

Status code was -1 and not [200]: Request failed: <urlopen error [Errno
111] Connection refused>

This isn't very helpful, I don't think, but the scheduler logs are more informative:

[2022-04-25T18:10:32.291Z] [INFO] [#tid_sid_unknown] [WatcherService] watch endpoint: WatchEndpoint(whisk/instance/scheduler0/lease,7802908987417965063,false,lease-service,Set(DeleteEvent))
Exception in thread "main" java.lang.ExceptionInInitializerError
        at java.base/java.lang.J9VMInternals.ensureError(J9VMInternals.java:186)
        at java.base/java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:175)
        at org.apache.openwhisk.core.scheduler.queue.ElasticSearchDurationCheckerProvider$.instance(ElasticSearchDurationChecker.scala:121)
        at org.apache.openwhisk.core.scheduler.queue.ElasticSearchDurationCheckerProvider$.instance(ElasticSearchDurationChecker.scala:112)
        at org.apache.openwhisk.core.scheduler.Scheduler.<init>(Scheduler.scala:91)
        at org.apache.openwhisk.core.scheduler.Scheduler$.main(Scheduler.scala:339)
        at org.apache.openwhisk.core.scheduler.Scheduler.main(Scheduler.scala)
Caused by: pureconfig.error.ConfigReaderException: Cannot convert configuration to a org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStoreConfig. Failures are:
  at 'whisk':
    - Key not found: 'activation-store'.

        at pureconfig.package$.getResultOrThrow(package.scala:139)
        at pureconfig.package$.loadConfigOrThrow(package.scala:161)
        at org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore$.<init>(ElasticSearchActivationStore.scala:420)
        at org.apache.openwhisk.core.database.elasticsearch.ElasticSearchActivationStore$.<clinit>(ElasticSearchActivationStore.scala)
        ... 5 more

It appears the scheduler is looking for ElasticSearch backend, and not CouchDB.

I'm hoping to learn if the scheduler can function with CouchDB as an activation store or not.

Additional information you deem important:

• Ansible scheduler documentation says it's recommended to use ElasticSearch at the activation store but does not say it's required.

[hunhoffe]

In the configuration instructions, I replaced the ElasticSearchDurationCheckerProvider with NoopDurationCheckerProvider, because this seemed a reasonable thing to do to try to fix the issue above.

However, the error remained the same even after that change.

[bdoyle0182]

I'm in the process of setting up the new scheduler and haven't had any issue with the components at least talking to one another. I'm using the NoopDurationCheckerProvider since we don't use elastic search. You should be able to use the new scheduler with activations in couchdb. What is the new error and what does your config look like for both the duration checker and configuring the activation store?

[hunhoffe]

Thank you for your response @bdoyle0182! I refreshed my environment and it appears that something just hadn't updated after I'd changed to the NoopDurationCheckerProvider. So I suppose I'd already fixed my own issue.

I updated the ansible directions in a PR to make it clearer that you need to change the DurationChecker.