Hook up integration test for Alerting Kibana plugin

[ftianli-amzn]

Request adding integration test for Alerting Kibana plugin to the build workflow with and without security

Assumption:
With security: Elasticsearch is running at http://localhost:9200, Kibana is running at http://localhost:5601
Without security: Elasticsearch is running at https://localhost:9200, Kibana is running at http://localhost:5601

Step to run the test:

1. Start Elasticsearch server with Alerting plugin
2. Start Kibana server with Alerting Kibana plugin
3. (Wait for enough time to make sure Kibana server fully started)
4. In the plugin directory (kibana/plugins/alerting-kibana-plugin), run command in below to start the Cypress test

With security: yarn run cypress run --env security_enabled=true
WIthout security: yarn run cypress run

A successful workflow run in Alerting Kibana plugin repo, without security enabled:
https://github.com/opendistro-for-elasticsearch/alerting-kibana-plugin/actions/runs/489045146
The PR to support running Cypress test in ODFE cluster with security enabled for Alerting Kibana plugin:
opendistro-for-elasticsearch/alerting-kibana-plugin#235

[gaiksaya]

Hi @ftianli-amzn ,

I was working on adding Alerting kibana tests. However, the tests are failing when tested against ODFE 1.12.0 (both with and without security). Here is the test run:
With Security: https://github.com/opendistro-for-elasticsearch/opendistro-build/runs/1752329066?check_suite_focus=true
Without Security: https://github.com/opendistro-for-elasticsearch/opendistro-build/runs/1752397189?check_suite_focus=true
Can you confirm if this is expected with ODFE-1.12.0 or do I need to make additional changes to make it work?
Please let me know.
Thanks!

[ftianli-amzn]

Hi @gaiksaya,

Thank you for your time in setting up the workflow according to my request. 😄
It is expected to work with ODFE 1.12.0 without failure, so the failure of one test case is a surprise to me.
I think the configuration for the workflow is correct, as most of the test passes. I will investigate the test cases, and I don't think there are changes for you to make.

[ftianli-amzn]

Hi @gaiksaya,

After my investigation, the test failure reflects a bug in Alering Kibana plugin v1.12.0.2, and I merged the fix into master branch 2 weeks ago. It was a long time so I forgot about the bug.
The test failure is expected in ODFE 1.12.0, I was wrong in my last comment. Thank you!