Skip to content

Add JUnit Test Reports to build outputs. #15271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Aug 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
79b2c3b
Comment out logging on the test for now as they are very verbose and …
davinchia Aug 3, 2022
705e51f
Test job summary.
davinchia Aug 3, 2022
d45fdfc
Format.
davinchia Aug 3, 2022
ee0f851
Test.
davinchia Aug 3, 2022
15a38e3
Force fail.
davinchia Aug 3, 2022
5288e45
Test using test reporter action.
davinchia Aug 3, 2022
c6bf4a0
Merge branch 'master' into davinchia/remove-unnecessary-test-logging
davinchia Aug 3, 2022
a9a7503
Look at the xml test report instead.
davinchia Aug 3, 2022
4a9db01
Merge branch 'davinchia/remove-unnecessary-test-logging' of github.co…
davinchia Aug 3, 2022
f66e8d4
This should be false.
davinchia Aug 3, 2022
5ef390d
make this faster.
davinchia Aug 3, 2022
aab4585
Force error in a submodule.
davinchia Aug 3, 2022
87c58e9
Only list failed suites.
davinchia Aug 3, 2022
76d89ae
Only list failed tests.
davinchia Aug 3, 2022
4b96c54
Testing for second level test failure detection.
davinchia Aug 3, 2022
c53791d
Comment this out to avoid PMD.
davinchia Aug 3, 2022
6518cc4
Does this help the action pick up subfolder test failures?
davinchia Aug 3, 2022
7477215
Add test report generation for all jobs and add fake tests to test this.
davinchia Aug 4, 2022
dd6de94
Modify kubernetes test report search path.
davinchia Aug 5, 2022
4ebb4ec
Does spaces mess this up?
davinchia Aug 5, 2022
06fda55
Remove all testing tests.
davinchia Aug 5, 2022
d4ece26
Merge branch 'master' into davinchia/remove-unnecessary-test-logging
davinchia Aug 5, 2022
e38655f
Remove unused imports.
davinchia Aug 5, 2022
950edbf
Merge branch 'davinchia/remove-unnecessary-test-logging' of github.co…
davinchia Aug 5, 2022
0c6b4f8
Correct paths.
davinchia Aug 5, 2022
cecb95b
Merge branch 'master' into davinchia/remove-unnecessary-test-logging
davinchia Aug 5, 2022
972605d
Remove old tee.
davinchia Aug 5, 2022
21c11f6
Merge branch 'davinchia/remove-unnecessary-test-logging' of github.co…
davinchia Aug 5, 2022
0548ccb
Merge branch 'master' into davinchia/remove-unnecessary-test-logging
davinchia Aug 6, 2022
9c2f82f
Do migration tests still fail?
davinchia Aug 6, 2022
089d2ae
Merge branch 'master' into davinchia/remove-unnecessary-test-logging
davinchia Aug 8, 2022
d3bbcbb
Remove E2E test to see if this speeds things up.
davinchia Aug 8, 2022
ad019a9
Put the test report generation back. Comment out the frontend e2e tes…
davinchia Aug 8, 2022
d0ff201
Increase the wait time.
davinchia Aug 8, 2022
3485b80
Remove testing stuff.
davinchia Aug 8, 2022
b06eeba
Remove testing stuff.
davinchia Aug 8, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 31 additions & 4 deletions .github/workflows/gradle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,15 @@ jobs:
- name: Ensure no file change
run: git --no-pager diff && test -z "$(git --no-pager diff)"

- name: Generate Test Report
uses: dorny/test-reporter@v1
if: always()
with:
name: Connectors Base Test Report
# Specify top-level and second-level modules. Note there cannot be a space between the comma.
path: '/actions-runner/_work/airbyte/airbyte/*/build/test-results/*/*.xml,/actions-runner/_work/airbyte/airbyte/*/*/build/test-results/*/*.xml'
reporter: java-junit

# In case of self-hosted EC2 errors, remove this block.
stop-connectors-base-build-runner:
name: "Connectors Base: Stop Build EC2 Runner"
Expand Down Expand Up @@ -471,7 +480,8 @@ jobs:
SUB_BUILD=PLATFORM ./gradlew build javadoc --scan

- name: Integration test
run: SUB_BUILD=PLATFORM ./gradlew newIntegrationTest
run: |
SUB_BUILD=PLATFORM ./gradlew newIntegrationTest

- name: Slow integration test
if: contains(github.ref, 'bump-version') || contains(github.ref, 'master')
Expand All @@ -485,16 +495,25 @@ jobs:
if: success() && github.ref == 'refs/heads/master'
run: ./tools/site/link_checker.sh check_docs

# This is only required on the usual github runner. The usual runner does not contain enough disk space for our use.
# - name: Get Docker Space
# run: docker run --rm busybox df -h
# This is only required on the usual github runner. The usual runner does not contain enough disk space for our use.
# - name: Get Docker Space
# run: docker run --rm busybox df -h

- name: Run End-to-End Acceptance Tests
run: ./tools/bin/acceptance_test.sh

- name: Automatic Migration Acceptance Test
run: SUB_BUILD=PLATFORM ./gradlew :airbyte-tests:automaticMigrationAcceptanceTest --scan -i

- name: Generate Test Report
uses: dorny/test-reporter@v1
if: always() # run this step even if previous step failed
with:
name: Platform Test Report with Docker E2E Test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really trying to take to heart the idea that we strive to communicate as clearly and loudly as possible when showing developers how to fix their own problems.

is there a name for this test that more clearly screams what it is? How can we get every developer to know there is a better error message process over here?

# Specify top-level and second-level modules. Note there cannot be a space between the comma.
path: '/actions-runner/_work/airbyte/airbyte/*/build/test-results/*/*.xml,/actions-runner/_work/airbyte/airbyte/*/*/build/test-results/*/*.xml'
reporter: java-junit

# In case of self-hosted EC2 errors, remove this block.
stop-platform-build-runner:
name: "Platform: Stop Build EC2 Runner"
Expand Down Expand Up @@ -626,6 +645,14 @@ jobs:
run: |
CI=true IS_MINIKUBE=true ./tools/bin/acceptance_test_kube.sh

- name: Generate Test Report
uses: dorny/test-reporter@v1
if: always() # run this step even if previous step failed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like the note

with:
name: Platform Kubernetes E2E Test Report
path: '/actions-runner/_work/airbyte/airbyte/*/build/test-results/*/*.xml'
reporter: java-junit

- uses: actions/upload-artifact@v2
if: failure()
with:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ public void startBlocking() throws IOException, InterruptedException {
@SuppressWarnings({"unchecked", "rawtypes"})
public void startAsync() throws IOException, InterruptedException {
final File cleanedDockerComposeFile = prepareDockerComposeFile(dockerComposeFile);
dockerComposeContainer = new DockerComposeContainer(cleanedDockerComposeFile).withEnv(env);
dockerComposeContainer = new DockerComposeContainer(cleanedDockerComposeFile)
.withEnv(env);

// Only expose logs related to db migrations.
serviceLogConsumer(dockerComposeContainer, "init");
serviceLogConsumer(dockerComposeContainer, "bootloader");
Expand Down
2 changes: 1 addition & 1 deletion tools/bin/acceptance_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ VERSION=dev TRACKING_STRATEGY=logging USE_STREAM_CAPABLE_STATE=true docker-compo
# Sometimes source/dest containers using airbyte volumes survive shutdown, which need to be killed in order to shut down properly.
shutdown_cmd="docker-compose down -v || docker kill \$(docker ps -a -f volume=airbyte_workspace -f volume=airbyte_data -f volume=airbyte_db -q) && docker-compose down -v"
# Uncomment for debugging. Warning, this is verbose.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice add for UX

#trap "echo 'docker-compose logs:' && docker-compose logs -t --tail 1000 && $shutdown_cmd" EXIT
# trap "echo 'docker-compose logs:' && docker-compose logs -t --tail 1000 && $shutdown_cmd" EXIT

echo "Waiting for services to begin"
while [[ "$(curl -s -o /dev/null -w ''%{http_code}'' localhost:8000/api/v1/health)" != "200" ]]; do echo "Waiting for docker deployment.."; sleep 5; done
Expand Down
3 changes: 2 additions & 1 deletion tools/bin/e2e_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ mkdir -p /tmp/airbyte_local

# Detach so we can run subsequent commands
VERSION=dev TRACKING_STRATEGY=logging docker-compose up -d
trap 'echo "docker-compose logs:" && docker-compose logs -t --tail 1000 && docker-compose down && docker stop airbyte_ci_pg' EXIT
# Uncomment for debugging. Warning, this is verbose.
# trap 'echo "docker-compose logs:" && docker-compose logs -t --tail 1000 && docker-compose down && docker stop airbyte_ci_pg' EXIT

docker run --rm -d -p 5433:5432 -e POSTGRES_PASSWORD=secret_password -e POSTGRES_DB=airbyte_ci --name airbyte_ci_pg postgres
echo "Waiting for health API to be available..."
Expand Down