[SUREFIRE-1643] surefire junit5 parallel tests

[olamy]

• [SUREFIRE-1643] Make surefire junit5 supporting parallel execution and not mixing result when re running tests

Following this checklist to help us incorporate your
contribution quickly and easily:

• Make sure there is a JIRA issue filed
  for the change (usually before you start working on it). Trivial changes like typos do not
  require a JIRA issue. Your pull request should address just this issue, without
  pulling in other changes.
• Each commit in the pull request should have a meaningful subject line and body.
• Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
  where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
  is to use the JIRA issue title in the pull request title and in the first line of the
  commit message.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Run mvn clean install to make sure basic checks pass. A more thorough check will
  be performed on your pull request automatically.
• You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• In any other case, please file an Apache Individual Contributor License Agreement.

[slawekjaranowski]

first question after code read ...
I will try to test on some of my projects

[slawekjaranowski]

Tested on my private project ... looks ok.

[fslevoaca-ionos]

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml
Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

[olamy]

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

please create another issue with a reproducer.

[fslevoaca-ionos]

Failed tests are no longer displayed inside target/surefire-reports/report.txt/xml Possible the fix for current issue would have introduced this critical regression bug inside maven-surefire 3.5.3.

please create another issue with a reproducer.

Problem is that I do not have the right to open issues inside https://github.com/apache/maven-surefire
Ah I think only in Jira is allowed. On which I have to make a request for Jira account for which I have to wait a few days
Your Jira account request has been successfully verified, and will be reviewed by the project you indicated. Please allow up to a few days for the project to review this request
This is somewhat bureaucratic : ). I might forget to log this issue by the time I get an account : )

[olamy]

approved but definitely need a project to demonstrate the problem you have

[fslevoaca-ionos]

Thanks,
The issue is reproducing only when Cucumber is added into the mix.
I've logged an issue on the Cucumber project
cucumber/cucumber-jvm#2984

However, if this proves to be a Surefire issue, I will move it here.

Might be related to https://issues.apache.org/jira/browse/SUREFIRE-2298

Cheers,

[pzygielo]

For cucumber report, on surefire side it seems to be: 5aeca19 is the first bad commit indeed. (The one reported in SUREFIRE-2298.)

[fslevoaca-ionos]

Opened a separate issue here:
https://issues.apache.org/jira/browse/SUREFIRE-2299
Feel free to close it as duplicated if that is the case.

Thanks,

[hankem]

FYI: This change seems to affect the discovery of ArchUnit tests, see TNG/ArchUnit#1442, where @erwanosouf was kind enough to provide a reproducer.

[olamy]

Yeah sorry for this. I'm looking at some fix for those other cases.

see https://issues.apache.org/jira/browse/SUREFIRE-2298
you might be able to see progress there #828