Multiple failures in testsuite for failures under single testcase

[sayantansls]

Hi all,
I am using jasmine-reporters for generating XML reports for protractor E2E tests which are then being used to create HTML reports. There seems to be a bug in the implementation. So, I have a test suite with 3 test cases, of which 2 test cases are success and 1 test case fails. On running the tests with the JUnitXmlReporter, I am getting the following XML node for the particular testsuite.

<testsuite name="Ingestion" timestamp="2022-02-20T02:59:46" hostname="localhost" time="98.866" errors="0" tests="3" skipped="0" disabled="0" failures="3">
  <testcase classname="Ingestion" name="should ingest file type 1" time="25.266"/>
  <testcase classname="Ingestion" name="should ingest file type 2" time="26.258"/>
  <testcase classname="Ingestion" name="should ingest file type 3" time="13.548">
    <failure type="toBeTruthy" message="Expected false to be truthy."><![CDATA[
   ]]></failure>
    <failure type="exception" message="Failed: No element found using locator: By(xpath, //section[@data-test=&quot;sample-classification&quot;]//mat-select//div//span)"><![CDATA[
   ]]></failure>
    <failure type="exception" message="Failed: element click intercepted: Element &lt;button _ngcontent-keo-c276=&quot;&quot; data-test=&quot;close-dialog&quot; mat-icon-button=&quot;&quot; class=&quot;mat-focus-indicator mat-icon-button mat-button-base&quot; style=&quot;margin-bottom: 5px;&quot;&gt;...&lt;/button&gt; is not clickable at point (1366, 102). Other element would receive the click: &lt;section _ngcontent-keo-c276=&quot;&quot; class=&quot;wrapper active ng-star-inserted&quot; style=&quot;&quot;&gt;&lt;/section&gt;
  (Session info: chrome=97.0.4692.71)
  (Driver info: chromedriver=97.0.4692.71 (adefa7837d02a07a604c1e6eff0b3a09422ab88d-refs/branch-heads/4692@{#1247}),platform=Linux 3.10.0-1160.49.1.el7.x86_64 x86_64)"><![CDATA[
   ]]></failure>
  </testcase>
</testsuite>

So, as visible, two of the three tests have succeeded, however, since the failed test case has 3 failure nodes under it, so the testsuite tag shows the failures attribute as 3, which is an incorrect representation, that gets carried on to my HTML report. I think this line 279 in the file junit_reporter.js is causing this bug.

if (isFailed(spec)) { spec._suite._failures += spec.failedExpectations.length; }

Do let me know if this is by design or is an issue that could potentially be fixed, thanks.

[putermancer]

It’s not intentional. I’d need to go re read the spec again, but I suspect failures is supposed to represent the number of specs (it) that failed, not the number of expectations. There is also a good chance this behavior is similarly flawed in the other reporters. If you make a pull request to address this, please consider each of the reporters in turn. This project is in heavy maintenance mode, so a pull request is the best option to ensure timely resolution for your use case.

…

On Sat, Feb 19, 2022 at 12:43 PM Sayantan Ghosh ***@***.***> wrote: Hi all, I am using jasmine-reporters for generating XML reports for protractor E2E tests which are then being used to create HTML reports. There seems to be a bug in the implementation. So, I have a test suite with 3 test cases, of which 2 test cases are success and 1 test case fails. On running the tests with the JUnitXmlReporter, I am getting the following XML node for the particular testsuite. <testsuite name="Ingestion" timestamp="2022-02-20T02:59:46" hostname="localhost" time="98.866" errors="0" tests="3" skipped="0" disabled="0" failures="3"> <testcase classname="Ingestion" name="should ingest file type 1" time="25.266"/> <testcase classname="Ingestion" name="should ingest file type 2" time="26.258"/> <testcase classname="Ingestion" name="should ingest file type 3" time="13.548"> <failure type="toBeTruthy" message="Expected false to be truthy."><![CDATA[ ]]></failure> <failure type="exception" message="Failed: No element found using locator: By(xpath, ***@***.***=&quot;sample-classification&quot;]//mat-select//div//span)"><![CDATA[ ]]></failure> <failure type="exception" message="Failed: element click intercepted: Element &lt;button _ngcontent-keo-c276=&quot;&quot; data-test=&quot;close-dialog&quot; mat-icon-button=&quot;&quot; class=&quot;mat-focus-indicator mat-icon-button mat-button-base&quot; style=&quot;margin-bottom: 5px;&quot;&gt;...&lt;/button&gt; is not clickable at point (1366, 102). Other element would receive the click: &lt;section _ngcontent-keo-c276=&quot;&quot; class=&quot;wrapper active ng-star-inserted&quot; style=&quot;&quot;&gt;&lt;/section&gt; (Session info: chrome=97.0.4692.71) (Driver info: chromedriver=97.0.4692.71 (adefa7837d02a07a604c1e6eff0b3a09422ab88d-refs/branch-heads/4692@{#1247}),platform=Linux 3.10.0-1160.49.1.el7.x86_64 x86_64)"><![CDATA[ ]]></failure> </testcase> </testsuite> So, as visible, two of the three tests have succeeded, however, since the failed test case has 3 failure nodes under it, so the testsuite tag shows the failures attribute as 3, which is an incorrect representation, that gets carried on to my HTML report. I think this line 279 in the file junit_reporter.js is causing this bug. if (isFailed(spec)) { spec._suite._failures += spec.failedExpectations.length; } Do let me know if this is by design or is an issue that could potentially be fixed, thanks. — Reply to this email directly, view it on GitHub <#209>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACRTNXETKKI2T3CYZNCE6DU37XFTANCNFSM5O3K3AKQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>