Test skip when class methods used more than 3 levels priorities and run by XML with package type.

[JFRabbit]

TestNG Version

6.14.3

Is the issue reproductible on runner?

• IntelliJ

Test case sample

XML:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="TestPasXML" verbose="1">
    <test name="Regression">
        <packages>
            <package name="tmp"/>
        </packages>
    </test>
</suite>

2 classes:

package tmp;

import org.testng.annotations.Test;

public class TestNormalClass {

    @Test
    public void a() {
        System.out.println("a");
    }
}

package tmp;

import org.testng.annotations.Test;

public class TestErrorClass {

    @Test(priority = 1)
    public void b() {
        System.out.println("b");
    }

    @Test(priority = 2)
    public void c() {
        System.out.println("c");
    }

    @Test(priority = 3)
    public void d() {
        System.out.println("d");
    }
}

• Case1: run class "TestNormalClass" by idea
  => OK
• Case2: run class "TestErrorClass" by idea
  => OK
• Case3: run XML
  => class "TestErrorClass" ignored

• Case4: remove "priority = 1" or remove one method of class "TestErrorClass", run XML
  ==> OK

[krmahadevan]

@JFRabbit - The latest released version of TestNG is 7.0.0-beta5. Please retry using that and post back your results.

Please note, that the TestNG team does not own the IntelliJ testng plugin. Please try running your test via a suite xml file. If it works via the suite xml file, but doesn't using the IntelliJ plugin, then you may have to file a bug with the IntelliJ team to get it sorted out there.

[JFRabbit]

@krmahadevan - I can't found 7.0.0-beta5, and I did below:

• run by a jar package, reproduce this issue on version 6.14.3 .
• run with version 7.0.0-beta3, can't reproduce. But it looks like that "priority" doesn't work on global, mean when run XML suite, ClassA.test() will be scheduled first than ClassB.test() with lower priority. On version 6.14.3, doc of "priority" is "The scheduling priority. Lower priorities will be scheduled first."

package tmp;

import org.testng.annotations.Test;

public class ClassAWithLowerPriority {

    @Test(priority = 1)
    public void lower(){
        System.out.println("lower");
    }

}

package tmp;

import org.testng.annotations.Test;

public class ClassBWithHigherPriority {

    @Test(priority = 2)
    public void higher(){
        System.out.println("higher");
    }

}

run result:
higher
lower

[krmahadevan]

@JFRabbit - I am still not able to understand what the issue is.

Can you please do the following

1. Create a sample project that uses TestNG 7.0.0-beta5 [ Its available in bintray here ]
2. Share that sample project here either as a github link or as a zip file
3. Include the expected behavior.

Reproducing this issue with 6.14.3 is not going to help, because we can't fix anything in that version.

[JFRabbit]

@krmahadevan - I upload a maven project with zip:
foobar.zip

You can run below cases and compare difference between version 7.0.0-beta5 and 6.14.3:

Case1:

• run foobar/foobar.xml with version 7.0.0-beta5:
  you can get result:
  priority 1
  priority 2
  priority -1

• run foobar/foobar.xml with version 6.14.3:
  you can get result:
  priority -1
  priority 1
  priority 2

Case2:
change foobar/foobar.xml line 6:

<package name="tmp.old"/>

• run foobar/foobar.xml with version 7.0.0-beta5:
  you can get result:
  priority 1
  priority 2
  priority 3
  priority 4
  priority -1

• run foobar/foobar.xml with version 6.14.3:
  you can get result:
  priority -1
  Test ignored.

The case2 proved issue doesn't reproduce on version 7.0.0-beta5.
The case1 proved function of "priority" have changed between 6.14.3 and 7.0.0-beta5.

So I think this issue doesn't reproduce on version 7.0.0-beta5 cause the change of "priority", doesn't mean fixed. Maybe the "change" is also a bug. Please confirm.

[juherr]

Thanks for the sample.
The behavior is strange and not expected but the initial issue looks already be fixed :)
Next step, manage -1 priority.

@krmahadevan Any time to have a look on it?

[krmahadevan]

@juherr - I will try to get to this. A bit held up at the personal front.