@BeforeClass methods in base class with dependsOnGroups and groups are not executed in the expected order.

[MateuszStefek]

This test case demonstrates an issue with the execution order of @BeforeClass methods when inheritance, dependsOnGroups, and groups attributes are used.

A @BeforeClass method in a parent class that has dependsOnGroups isn't running before a @BeforeClass method in the child class, even though TestNG's rules are that all @BeforeClass methods from parent classes should run before those in the child class.

Also, when the name of ZBaseClass is changed to ABaseClass, the test passes. This suggests that the way TestNG plans the method execution might be affected by the initial sorting order of methods (see MethodSorting.Instances).

import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import org.assertj.core.api.Assertions;
import java.util.ArrayList;
import java.util.List;

public class BeforeClassOrderingTest extends ZBaseClass {
	@BeforeClass
	public void thisSetup() {
		methodInvocationLog.add("thisSetup");
	}

	@Test
	public void executionOrderOfBeforeClassMethodShouldBeCorrect() {
		Assertions.assertThat(methodInvocationLog)
				.describedAs("@BeforeClass methods invocation order")
				.containsExactly("zSetup", "ySetup", "thisSetup");
	}
}

// The names of the class and the methods are intentional to detect unwanted ordering
abstract class ZBaseClass {
	protected List<String> methodInvocationLog = new ArrayList<>();

	@BeforeClass(alwaysRun = true, dependsOnGroups = "SomeTestGroup")
	protected final void ySetup() {
		methodInvocationLog.add("ySetup");
	}

	@BeforeClass(alwaysRun = true, groups = "SomeTestGroup")
	protected final void zSetup() {
		methodInvocationLog.add("zSetup");
	}
}

The test fails, showing that the order is ["zSetup", "thisSetup", "ySetup"], which doesn’t make any logical sense

Actual and expected have the same elements but not in the same order, at index 1 actual element was:
  <"thisSetup">
whereas expected element was:
  <"ySetup">

Environment

• TestNG 7.11.0
• Java 17

When noticed

We noticed this while upgrading TestNG 6.9.10 -> 7.11.0.
Imagine, debugging this issue was quite painful. I learned more about TestNG than was anticipated :).