Skip to content

Run the integration tests in a random order #19065

Closed
@timvandermeij

Description

@timvandermeij

The integration tests currently run in a fixed order, see https://github.com/mozilla/pdf.js/blob/master/test/integration-boot.mjs#L24, which is in contrast with e.g. the unit test that deliberately run in a random order to bring any isolation issues (dependencies between tests) to the surface. Most likely this wasn't done for the integration tests yet because we used to have quite a lot of intermittent issues and dependencies between tests when we started with the integration tests, but over time we have fixed most of them so hopefully running the integration tests in a random order is more feasible nowadays.

We should try this out and make a list of any issues we encounter. This could involve closing the document between tests in the same describe block to avoid state leaking through to subsequent tests; see e.g. PR #19064 for which an alternative solution was implemented.

The objective is to reduce the number of test failures (currently if there is a failure in one test it'll currently cause all subsequent dependent tests to fail too) and to improve debugging of the remaining intermittents (currently we sometimes need to run a full describe block only to debug one test which is time-consuming).

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions