Skip to content

Create test pass/fail checker #3

Open
@multiplemonomials

Description

@multiplemonomials

I'd like to make a script that compares two SQLite databases containing test results (the old and the new test results) with each other. The comparison shall pass if:

  • for each target T in both old and new:
  • Each test case which passed for T in old is also a pass in new

This logic is a bit more lenient than your average CI runner -- it allows tests to fail that have always been failing, and it allows new tests to be added which don't pass on every device. All it does is make sure that everything that used to work in the old version still works!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions