Skip to content

Investigate test flakiness #293

Open
@MMMalik

Description

@MMMalik

Are you reporting a feature request or a bug?

Task

Provide detailed reproduction steps (if any)

It happens occasionally that unit tests start failing without any apparent cause even if there were no code changes: #292.

While they are called "unit tests", in fact these tests don't really fulfill the requirements of unit testing:

  • There is a dependency on CDN
  • Singleton CKEDITOR namespace is used across all tests and there is an unreliable teardown for editor instances
  • Test cases are run sequentially
  • Test cases are run in predefined order

Current stack:

  • Karma is used because it allows to easily run tests in real browsers. Real browser is currently necessary due to fact that editor is loaded from CDN.
  • @testing-library/react and related tools are used because they are pretty much staples when it comes to testing React code. In a typical scenario it's usually used with Jest and jsdom but we need a real browser here.
  • Jasmine is used because it's a popular choice for Karma.

Perhaps our current approach can be improved?

Expected result

Unit tests are fast and reliable.

Actual result

Unit tests occasionally fail for unclear reasons.

Metadata

Metadata

Assignees

No one assigned

    Labels

    status:confirmedAn issue confirmed by the development team.type:taskAny other issue (refactoring, typo fix, etc).

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions