Test our JavaScript with Playwright

[choldgraf]

Description

Currently, we only test things that can be statically-generated by Python, using pytest.

However, a lot of the functionality in this theme comes from JavaScript, which we cannot test simply by looking at the statically-generated HTML files. We currently do not test for interactive things like the dropdown switcher, which makes us more likely to miss regressions and bugs.

Proposed solution

We should investigate using the playwright pytest plugin to do basic "live behavior" testing of important aspects of the theme. playwright can install its own bundled version of chromium and is very well supported.

Here's an example of their pytest plugin in-action from their docs:

# test_my_application.py
def test_example_is_working(page):
    page.goto("https://example.com")
    assert page.inner_text('h1') == 'Example Domain'
    page.click("text=More information")

I think we'd have access to anything in the page object. So for example we could check the inner HTML of a page after it was loaded in chromium. This could let us test for things in a more reliable fashion.

[jorisvandenbossche]

It would indeed be very good to have such testing. I don't have any experience with this, so if playwright seems a good solution for this, that sounds good to me

[choldgraf]

It is at least what most people on Twitter recommended :-)

[damianavila]

I agree this would be an improvement!
I also did not know about playwright!
Glad there are alternatives to the whole selenium route...

[drammock]

nowadays we use Playwright in our a11y tests, and we've discussed (here and following comments) expanding playwright to testing other aspects of the site too. So I think this can be closed.