Better Simple Logo

Playwright & pytest techniques that bring me joy

I’ve been working with playwright more often to do end to end tests. As a project grows to do more with HTMX and Alpine in the markup, there’s less unit and integration test coverage and a greater need for end to end tests. Here are some techniques I’ve used that bring me joy.

Open new pages / tabs to be tested

# Capture the new tab/window in a variable
with page.expect_page() as popup_page_info:
    page.get_by_role("button", name="Some trigger").click()
    
# Rename the actual page to something usable.
popup_page = popup_page_info.value

# We can now run test against the popup page
expect(popup_page.get_by_role("button", name="Action on popup page")).to_be_visible()

Playwright supports opening a window or a tab and then running tests against it. The reference docs about expect_page() don’t show the full utility of it, but once you understand the above you’re good to go!

I’m sure I’m missing some functionality in regards to the arguments of expect_page(), but it’s pretty powerful for me without understanding that.

Using a pytest marker to identify playwright tests

I like to have a marker decorator to apply to my playwright tests. This makes it easier to choose when to run playwright tests versus when not to. This involves adding to the pytest configuration area (pytest.ini, setup.cfg, pyproject.toml or similar)

# content of pytest.ini
[pytest]
markers =  
  playwright: mark test that requires playwright

With this, you can run your tests playwright selectively:

pytest -m playwright

I like to combine this with changing my the tests to always exclude the playwright tests. This does mean CI will need to explicitly run pytest -m playwright, but that’s a reasonable tradeoff to me.

# content of pytest.ini
[pytest] 
addopts = -m 'not playwright'

Using pytest markers in place of fixtures

This may be specific to me, so take this section with a grain of salt. The project I’m working on has two different types of access. If it were a CMS, there’d be the creator and the viewer. They are similar, but definitely different.

I could have certainly used a fixture to adjust the context of the test, but I wanted to call it out more specifically on the test. See the following code example:

@pytest.mark.use_viewer_context
def test_can_toggle_between_sections(page, several, other, fixtures):
    pass


def test_can_toggle_between_sections(viewer_page, several, other, fixtures):
    pass

To implement this, I needed to make use of pytest’s markers.

@pytest
def page(request, new_context, live_server):
	# The live_server is Django specific
    context = new_context.new_context(base_url=live_server.url)
    page = context.new_page()
    if request.node.get_closest_marker("use_viewer_context"):
        page.goto("/login/viewer/")
        # Login as viewer
	else:
		page.goto("/login/creator/")
		# Login as creator
	
    yield page
    # Clean up the context and page
    page.close()
    context.close()

I also needed to add a marker to my pytest configuration area (pytest.ini, setup.cfg, pyproject.toml or similar)

# content of pytest.ini
[pytest]
markers =  
  use_course_launch: mark playwright tests that should use a course launch

Using page.pause() and Playwright’s debugging tool

Playwright provides powerful debugging tools that’s accessible in a few ways. I love this for the inspector tool specifically because it helps me craft the selectors that I can never remember.

To start, you’ll want to drop a page.pause() in your testing code as a breakpoint.

def test_action(page):
    page.pause()
    expect(page.get_by_role("button", name="Submit")).to_be_visible()

To run your tests, you can either run the tests in headed mode:

pytest --headed

Or run the tests with the playwright debug mode enabled. I believe this is just headed with no default timeout for selects.

PWDEBUG=1 pytest

Personally, I prefer --headed since it’s easier to append to a command than prepending something.

Using assert_axe_violations to prevent accessibility regressions

I like Pamela Fox’s axe-playwright-python package for providing a Python wrapper around axe-core. It provides a nice hook to do assertions, the only thing that’s missing is a pytest assertion fixture to make everything a bit more re-usable. Here’s what I have started with:

import pytest  
from axe_playwright_python.sync_playwright import Axe  
from playwright.sync_api import Page

@pytest.fixture  
def assert_axe_violations(pytestconfig):  
    """  
    pytest assertion fixture for validating that a page has no violations.
    """ 
    axe = Axe()  
  
    def _assert(page: Page, context: None | str | list[str] = None):  
        results = axe.run(page, context=context)  
        if results.violations_count != 0:  
            if pytestconfig.getoption("verbose") > 0:  
                message = results.generate_report()  
            else:  
                count = (  
                    "1 violation"  
                    if results.violations_count == 1  
                    else f"{results.violations_count} violations"
                )  
                message = f"There were {count} found.\n\n{results.generate_snapshot()}"  
            pytest.fail(message)  
  
    return _assert

Then using it becomes:

def test_page_accessibility(page):
    """Test the whole page for accessibility concerns."""
    assert_axe_violations(page)
    
def test_section_accessibility(page):
    """Test specific sections for accessibility concerns."""
    assert_axe_violations(page, context="section.sections-to-check")

Using page.expect_response() to confirm a background request occurred

There are times when a user interaction causes a request to the server, but the user interface won’t change. This makes it difficult to determine if that request has occurred. Thankfully, expect_response.

def test_request_occurs(page):
    with page.expect_response("**/my-particular-url/"):
        page.get_by_role("button", name="Submit").click()

There’s more you can do with this too. You could inspect the response to confirm it was actually a 200 status response.

It can also serve as another tool when trying to wait for an action to occur, though use this with caution. Just because a request occurred, doesn’t mean the frontend of the application updated so the elements in the DOM may not be what you expect. This is no different than the other difficulties with making assertions about the DOM in an interactive web app.

If you have thoughts, comments or questions, please let me know. You can find me on the Fediverse, Django Discord server or via email.