📅 April 1, 2022

End-to-End (E2E) testing with Playwright & Azure DevOps

Through email, Reddit threads, and hacker news articles, I kept seeing Playwright being mentioned as an improved option for end-to-end testing. I had it in my backlog as something to try out. This post is the result of that experiment.

The tech

Install Microsoft.Playwright

  1. Create an E2E test project. I used dotnet new xunit.
  2. Add Microsoft.Playwright as a package reference to the newly created project/.csproj file.

    <PackageReference Include="Microsoft.Playwright" Version="1.19.0" />

Verify things are working with playwright codegen

From a command line at the root of your newly created project, you should be able to run playwright codegen http://www.google.com which will open a browser and the Playwright Inspector. As you interact with the page, codegen will automatically generate the respective Playwright logic. More details here.

Create a simple test

PlaywrightTest.cs

namespace Playwright;

using Microsoft.Playwright;
using System;
using System.Threading.Tasks;
using Xunit;

public class PlaywrightTest
{
    public PlaywrightTest()
    {
        var exitCode = Microsoft.Playwright.Program.Main(new[] { "install" });
        if (exitCode != 0)
        {
            Console.WriteLine("Failed to install browsers");
            Environment.Exit(exitCode);
        }

        Console.WriteLine("Browsers installed");
    }

    [Fact]
    public async Task PlaywrightDocsLoadSuccessfully()
    {
        using var playwright = await Playwright.CreateAsync();
        var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
        {
            Headless = false,
        });

        var page = await browser.NewPageAsync();

        await page.RunAndWaitForNavigationAsync(async () =>
        {
            await page.GotoAsync("https://playwright.dev/docs/next/intro");
        });

        await Task.Delay(3000);
    }
}

Run the test on local

You should now be able to run the test. If you have Headless set to false, you should see a browser launch and navigate to the Playwright docs.

Playwright Test

Configure the build pipeline

There are build steps needed to be able to run a Playwright test in a CI/CD pipeline with Azure DevOps. The scenario I will be showing involves a build using Pipelines (with YAML) and a release using a Release Pipeline (GUI).

The YAML task below looks for Playwright.csproj and runs dotnet publish.

- task: DotNetCoreCLI@2
    displayName: dotnet publish - E2E tests
    inputs:
    command: publish
    publishWebProjects: false
    projects: 'Playwright.csproj'
    arguments: '--output $(Build.ArtifactStagingDirectory)'
    zipAfterPublish: false

We then want to publish the files so that we can run our Playwright tests against the outputted dll in our Release pipeline later.

- publish: $(Build.ArtifactStagingDirectory)
    displayName: Publish artifact to Azure
    artifact: Playwright

Configure the release pipeline

In the release pipeline, we need to add and configure the jobs to run our Playwright tests. The jobs I am using are Visual Studio test platform installer and Visual Studio Test.

Visual Studio test platform installer job

Visual Studio test platform installer job

NOTE: I'm specifying a Test Platform Version of 16.11.0 because of this issue.

Visual Studio Test job

Visual Studio Test job

We can now run the release and hopefully see some green checkmarks!

E2E result

# development | dotnet | devops