ryanbuening.com | End-to-End (E2E) testing with Playwright & Azure DevOps

📅 April 1, 2022

End-to-End (E2E) testing with Playwright & Azure DevOps

Through email, Reddit threads, and hacker news articles, I kept seeing Playwright being mentioned as an improved option for end-to-end testing. I had it in my backlog as something to try out. This post is the result of that experiment.

The tech

Install Microsoft.Playwright

Create an E2E test project. I used dotnet new xunit.
Add Microsoft.Playwright as a package reference to the newly created project/.csproj file.
```
 <PackageReference Include="Microsoft.Playwright" Version="1.19.0" />
```

Verify things are working with `playwright codegen`

From a command line at the root of your newly created project, you should be able to run playwright codegen http://www.google.com which will open a browser and the Playwright Inspector. As you interact with the page, codegen will automatically generate the respective Playwright logic. More details here.

Create a simple test

PlaywrightTest.cs

namespace Playwright;

using Microsoft.Playwright;
using System;
using System.Threading.Tasks;
using Xunit;

public class PlaywrightTest
{
	public PlaywrightTest()
	{
		var exitCode = Microsoft.Playwright.Program.Main(new[] { "install" });
		if (exitCode != 0)
		{
			Console.WriteLine("Failed to install browsers");
			Environment.Exit(exitCode);
		}

		Console.WriteLine("Browsers installed");
	}

	[Fact]
	public async Task PlaywrightDocsLoadSuccessfully()
	{
		using var playwright = await Playwright.CreateAsync();
		var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
		{
			Headless = false,
		});

		var page = await browser.NewPageAsync();

		await page.RunAndWaitForNavigationAsync(async () =>
		{
			await page.GotoAsync("https://playwright.dev/docs/next/intro");
		});

		await Task.Delay(3000);
	}
}

Run the test on local

You should now be able to run the test. If you have Headless set to false, you should see a browser launch and navigate to the Playwright docs.

Playwright Test

Configure the build pipeline

There are build steps needed to be able to run a Playwright test in a CI/CD pipeline with Azure DevOps. The scenario I will be showing involves a build using Pipelines (with YAML) and a release using a Release Pipeline (GUI).

The YAML task below looks for Playwright.csproj and runs dotnet publish.

- task: DotNetCoreCLI@2
	displayName: dotnet publish - E2E tests
	inputs:
	command: publish
	publishWebProjects: false
	projects: 'Playwright.csproj'
	arguments: '--output $(Build.ArtifactStagingDirectory)'
	zipAfterPublish: false

We then want to publish the files so that we can run our Playwright tests against the outputted dll in our Release pipeline later.

- publish: $(Build.ArtifactStagingDirectory)
	displayName: Publish artifact to Azure
	artifact: Playwright

Configure the release pipeline

In the release pipeline, we need to add and configure the jobs to run our Playwright tests. The jobs I am using are Visual Studio test platform installer and Visual Studio Test.