Test Frameworks

Four packages give tests base classes that manage browsers for them, one for each of xUnit, NUnit, MSTest, and TUnit:

One browser launch per test run, shared by every test class, unless a class configures a launch of its own.
An isolated browser and page for each test, closed when the test ends, so that no cookie or storage carries over from another test.
Screenshots of a failed test's pages, and videos and a trace if asked for, saved to files and attached to the test's result.
The browser chosen by environment variables, so that a CI matrix runs the same tests in each browser, or in code.

Package	Framework
`Dramaturge.Xunit`	xUnit v3, 3.2.2 and later
`Dramaturge.NUnit`	NUnit 4 and later
`Dramaturge.MSTest`	MSTest 4 and later
`Dramaturge.TUnit`	TUnit 1.6 and later

To start a test by recording it, dramaturge codegen writes a PageTest class for any of the four; see Code Generation.

The packages are for .NET 10. Each has the same two base classes, with the same members:

PageTest gives each test a Browser and a Page in it. Most tests use it.
BrowserTest gives each test the Group, and NewBrowserAsync to open browsers, for tests that need several, such as two users of a chat.

Registering the Shared Browser

Each test project registers the browser its test classes share, once. The registration closes the browser when the run ends; without it, the first test that needs the browser fails, with a message giving the code to add.

dotnet add package Dramaturge.Xunit

[assembly: AssemblyFixture(typeof(Dramaturge.Xunit.DramaturgeAssemblyFixture))]

dotnet add package Dramaturge.NUnit

A set-up fixture applies to the tests of its namespace and the namespaces within it, so put it outside any namespace, or in the namespace that holds all the tests:

[SetUpFixture]
public class DramaturgeSetUp : DramaturgeSetUpFixture
{
}

dotnet add package Dramaturge.MSTest

MSTest runs assembly hooks only from the test project's own classes, so the registration is a class of two hooks:

[TestClass]
public static class DramaturgeSetUp
{
    [AssemblyInitialize]
    public static void Initialize(TestContext context) => DramaturgeAssemblyFixture.Register(new());

    [AssemblyCleanup]
    public static Task CleanupAsync() => DramaturgeAssemblyFixture.CloseAsync();
}

dotnet add package Dramaturge.TUnit

TUnit runs hooks only from the assembly that declares them, so the registration is a class of two hooks:

public static class DramaturgeSetUp
{
    [Before(HookType.TestSession)]
    public static void Register() => DramaturgeAssemblyFixture.Register(new());

    [After(HookType.TestSession)]
    public static Task CloseAsync() => DramaturgeAssemblyFixture.CloseAsync();
}

Writing Tests

Derive the test class from PageTest, and use Page. Expect is a static method of Assertions, brought in with using static Dramaturge.Assertions;.

public class SignInTests : PageTest
{
    [Fact]
    public async Task SignsIn()
    {
        await this.Page.NavigateAsync("https://example.com/sign-in");
        await this.Page.GetByLabel("Email").FillAsync("someone@example.com");
        await this.Page.GetByRole("button", "Sign in").ClickAsync();
        await Expect(this.Page.GetByRole("heading", "Welcome")).ToBeVisibleAsync();
    }
}

public class SignInTests : PageTest
{
    [Test]
    public async Task SignsIn()
    {
        await this.Page.NavigateAsync("https://example.com/sign-in");
        await this.Page.GetByLabel("Email").FillAsync("someone@example.com");
        await this.Page.GetByRole("button", "Sign in").ClickAsync();
        await Expect(this.Page.GetByRole("heading", "Welcome")).ToBeVisibleAsync();
    }
}

[TestClass]
public class SignInTests : PageTest
{
    [TestMethod]
    public async Task SignsIn()
    {
        await this.Page.NavigateAsync("https://example.com/sign-in");
        await this.Page.GetByLabel("Email").FillAsync("someone@example.com");
        await this.Page.GetByRole("button", "Sign in").ClickAsync();
        await Expect(this.Page.GetByRole("heading", "Welcome")).ToBeVisibleAsync();
    }
}

public class SignInTests : PageTest
{
    [Test]
    public async Task SignsIn()
    {
        await this.Page.NavigateAsync("https://example.com/sign-in");
        await this.Page.GetByLabel("Email").FillAsync("someone@example.com");
        await this.Page.GetByRole("button", "Sign in").ClickAsync();
        await Expect(this.Page.GetByRole("heading", "Welcome")).ToBeVisibleAsync();
    }
}

A test that needs several isolated browsers derives from BrowserTest, and opens each with NewBrowserAsync. The examples in the rest of this article use xUnit; the members are the same in every package.

public class ChatTests : BrowserTest
{
    [Fact]
    public async Task MessageReachesTheOtherUser()
    {
        // Each browser has its own cookies and storage, so each signs in as a different user.
        Page alice = await (await this.NewBrowserAsync()).NewPageAsync();
        Page bob = await (await this.NewBrowserAsync()).NewPageAsync();
        await alice.NavigateAsync("https://example.com/chat?user=alice");
        await bob.NavigateAsync("https://example.com/chat?user=bob");

        await alice.GetByLabel("Message").FillAsync("Hello, Bob");
        await alice.GetByRole("button", "Send").ClickAsync();

        await Expect(bob.GetByRole("log")).ToContainTextAsync("Hello, Bob");
    }
}

The browsers a test opens are closed when it ends, whether it passes or fails. Group, Browser, and Page are available from when the test starts, and throw InvalidOperationException if read before.

Choosing the Browser

The shared browser is launched from BrowserLauncher.ConfigureFromEnvironment: Chrome, stable, and headless, unless environment variables say otherwise. Browsers are downloaded on first use, as they are anywhere else, and the CHROME_EXECUTABLE and other executable variables still pick an installed browser, as Browser Setup describes.

Variable	Values	Default
`DRAMATURGE_BROWSER`	`Chrome`, `Firefox`, `Edge`, `Safari`	`Chrome`
`DRAMATURGE_CHANNEL`	`Stable`, `Beta`, `DeveloperPreview`, `Alpha`, `ExtendedSupport`	`Stable`
`DRAMATURGE_HEADED`	`1` or `true` shows the browser, for debugging	Headless

A CI matrix sets DRAMATURGE_BROWSER to run the same tests in each browser:

strategy:
  matrix:
    browser: [chrome, firefox]
steps:
  - run: dotnet test
    env:
      DRAMATURGE_BROWSER: ${{ matrix.browser }}

Configuring in Code

A test class overrides members of its base class:

Member	Sets	For
`BrowserOptions`	The browser options of each browser a test opens without options of its own	Each test
`ConfigureLauncher`	The launcher, given the one the environment chooses; return it changed, or another	A class's own browser launch
`GroupOptions`	The group's options, such as timeouts	A class's own browser launch

public class MobileLayoutTests : PageTest
{
    // Applies to the browser of each test of this class.
    protected override BrowserOptions? BrowserOptions => new() { Viewport = new Viewport() { Width = 390, Height = 844 }, Locale = "en-GB" };

    // Overriding the launcher or the group's options gives the class a browser of its own, launched for its tests.
    protected override BrowserLauncherBuilder ConfigureLauncher(BrowserLauncherBuilder builder)
    {
        return builder.WithLaunchTimeout(TimeSpan.FromMinutes(2));
    }

    [Fact]
    public async Task MenuIsCollapsed()
    {
        await this.Page.NavigateAsync("https://example.com");
        await Expect(this.Page.GetByRole("button", "Menu")).ToBeVisibleAsync();
    }
}

A class that overrides ConfigureLauncher or GroupOptions has a browser launch of its own, shared by its tests and closed when the class finishes, because its tests need a browser launched differently from the others. To configure the shared browser, register a class derived from the shared browser's fixture in place of it, overriding the same members:

// Registered in place of DramaturgeAssemblyFixture:
// [assembly: AssemblyFixture(typeof(ShopFixture))]
public class ShopFixture : DramaturgeAssemblyFixture
{
    protected override DramaturgeOptions? GroupOptions => new() { ActionTimeout = TimeSpan.FromSeconds(10), TestIdAttribute = "data-test" };

    protected override BrowserLauncherBuilder ConfigureLauncher(BrowserLauncherBuilder builder)
    {
        string? grid = Environment.GetEnvironmentVariable("SELENIUM_GRID_URL");
        return grid is null ? builder : BrowserLauncher.Configure(BrowserKind.Chrome).LaunchUsingRemoteGrid(new Uri(grid));
    }
}

With NUnit, derive the set-up fixture from DramaturgeSetUpFixture as the registration does; with MSTest and TUnit, pass an object of the derived class to DramaturgeAssemblyFixture.Register in place of new().

A remote grid or a running browser cannot be launched headless, so a launcher from the environment changed to connect to one needs WithHeadlessOption(false); or start from BrowserLauncher.Configure, as the example does.

Screenshots of Failed Tests

When a test fails, every open page of every browser it opened is captured, before the browsers are closed, to TestResults/Dramaturge in the test assembly's directory: page-1.png, page-2.png, and so on, in a directory named for the test, which replaces any earlier run's. The directory's name keeps the letters, digits, ., -, and _ of the test's name, with others replaced by _, and at most 120 characters of it.

Each screenshot is also attached to the test's result, where the framework's reports and IDE show it:

Framework	Attached with
xUnit	`TestContext.AddAttachment`, as an `image/png` attachment, `video/webm` for a video, or `application/zip` for a trace
NUnit	`TestContext.AddTestAttachment`
MSTest	`TestContext.AddResultFile`
TUnit	`TestContext.Output.AttachArtifact`

A page that cannot be captured, and a browser that cannot be closed, are reported as a warning (xUnit) or in the test's output (the others), and never change the test's result.

ScreenshotOnFailure turns capturing off, and ArtifactsDirectory changes the directory, for a class in its constructor or for one test:

public class ReportTests : PageTest
{
    public ReportTests()
    {
        this.ArtifactsDirectory = Path.Combine(AppContext.BaseDirectory, "screenshots");
    }

    [Fact]
    public async Task PrintsWithoutScreenshots()
    {
        // This test's failures are not captured.
        this.ScreenshotOnFailure = false;
        await this.Page.NavigateAsync("https://example.com/report");
        await Expect(this.Page.GetByRole("button", "Print")).ToBeEnabledAsync();
    }
}

Videos

With VideoOnFailure on, every page a test opens is recorded from when it opens, in every browser the test opens, including popups. When the test fails, each page's video is saved beside the screenshots, as page-1.webm and so on, and attached to the test's result; when it passes, the videos are deleted. VideoOptions sets the videos' size and frame rate. Turn it on before the test opens a page, which for a PageTest means in the constructor:

public class CheckoutTests : PageTest
{
    public CheckoutTests()
    {
        this.VideoOnFailure = true;
        this.VideoOptions = new VideoRecordingOptions() { Width = 1280, Height = 720 };
    }

    [Fact]
    public async Task PlacesAnOrder()
    {
        await this.Page.NavigateAsync("https://example.com/checkout");
        await this.Page.GetByRole("button", "Place order").ClickAsync();
        await Expect(this.Page.GetByRole("heading", "Thank you")).ToBeVisibleAsync();
    }
}

Pages are numbered in the order they opened, across all of a test's browsers, so a page's screenshot and video share a number. A page closed before the test failed has a video but no screenshot, so the screenshots' numbers can skip it: page-1.png and page-3.png, beside page-1.webm, page-2.webm, and page-3.webm.

A browser that cannot record video, such as Chrome, is reported once, and the test runs as usual. A video that cannot be recorded or saved is reported for a failed test, as is one a browser on another machine, such as a remote grid's, wrote there; it is not copied from that machine.

Traces

With TraceOnFailure on, every browser a test opens records a trace from when it opens. When the test fails, the browsers' traces are merged into one, trace.zip, beside the screenshots, and attached to the test's result; each browser is a context of its own in the viewer, on one timeline. When the test passes, the traces are deleted. Turn it on before the test opens a browser, which for a PageTest means in the constructor:

public class CartTests : PageTest
{
    public CartTests()
    {
        this.TraceOnFailure = true;
    }

    [Fact]
    public async Task AddsAnItem()
    {
        await this.Page.NavigateAsync("https://example.com/catalog");
        await this.Page.GetByRole("button", "Add to cart").ClickAsync();
        await Expect(this.Page.GetByTestId("cart-count")).ToHaveTextAsync("1");
    }
}

TraceOptions sets what the traces record; by default they have snapshots, screenshots, and sources. A trace that cannot start is reported for a failed test, and the test runs as usual. Open trace.zip at trace.playwright.dev or with npx playwright show-trace.

Lifecycle and Parallel Tests

Each package hooks into its framework's own lifecycle. A test class can add its own set-up and clean-up as usual:

Framework	Browsers are opened and closed in	Notes
xUnit	`InitializeAsync` and `DisposeAsync` (`IAsyncLifetime`)	An override must call the base method.
NUnit	`SetUpBrowsersAsync` and `TearDownBrowsersAsync` (`[SetUp]`, `[TearDown]`)	An override must call the base method.
MSTest	`SetUpBrowsersAsync`, then `PageTest.OpenPageAsync` (`[TestInitialize]`), and `TearDownBrowsersAsync` (`[TestCleanup]`)	MSTest runs these before the test class's own `[TestInitialize]` methods and after its `[TestCleanup]` methods.
TUnit	`SetUpBrowsersAsync` and `TearDownBrowsersAsync`, on TUnit's test start and end events	An override of `SetUpBrowsersAsync` must call the base method.

Tests can run in parallel, each with its own browser. xUnit, MSTest, and TUnit create a test class object for each test. NUnit creates one for a fixture's tests, and a test's browsers are kept on it, so run the tests of one fixture in parallel only with [FixtureLifeCycle(LifeCycle.InstancePerTestCase)].

Table of Contents