Testing UI Flow Performance with Playwright, Artillery and Application Insights

We quickly rolled out a hotfix and got the app back up to speed. But we knew we needed to get smarter and avoid this scenario in the future. So, we asked ourselves:

"How can we find performance issues like this before they hit production?"

Our solution was to combine Playwright, Artillery, and Application Insights:

• Playwright: A reliable tool for automating browser interactions, letting us mimic real user behavior and UI flows.
• Artillery: Perfect for load-testing, letting us simulate many virtual users interacting with the app simultaneously, with realistic timing using duration and arrivalCount.
• Application Insights Performance Tab: Provides detailed metrics, such as response times and API dependencies, immediately highlighting performance bottlenecks and unnecessary API calls.

Previously, we performed individual API-level performance tests separately from UI flows. However, this strategy evolved by integrating performance testing directly within realistic UI scenarios.

We defined several user flows based on common usage patterns and combined them into a larger UI scenario. Then, we ran this scenario in parallel with multiple virtual users. To closely mimic real-world scenarios, the flows between users aren’t synchronized; each user can be at a different step at any given time (for example, one might be at step 1 while another is already at step 3).

Despite Artillery's resource demands, it enabled us to run comprehensive simulations with 100 concurrent users over 30 minutes, closely replicating actual usage conditions.

Once the tests are complete, we look into Application Insights to analyze the impact. We compare metrics such as request counts and durations across releases to understand how each deployment impacts performance. For example, spikes in request volume or increased response times can help us quickly spot issues.

Now, every release goes through this performance-checking process. We've effectively turned a stressful production issue into a proactive QA practice.

Here is an example of a small Playwright and Artillery project. Will create two simple commands: one that navigates to the Playwright official website and one that navigates to the Artillery website, and run these into a single user flow.

Quick guide:

• Install Node.js.
• Install Playwright (in your project’s root folder (e.g., Artillery_Playwright), open a terminal in Visual Studio Code, and run: npm init playwright@latest
• Install Artillery (in the same terminal, run command: npm install -g artillery)
• Create scripts to simulate user interactions in the UI using Playwright.
• GoToArtilleryPage.js

Defines a Playwright-based script that navigates through the Artillery website. It simulates a user browsing the Docs section and accessing the “Install Artillery” guide.

• GoToPlaywrightPage.js

Contains a Playwright flow for visiting Playwright’s official website, clicking the “Get Started” and “Writing tests” sections — simulating a realistic user flow.

• LoadRun.js

Aggregates both flows (GoToPlaywrightPage and GoToArtilleryPage) into a single test function called artilleryScript. This function is used by Artillery as a processor to simulate realistic UI load scenarios.

• load-test.yml

This YAML file defines the performance test configuration for Artillery.

It specifies:

• test duration - total time (in seconds) during which users are launched
• user arrival count - total number of virtual users to inject over that time (ex., If arrivalCount: 10 and duration: 20, then 1 new user starts every 2 seconds).
• The custom script (LoadRun.js) to be executed using the Playwright engine

Run the test from the terminal in Visual Studio Code using the command: artillery run load-test.yml

GoToArtilleryPage.js 
async function GoToArtilleryPage(page) {
  await page.goto('https://www.artillery.io/');
  await page.waitForTimeout(2000);// simulate real user pause between actions
  await page.click('a:has-text("Docs")');
  await page.waitForTimeout(2000);// simulate real user pause between actions
  await page.click('span:has-text("Install Artillery")');
  await page.waitForTimeout(2000);// simulate real user pause between actions
}
module.exports = {GoToArtilleryPage};

GoToPlaywrightPage.js
async function GoToPlaywrightPage(page) {
  await page.goto('https://playwright.dev/');
  await page.waitForTimeout(2000);// simulate real user pause between actions
  await page.click('[class*="getStarted"]');
  await page.waitForTimeout(2000);// simulate real user pause between actions
  await page.click('a:has-text("Writing tests")');
  await page.waitForTimeout(2000);// simulate real user pause between actions
}
module.exports = {GoToPlaywrightPage};

LoadRun.js
const { GoToArtilleryPage } = require('./GoToArtilleryPage.js');
const { GoToPlaywrightPage } = require('./GoToPlaywrightPage.js');

async function artilleryScript(page) {
 await GoToPlaywrightPage(page);
 await GoToArtilleryPage(page);
}
module.exports = {artilleryScript};

load-test.yml
config:
  target: https://app.mysite.com  # not used in this script, but required by Artillery
  engines:
      playwright:
        launchOptions:
          headless: false
  processor: "./tests/LoadRun.js"
  phases:       
    - name: Phase1
      duration: 10
      arrivalCount: 4
      
scenarios:
  - engine: playwright
    testFunction: "artilleryScript"

In the example above, we simulate user flows across the Playwright and Artillery websites. These sites are used purely for demonstration purposes and are not connected to any Azure Application Insights instance.

However, in a real-world scenario where you have your own application instrumented with Application Insights, this setup becomes extremely powerful. After running the same test flow against your app, you can navigate to the Performance tab in Azure, set the timestamp based on the test run, and analyze request volumes, response durations, and identify any degradation compared to previous releases.

Conclusion:

What started as an unexpected production issue turned into a long-term improvement in our test strategy. Instead of relying only on functional tests, we now validate performance under more realistic user behavior using UI-based load tests.

With Playwright simulating user flows, Artillery adding load and concurrency, and Application Insights giving deep visibility into performance regressions, we’ve built a feedback loop that catches problems before our users do.

Each new release now includes performance benchmarking as part of the pipeline. By comparing key metrics like response times and request counts, we ensure we’re not just adding features but also maintaining speed and reliability.

References:

• Playwright Documentation
• Artillery Documentation
• Microsoft Application Insights