[Bug]: Snapshots, events and screencast missing from trace (webview2 app) about playwright

[dgozman]

@asinanovic Call test.info().attach() to attach the trace file, and it should get into the report.

[yury-s]

Tracing should work with webview2, can you share a repro where it fails?

[asinanovic]

I don't get any error message, it's just that snapshots and screencast are missing. I can only see a list of actions.

Below is my webView2Test.ts implementation:

import { test as base } from '@playwright/test';
import fs from 'fs';
import os from 'os';
import path from 'path';
import childProcess from 'child_process';

const EXECUTABLE_PATH = path.join(
    __dirname,
    '../../../../pathToMyWebView2.exe',
);

export const test = base.extend({
  context: async ({ playwright }, use, testInfo) => {
    const cdpPort = 10000 + testInfo.workerIndex;
    // Make sure that the executable exists and is executable
    fs.accessSync(EXECUTABLE_PATH, fs.constants.X_OK);
    const userDataDir = path.join(
        fs.realpathSync.native(os.tmpdir()),
        `playwright-webview2-tests/user-data-dir-${testInfo.workerIndex}`,
    );
    const webView2Process = childProcess.spawn(EXECUTABLE_PATH, [], {
      shell: true,
      env: {
        ...process.env,
        WEBVIEW2_ADDITIONAL_BROWSER_ARGUMENTS: `--remote-debugging-port=${cdpPort}`,
        WEBVIEW2_USER_DATA_FOLDER: userDataDir,
      }
    });
    await new Promise<void>(resolve => webView2Process.stdout.on('data', data => {
      if (data.toString().includes('WebView2 initialized'))
        resolve();
    }));
    const browser = await playwright.chromium.connectOverCDP(`http://127.0.0.1:${cdpPort}`);
    await use(browser.contexts()[0]);
    await browser.close();
    childProcess.execSync(`taskkill /pid ${webView2Process.pid} /T /F`);
    fs.rmdirSync(userDataDir, { recursive: true });
  },
  page: async ({ context }, use) => {
    const page = context.pages()[0];
    await use(page);
  },
});

export { expect } from '@playwright/test';

... and below is playwright configuration:

import { defineConfig, devices } from '@playwright/test';

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
require('dotenv').config();

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: './tests',
  /* Timeout for each test in milliseconds */
  timeout: 2 * 60 * 1000,
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://127.0.0.1:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    // trace: 'on-first-retry'
    trace: 'on'
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    }
  ],
});

[yury-s]

You can try manually record trace by calling start and stop and see if the traces contain the snapshots. We were not able to reproduce it with provided code. Could you share full repro including the app at pathToMyWebView2, so that we could run it locally?

[asinanovic]

I already tried to record manually, but I only called tracing.start. 🙃 Now, after calling both, tracing.start and tracing.stop, it works just fine.

const browser = await playwright.chromium.connectOverCDP(`http://127.0.0.1:${cdpPort}`);
await browser.contexts()[0].tracing.start({ screenshots: true, snapshots: true })
await use(browser.contexts()[0]);
await browser.contexts()[0].tracing.stop({ path: 'trace.zip' });
await browser.close();

Though, I'm still wondering why it doesn't work when I set trace: 'on' in the playwright.config file - any idea?

In addition, now I notice that for the snapshots, css is not loaded. If I inspect snapshots, I can see bunch of 'Failed to load resource (404)' errors while screencast shows images with css, as expected.

[mxschmitt]

Though, I'm still wondering why it doesn't work when I set trace: 'on' in the playwright.config file - any idea?

This is expected as of today, since there is no integration between WebView2/connectOverCDP and the tracing/test-runner options like trace/video/screenshot. Calling tracing.stop is required as of today. This gives you also more control in order to when you want to start a new trace and when not, since based on the scenario you want to re-start the app or not, perform special cleanup logic etc.

In addition, now I notice that for the snapshots, css is not loaded. If I inspect snapshots, I can see bunch of 'Failed to load resource (404)' errors while screencast shows images with css, as expected.

We need a repro in order to look into this.

[asinanovic]

This is expected as of today, since there is no integration between WebView2/connectOverCDP and the tracing/test-runner options like trace/video/screenshot. Calling tracing.stop is required as of today. This gives you also more control in order to when you want to start a new trace and when not, since based on the scenario you want to re-start the app or not, perform special cleanup logic etc.

It would be valuable to mention this somewhere in trace documentation.

Final question - since for WebView2/connectOverCDP apps I have to start/stop the trace manually, is there a way for the trace to be part of the Playwright report? Currently, the trace is saved to the path we specify, but it would be valuable for it to be part of the generated report, so I have everything in one place when calling npx playwright show-report.

[asinanovic]

Thanks!