[Feature]: Allow receiving screencast frame timestamps without JPEG data

[skyaara]

Feature request

page.screencast.start() can save a WebM recording and provide frame callbacks:

await page.screencast.start({
  path: 'video.webm',
  onFrame: ({ data, timestamp, viewportWidth, viewportHeight }) => {
    // ...
  },
})

Some consumers need only the frame timing metadata, particularly the timestamp of the first presented frame. They do not need the JPEG data.

A general use case is synchronizing a recorded browser video with an external timeline, such as test events, telemetry, captions, narration, annotations, or recordings from several pages.

Currently, using onFrame for synchronization also delivers the JPEG buffer for every frame. Ignoring data prevents application-level storage and decoding, but the buffer still travels through the callback path.

Would you consider supporting a metadata-only callback or an option that omits frame data?

For example:

await page.screencast.start({
  path: 'video.webm',
  includeFrameData: false,
  onFrame: ({ timestamp, viewportWidth, viewportHeight }) => {
    // Align the recorded video with an external timeline.
  },
})

Another possible shape:

await page.screencast.start({
  path: 'video.webm',
  onFrameMetadata: ({ timestamp, viewportWidth, viewportHeight }) => {
    // ...
  },
})

A callback for the first recorded frame timestamp would also cover the simplest synchronization case.

I would be happy to work on an implementation if the maintainers think this API belongs in Playwright.

[dcrousso]

can you please elaborate more on why you need the frame timestamp?

would a single timestamp when the video starts also suffice?

[skyaara]

Devin Rousso (@dcrousso) Yes, a single timestamp would be sufficient for our use case, provided it represents the first frame actually included in the recorded video.

We need to align the saved WebM with a separate timeline containing browser actions, narration, captions, and annotations. The time when screencast.start() is called or returns is not a reliable anchor because the browser may produce the first recorded frame later.

Our current workaround is:

let firstFrameTimestamp: number | undefined;

await page.screencast.start({
  path: 'video.webm',
  onFrame: ({ timestamp }) => {
    firstFrameTimestamp ??= timestamp;
  },
});

Enabling onFrame transfers the JPEG buffer for every frame, even though our callback does not use or store the image data.

Playwright still needs those frames internally to encode the WebM. The unnecessary part for our use case is transferring every JPEG buffer through the public callback when we only need one timestamp.

A focused API could look like:

let firstFrameTimestamp: number | undefined;

await page.screencast.start({
  path: 'video.webm',
  onFirstFrame: ({ timestamp }) => {
    firstFrameTimestamp = timestamp;
  },
});

onFirstFrame would fire exactly once with the timestamp of the first frame included in the WebM. It would not send the JPEG buffer to the consumer.

I would be happy to prepare a PR once we agree on the timestamp semantics and preferred API shape.

[dgozman]

We can probably include the creation_time metadata into the video file.

[skyaara]

Dmitry Gozman (@dgozman) That makes perfect sense.

After looking through the recorder path, I think this we can make this a focused change. videoRecorder already detects the first frame, so we can write the Matroska header at that point with DateUTC from Playwright’s server wall clock. We can then use -map_metadata 0 so FFmpeg preserves it as creation_time in the final WebM. The existing frame timestamp remains responsible for relative video timing.

We could just read format.tags.creation_time from the final WebM without subscribing to onFrame just to obtain the start time.

I tested this approach locally across Chromium, Firefox, and WebKit, and the full screencast test suite passes.

I’d love to contribute and send a focused PR for this change.

[skyaara]

I’ve looked through the PRs and discussion around this. I think creation_time should use the timestamp associated with the first JPEG frame, normalized to Unix wall time, rather than the recorder initialization time.

I’ve put together a minimal implementation in my fork for review(adhering to the contributing guidelines) : skyaara/playwright#5.