Skip to content

Software path: publish the display size the renderer settled on, so an overlay lays out against the picture and not the coded frame #353

Description

@superuser404notfound

Raised by @edde746 while adopting #311 (comment), filed here so it is tracked separately from the frame-time work.

What should the engine do?

Publish the display size the software path settled on: the coded frame with the sample aspect ratio the renderer actually attached. Nothing on the public surface carries it today, and no host can derive it.

Motivating media or use case

Laying out a frame-exact overlay (libass, ASS/SSA) over software-decoded video. #311 gave the overlay its clock and its frame boundaries; what is still missing is the rectangle to draw into.

A host derives that rectangle from a picture aspect under the active gravity. The only public size is sourceVideoWidth/sourceVideoHeight, which are coded dimensions, so anamorphic content lays out against a rect off by the pixel aspect: 720x576 with a 64:45 SAR presents as 1024x576, and an overlay sized 5:4 sits inside a 16:9 picture.

AVSampleBufferDisplayLayer offers no videoRect, unlike AVPlayerLayer, so there is nothing to measure on the layer either.

Why the host cannot compute it

The SAR that reaches the screen is decided per frame inside SoftwareVideoDecoder.attachPixelAspectRatio, and neither input nor result is reachable:

SourceProbe carries videoWidth/videoHeight and no SAR, so a host reconstructing the ratio from container metadata disagrees with the picture in exactly the cases the policy exists for: for a rejected SAR the engine shows square pixels while the container still declares the ratio it rejected.

Shape

Read it from the format description the renderer enqueues rather than recomputing the policy:

CMVideoFormatDescriptionGetPresentationDimensions(desc, usePixelAspectRatio: true, useCleanAperture: true)

That description is the object on screen, so the published value cannot drift from the picture, and its cache key already invalidates on a dimension or PAR change (#177), which is exactly when the size has to be republished.

Two constraints:

Not a field on SoftwareVideoFrameTime: it changes once per format, and that callback runs on the decode thread.

Scope

Software path only. The native and bypass paths mount an AVPlayerLayer, which measures its own videoRect and carries AVPlayerItem.presentationSize.

Area

Public API surface

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions