Skip to content

video: merge the tiles of grid heif and avif images#17738

Open
guidocella wants to merge 1 commit intompv-player:masterfrom
guidocella:tile
Open

video: merge the tiles of grid heif and avif images#17738
guidocella wants to merge 1 commit intompv-player:masterfrom
guidocella:tile

Conversation

@guidocella
Copy link
Copy Markdown
Contributor

@guidocella guidocella commented Apr 11, 2026

Fixes #13585.
Fixes #16486.

Switching between different grid images added with --external-files is also supported.

The prority of independent tracks is reverted for images, because we now want to select a dependent track to trigger the merging, rather than small preview tracks.

Mostly generated by Claude.

--hwdec=vaapi works for regular heif images, but breaks grid ones. nvdec works. I get CUDA_ERROR_OUT_OF_MEMORY but at least it fallsback to software decoding.
I can't test av1, but hardware decoding already doesn't work at least for HDR avif according to #14062.

return 0;
}

static char *tile_grid_graph(void *ctx, const struct mp_tile_grid *grid)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these things belong to f_auto_filters.c, not here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added an auto filter at first but it doesn't seem to be able to handle different streams.

@Andarwinux
Copy link
Copy Markdown
Contributor

I tested this PR. If the preferred hwdec is not a copy variant (--hwdec=d3d11va,d3d11va-copy), grid images fail to load entirely, and it is unable to failback to a copy hwdec and reload.

I can't test av1, but hardware decoding already doesn't work at least for HDR avif according to #14062.

MPV-00_00_00 000-N0001

I suspect the problem in #14062 is that the pixel format isn't yuv420, and currently there are no GPUs capable of yuv422/444 av1.

@guidocella
Copy link
Copy Markdown
Contributor Author

vaapi-copy also works.

@kasper93
Copy link
Copy Markdown
Member

kasper93 commented Apr 12, 2026

This should be merged after scaling individual tiles, preferably in libplacebo. Otherwise big images won't work.

EDIT: But after discussion, we don't care about big images > 16k, so it's not needed.

@guidocella
Copy link
Copy Markdown
Contributor Author

guidocella commented Apr 12, 2026

Is that related to tile grids? Non-grid large images don't work all the same. Also changing libplacebo won't work for dmabuf-wayland.

@llyyr
Copy link
Copy Markdown
Contributor

llyyr commented Apr 12, 2026

Also changing libplacebo won't work for dmabuf-wayland.

Nothing will, you need to make compositor support rendering larger textures or split textures in dmabuf-wayland and make compositor stitch it

{
struct bstr buf = {0};

for (int i = 0; i < grid->nb_tiles; i++)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the best layer to do the stitching. Also at this point we probably have information about the source and destination rectangle. Or at least source one based on crop, so we have decode only tiles that contribute to the final image.

Most of those pan-scan and video-zoom props are VO related, so it's interesting task to figure out how to do software pre-stiching to include this information.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs before the window is even created.

And is there a risk of dynamically updating the filter being slower? If I run set lavfi-complex [vid1][vid2]hstack[vo] at runtime, it is far from instant.

Fixes mpv-player#13585.
Fixes mpv-player#16486.

Switching between different grid images added with --external-files is
also supported.

The prority of independent tracks is reverted for images, because we now
want to select a dependent track to trigger the merging, rather than
small preview tracks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AVIF Grid images are not decoded correctly mpv can't properly parse tile HEIF still images

5 participants