Skip to content

Make comparsion of images more reliable#33

Merged
judovana merged 1 commit intorh-openjdk:mainfrom
zzambers:img-cmp-fix2
Oct 1, 2025
Merged

Make comparsion of images more reliable#33
judovana merged 1 commit intorh-openjdk:mainfrom
zzambers:img-cmp-fix2

Conversation

@zzambers
Copy link
Collaborator

@zzambers zzambers commented Sep 30, 2025

Current code comparing screenshots is not very reliable and does not work consistently on different OSes:

  • seems like there were some changes in ImageMagic's metrics for compare. Different imagemagick/os versions show different results (see also previous PR). Also output format sometimes differ.
  • generated diff images also look different on different imagemagick/os versions (some are grascale, some use red color for differences)
  • for solid black background (first screenshot) import uses 1-bit depth png, while later one it uses 24-bit depth. When compared by some versions of imagemagick, it then compares only first channel (red) instead of all channels.
  • values describing similarity seem very abstract/artificial, not sure how they corelate to diff image

This tries to fix above issues. I have tried to play with metrics in compare command, but they just don't behave consistently in different versions. So I switched to different approach. Compare uses explicit parameters to generate mask of differences (black and white). Difference as proportion of changed pixels in % is then computed. (this now has direct correlation to what is seen on difference mask and is easier to reason about) Also 24-bit png is explicitly specified for import, to behave more consistently. I have done testing on different rhels and this code seems to behave consistently.

@judovana judovana self-requested a review October 1, 2025 13:24
@judovana judovana merged commit a7f6d8c into rh-openjdk:main Oct 1, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants