Resizing animated GIFs causes quality loss after 3.0.0

[sschim]

Prerequisites

• I have written a descriptive issue title
• I have verified that I am running the latest version of ImageSharp
• I have verified if the problem exist in both DEBUG and RELEASE mode
• I have searched open and closed issues to ensure it has not already been reported

ImageSharp version

3.0.1

Other ImageSharp packages and versions

None

Environment (Operating system, version and so on)

Windows 11

.NET Framework version

7

Description

Using ImageSharp to resize animated GIFs produces different results on 2.1.4 versus 3.0.0 and 3.0.1.
The output size is much smaller on 3.x, but the quality is a lot worse.

Steps to Reproduce

This is the sample code I've used to generate the samples:

var image = Image.Load("input.gif");

image.Mutate(x =>
{
    x.Resize(250, 0);
});

image.Save("resized.gif");

Images

Original:

Resized using 2.1.4:

Resized using 3.0.1:

[JimBobSquarePants]

A quick decode test shows that the issue is in the decoder. Will have to have a proper dig during daylight hours.

[JimBobSquarePants]

Actually....

The decoder is fine and working correctly. I'll try to describe exactly what is happening.

This gif is comprised of frames that only contain the difference in pixels between the current and previous frame.

Here's frames 2, 3, and 4.

This is an optimization used by some encoders to reduce the size of the output file.

There's two issues.

First noise is introduced during resizing. This is because the resampler creates a new pixel from the weighted average of surrounding pixels. When you have large differences in the surrounding pixels then that difference between frames becomes obvious. When we turn off dithering and resize using a Triangle resampler (smaller sample radius than the default Bicubic so less noise) we see this output.

There's noise there and it's wholly expected due to how resampling works.

Secondly the default gif encoder quantizes the output applying a dithering algorithm (Floyd-Steinburg by default) to reduce banding. Since that pushes introduced rounding error down and right as it runs through the pixels, we see further distortion as in your sample. (For animation it's best to use ordered or Bayer dithering rather than error diffusion to reduce pixel dancing).

Now.... All this was invisible in V2 because it was masked by a bug which caused us to ignore the disposal method in animated gif and load each frame using the previous frame as a background causing file sizes of previously optimized gifs to explode.

So...

The default behavior works as it should because the pipeline is built for all image formats.

I'm not sure what the best plan would be now. Maybe it would be best to always merge by default on decode but allow optimization by diffing on encode.

Calling the Diamond Dogs.

@saucecontrol @dlemstra

[saucecontrol]

Maybe it would be best to always merge by default on decode but allow optimization by diffing on encode.

Yep, processing 'difference-only' frames independently will only work if you're not dependent on the missing pixels. Any filter that blends neighboring pixels should be run against the merged canvas.

I haven't seen too many cases where the crawling noise from error-diffusion dithering is problematic, but you're right on with the analysis on that as well.

I've got a couple of filters built to handle differential encoding for animations as well as noise reduction (either noise from dithering or film grain or the like from the source), which you can find here. Per usual, they're built more for speed than for readability, but feel free to use them as reference.

The dedupe filter is the most important, as it finds bounds for the difference frame while clearing duplicated pixels within those bounds to bg or transparent. The denoise filter identifies variation between current frame and the ones preceding and following it to smooth out or eliminate temporal noise (this one may have limited benefit depending on the source).

Here's the result of those filters on the sample above, which looks correct while still keeping the file size reasonable:

[JimBobSquarePants]

Thanks for the sense check here @saucecontrol I have the beginnings of a cunning plan.

I'll decode using the full details per frame as in V2.

During encode, when using a global palette, I can work from last to first frame and do a dedupe between the indexed frames before sending for encoding which allows me to preserve all quantization and dithering output. local palettes are more difficult since the indexes will be different so I might leave that for now.

[saucecontrol]

Oh yeah, that could work out very nicely. Looking forward to seeing that!

For local palette, just setting the dupe pixels to transparent tends to be good because your palette selection is then looking only at colors unique to the frame, reducing the need for dithering in the first place. Though I've seen a few cases where compression ends up worse than just keeping the dupe values (I see this with greyscale images sometimes)

[JimBobSquarePants]

This is actually a little trickier than I thought since I wrote the encoder to be super lean and no buffer are preserved longer than required.

To do the optimization I'm having to allocate 2 byte buffers equivalent to a single frame which is a little annoying I can do it with a single buffer.

That said the output size difference is excellent. Here's the output of some early experiments. All using a global palette.

No dither - 322KB

FloydSteinberg Dither at 75% - 640KB

FloydSteinberg Dither at 100% - 1267KB

Bayer 16x16 at 100% - 364KB
I'd only ever use this for color images if you want the retro look.

The v2 output is 2290KB so there are massive savings to be found all around.

[JimBobSquarePants]

BTW @saucecontrol your output quality is gorgeous!

I could improve mine a little by increasing the memeory available to my color distance lookup I use during dithering, but I think you'd still notice the difference due to your awesome resize/sharpening approach.

Here's the best I can do with the current map memory constraints and linear resizing.
FloydSteinberg Dither at 100% - 1241KB

Here's what I get if I bump up the accuracy of my lookups.

FloydSteinberg Dither at 100% - 1217KB

FloydSteinberg Dither at 75% - 641KB

[devedse]

I'm currently running an issue that might be related. I've got 2 gifs where one is losslessly compressed. The first gif loads fine but the losslessly compressed one doesn't. It did load fine in V2.1.3 but doesn't anymore in 3.0.1:

Gif 1:

Gif 2:

If you want I'll make a separate issue, but I think the root cause might be related.

Here's a link to my failing unit test:
https://github.com/devedse/DeveImageOptimizer/blob/05557b0dec85174a0f512e25a1b548c66a0cc4be/DeveImageOptimizer.Tests/ImageOperations/ImagePixelComparerFacts.cs#L115

It tries to compare both images frame by frame but the second image frames just load as "transparent" frames.

Edit:
Here's an example of the first frame from both images as exported by ImageSharp:
1:

2:

[JimBobSquarePants]

Thanks @devedse I'll test it against #2455 asap.