recode large PNG

[r10s]

PNG files may become quite large, eg. when used for screenshots by the system.

therefore, when sent as DC_MSG_IMAGE, we could try to recode these PNG to a smaller bytesize, similar to what we're doing with large JPG (original PNG can still be sent when sending as DC_MSG_FILE).

recoding could be done by:

1. scale down image width/height and/or palette and keep PNG format
2. recode to JPG; JPG of the same width/height usually have much smaller bytesize
3. a combination of 1. and 2.
4. ?

this needs to be tried out. the destination bytes should be comparable with the result of WORSE_IMAGE_SIZE/BALANCED_IMAGE_SIZE, if that is too worse, a factor of that.

this issue is not about introducing new options in the UI, it is just about respecting exiting media-quality setting also for PNG.

nb: for avatars, we are already recoding PNG to JPG (as 2. above), so, the basic functionality is there.

most work will be testing and checking that things work with a changed file name+type.

ftr: this is a succcessor of #1563

[Simon-Laux]

there is also the option for lossless compression/optimization with libs/tools like https://github.com/shssoichiro/oxipng

[r10s]

i played a bit around with a random screenshot with lots of text but also some images, 1125 x 2436 px.

original             4.6 mb
oxipng               3.0 mb
oxipng -Z            2.9 mb (recoding took 5 minutes)
jpg 90%              0.5 mb
jpg 90%, 50% scaled  0.2 mb
jpg 70%              0.3 mb

as oxipng is lossless, there is nothing to compare visually, below the original and the three jpg:

Click me

original:

---

jpg 90%:

---

jpg 90%, 50% scaled:

jpg 70%:

not sure about scaling, just added that by interest. by 1/4th of pixels and potential details, the bytesize is down only by ~ 1/2, probably as by antialiasing things are harder to compress, esp. lines and fonts.

other typical images to try out would be diagrams, a statistics etc.

[Septias]

Do you want to add recoding for all pngs sent? Because I know a lot of people who are annoyed with e.g. whatsapp for downscaling images when sent.
Maybe an option before sending out images where you can decide whetr they should be recoded is a nice option. Would need ui adoption though which is another burden...

[r10s]

Do you want to add recoding for all pngs sent?

all png that appear too large in bytes, yes.

for now, this is not about adding an option, that would be another effort.

so, as for jpg, this issue is about finding a good compromise, eg. leaving things alone that are not too large (already well created png). that also whatsapp is doing downscaling by default amplifies this approach, even if there are some who do no like that :)

finally, even if not obvious, you can send things unchanged when attaching as files (this is true also for JPG already today). but we should add that to the faq.

[Septias]

ah okay, if it is still possible to send images uncompressed, then I'm fine

[Simon-Laux]

but we should add that to the faq.

that has not really much effect, adding to the UI like telegram does it will be the real solution.

[r10s]

maybe, but that is another issue/effort.

[iequidoo]

Btw, we also should remove EXIF from PNGs and other images because it can contain sensitive info like your location. Here i do that for JPEG: #4037. And now i'm thinking about doing that for all images.

So, when i send images as images :), anyway a part of information is lost. At least there's no guarantee to the contrary. So why not also recode to JPEG to save traffic and storage and preserve code simplicity? If i want, i can send as file anyway. All gradations in between are o(all_use_cases) i think. If there are no objections, i can do because already touching this code

[r10s]

i think, there are still reasons to leave a png as is, eg if it is already small (a diagram in a png with few colors, already optimised, can be pretty small).

maybe we should think about PNG also more as of PDF or DOCX - yes there can be any meta data in both, however, they are not added "just so" in masses by the device when doing eg. a photo with the camera. at least, that could be our thinking until we have concrete counterexamples where PNG are filled with metabdata at scale without knowledge of the user.

so, i would handle this issue with focussing on PNG bytesize mainly.

maybe sth. like the following

// oxipng(img) // EDIT: maybe for some future, see comments below
if (img.bytesize > 500_000) {
   force_jpg(img)
   recode_to_image_size(img) // do some max. here,
                             // however, be more generous as WORSE|BALANCED_IMAGE_SIZE
                             // to not make screenshots look too bad, maybe double the max. allowed size;
                             // also take care not to scale up
}

as a result, most screenshots and other PNG should not be larger than 500k
cc @adbenitez

we could maybe expand the condition to !img.is_transparent however, that information may not be available "just so" - and i am also not sure about that - if there is a huge png that is not scaled down - that may be much worse than the removed transparency (which may not even be noticed - or is even better in some cases - and if in doubt, it may still be better to send a scaled-down jpg with 500k as a worldmap of the world in 20mb.

but i think, if the PNG pipeline is bootstrapped once, we can easily tweaked and optimized anyway.

[r10s]

i just came over sending screenshots on ios:

when drawing something inside a screenshot, apple also saves the result as JPG, keeping the resolution, just so, without asking the user. this underlines our approach of using JPG instead of PNG as outlined above, it will probably not even be noticed in many cases (i was not aware of the apple behaviour as well :)

i would definitely go for the PNG->JPG step and trying that out in practise before trying out oxipng - maybe the dependency and the effort for oxipng is just not needed for the vast majority of cases then.

[iequidoo]

@r10s, the nuance is also that now we have only limits on image width and height:

// max. width/height of images                                                                                                                                                                                       
pub const BALANCED_IMAGE_SIZE: u32 = 1280;
pub const WORSE_IMAGE_SIZE: u32 = 640;

So, i'm thinking about applying them for PNGs and other formats also (now it's done only for JPEG, but if we start doing it also for PNG then why not for all formats). But 1280x1280 PNG may be too big, so we should introduce such constants for the byte size also. Any suggestions? Please note however that screenshots will be likely always recoded because of this 1280 px limit. So, maybe increase limits on width/height (or remove at all) if we introduce limits on a byte size?

Also since i'm going to recode all EXIF-containing JPEGs (it is the easiest way to remove EXIF now), better to do that for all image formats. I checked on my Android that screenshots don't contain EXIF so they won't be recoded because of that.

EDIT: Maybe 500k for "balanced" and 150k for "worse"?