RFE: Use PNG post-processing compression improvement tools

[johnezang]

Request For Enhancement

Please consider the use of PNG post-processing compression improvement tools as one of the final steps before "packaging a release distribution". Using the PNG compression tools

• advpng
• optipng

And the command (in the MathJax/fonts/HTML-CSS/TeX directory of the v1.0.1a zip distribution):

find png -type d -exec sh -c "optipng -q -o7 {}/*.png ; advpng -q -z4 {}/*.png" \;

I was able to reduce the size of the png directory by approximately ~381 KBytes. Granted, it's not much, maybe 1.2% or so of the total, but each little bit helps. The above find command is obviously just a quick hack, season to taste.

Obviously doing a PNG optimization pass on the 'originals' will produce the most benefit the first time it's run. This is a fairly trivial step to add and once the results are 'checked in' it will produce lasting benefits. Again, season the specifics to your projects work-flows taste.

[dpvc]

This has come up in the user's group as well.

I point out the following:

The real reason why the font directory takes up so much space, in
general, is not the actual data size, it is the block size for files
on disk. This is the size of the "chunks" of disk space that are
allocated for files. Every file takes up a multiple of the block
size. For my disk, the block size is 4K, so every file will take up
at least 4K of disk space, even when the data stored is less than that
and most of that space will be unused. With about 29,000 files, that
means that the image fonts will take up a minimum of about 116,000K
or about 113MB regardless of how compressed they are. Nearly all the
individual files are smaller than the block size, so further
compression will do nothing for the total space used. On my system,
the font directory takes up 114.2 MB of disk space (for only 7.75 MB
of actual data), so it is already pretty much at the lower limit for
this block size. That means the only way to reduce the disk space
used would be to reduce the number of files involved (which would mean
reducing the number of sizes maintained), not compressing the data
further.

which has relevance here as well. I just did a quick check, and no image is larger than the 4K block size, and less than 20 are over 1K. So it looks to me that there is really no actual savings to be had by further compression.

Davide

[johnezang]

This is specious. First, not every file system works as described, and second, making the images smaller via compression optimization means that the amount of data that HTTP has to transfer across the network is less than it would be otherwise. This means that over time, the amount of bytes, and time, saved continues to accumulate with every HTTP GET request.

[dpvc]

Can you say how much disk space is actually used by the fonts on your system (not data size but actual blocks on the disk)? I don't know of a common filing system that doesn't use a minimum block size. In the old days, I have seen the block size as small as 512 bytes, but with the larger disks these days, 4K is pretty common. I would be surprised to see anything as small as 1K, which is already too big to make a difference. The comments from users who are concerned with the size of the image fonts usually report sizes in accordance with those I described above.

As for the network transmission, I don't know the TCP/IP protocols, but if they use a fixed packet or frame size, then the same argument may apply there as well.

Davide

[johnezang]

IMHO, the file system argument is a red herring. In the worst case, that is to say it is "true" universally, it's sort of moot- the amount of storage saved or gained is fairly trivial relative to the size of hard disk space available.

Where it can make a difference is on the network side of things. Every byte saved is one less byte that needs to be transmitted. TCP/IP itself is a "variable length packet" protocol, but this is not the same thing as saying that the layer 2 TCP/IP is being run on top of is variable length (i.e., TCP/IP over ATM must be decimated in to fixed sized cells, hence there is a "cell tax" for running TCP/IP over such layer 2 networks).

This can pay the most dividends when the bytes/sec is either "slow" or has a per unit charge, such as what you would find on contemporary "cell phone networks", i.e using an iPhone to view MathJax content.

By optimizing the PNG images, this potentially reduces the amount of traffic from web server to browser. This means the content can be displayed sooner (particularly over slow byte/sec links, or links that are prone to a high error rate, at least relative to a hard wired connection), and possibly cost the browser user less money.

This benefit is quite real, and the effort to maximize it is trivial (i.e., the instructions at the start of this ticket). There is no downside to doing this that I know of.

[dpvc]

As I said, I am not familiar with the details of the underlying network protocols, so I am not in a position to evaluate the situation in detail, and will bow to your greater knowledge. I didn't realize your original message was about network traffic, and assumed you were talking about disk space (which is what most people have complained about). Sorry for the confusion. I agree with you that the disk space is trivial in comparison to disk space available these days, but people complain about it anyway. So I responded with that in mind.

I'm hoping to eliminate the image fonts in the near future, so am not looking to put more time into them (I'm not saying we won't take your suggestion, but simply that it may not be right away). With the recent release of iOS4.2 for the iPhone and iPad, which make web-based fonts available on those platforms, the only use cases left for the image fonts in a network setting are for old browser versions (Safari before 3.1, Firefox before 3.5, Opera before 10, iPhone before iOS4.2). As these recede into the distance, we should be able to decommission the image fonts entirely. They may be made into an optional install to support older browsers for those who require that. Version 1.1-beta includes support for web-fonts in iOS4.2 (which is not the case for earlier versions of MathJax).

I'm curious about the amount of usage the image fonts get. You might look at the number of times MathJax/fonts/HTML-CSS/TeX/imagedata.js is loaded in comparison to the number of times MathJax/jax/output/HTML-CSS/jax.js is loaded, for example. It should mainly be the iPhone/iPad users.

Davide

[fred-wang]

I've added the PNG compression to my refactoring branch of MathJax-dev:

mathjax/MathJax-dev#4

However, as Davide said we are probably going to remove these PNG files from MathJax in the future. Perhaps that could still be useful for people willing to prepare a customized version with PNG fallback (but it seems that by default people prefer to remove these images).

[fred-wang]

I've opened #631. I only called

        find png/ -name *.png | xargs optipng -q -o7; find png/ -name *.png | xargs advpng -q -z4; 

on the current image fonts.

[dpvc]

=> Merged