Web Performance Calendar

The speed geek's favorite time of year
2015 Edition

New Year’s Diet: Brotli Compression

25thDec 2015 by Eric Lawrence
ABOUT THE AUTHOR

Eric Lawrence (@ericlaw) spent eight years on the Internet Explorer team at Microsoft. After three years developing Fiddler full-time at Telerik, he will join the Google Chrome Security team in 2016. His updated book on Fiddler is available in paperback and PDF formats.

Did your site put on a bit of weight in 2015? The new Brotli compression algorithm can help you lose the weight (and the wait) in 2016!

Brot—what??

I love Zopfli, Google’s DEFLATE compression engine that often shrinks output by 5% or better when compared to the popular zlib engine. The beauty of Zopfli is that its output is compatible with all of the billions of existing DEFLATE encoders deployed worldwide, making its use an easy choice for any static text content.

But imagine for a moment what compression ratios we could achieve if we weren’t limited by compatibility with existing decoders? If we could add a new compression engine to the web, what might it look like?

The Brotli compression engine, co-written by Jyrki Alakuijala (inventor of Zopfli), provides one answer. Brotli combines the LZ77 and Huffman algorithms of DEFLATE with a larger sliding window (up to 16mb1 vs. DEFLATE’s 32kb) and context modeling; the specification also calls for a 122kb static dictionary. Today, Brotli is the compression engine behind the WOFF 2.0 font format, providing savings of approximately 25% over WOFF 1.0 fonts compressed with using Zopfli/DEFLATE.

Not content to rest on their laurels, Google has announced their Intent to Implement Brotli as a general purpose HTTP Content-Encoding, allowing web developers to use it to compress script, stylesheets, svg, xml, and the like. Mozilla beat Google to the finish and Firefox 44 will feature Brotli support in January 2016.

Results

Facebook investigated Brotli and found it would save about 17% of CSS bytes and 20% of JavaScript bytes (compared with Zopfli). When run on the CSS and JavaScript from the Alexa top-300k, Brotli saved an additional 12% of CSS bytes and 9% of JavaScript bytes (compared to gzip).

Running a few simple tests with Fiddler, I saw great results with Brotli:

Brotli Results charts

Brotli is optimized for high-speed decompression. When compressing, Brotli is slower than zlib’s DEFLATE, but considerably faster than Zopfli, lzma and bzip2. Brotli makes the most sense for pre-compression of static content where the server pays the compression cost once and reaps the bytes-transferred benefit over and over.

Google has released a Brotli module for NGINX that compresses both dynamic and static content. The process of bringing Brotli to other platforms (like Java/Tomcat) is well underway.

HTTPS Only

Past attempts to add new compression algorithms (bzip2 and SDCH) have demonstrated that a non-trivial number of intermediaries (proxies, gateway scanners) fail when Content-Encodings other than GZIP and DEFLATE are specified, so Brotli is likely to only be supported over HTTPS connections, where intermediaries are less likely to interfere.

    Accept-Encoding: br, gzip, deflate, sdch

For unsecure HTTP connections, the client will not advertise or accept Brotli-compressed content.

A Quick Test

To see whether Brotli is supported in your browser already, try downloading this image which is served with Brotli encoding:

Brotli didn't decode

If your browser supports Brotli, it will render a star.

Try Brotli Using Fiddler

Fiddler supports Brotli as a Content-Encoding everywhere compression is supported; simply download the Authenticode-signed Windows Brotli.exe and place it in the \Fiddler2\Tools\ subfolder in your Program Files folder. After you restart Fiddler, you will find a new Brotli option on the Transformer tab and Fiddler APIs like utilDecodeResponse() will be able to decompress Brotli-encoded content:

Fiddler Transformer tab with Brotli option

This capability allows you to easily explore the impact of Brotli compression on any of your live site’s assets without making any serverside changes.

Running Brotli.exe on Windows

The Brotli executable can also easily be run from the command line.

To compress a file, specify the input and output filenames:

  • --in filename
  • --out compressed_filename

… and optionally specify any of the following arguments:

  • --quality n
  • --force
  • --verbose

The quality parameter controls the compression-speed vs. compression-ratio tradeoff; the higher the quality, the slower but denser the compression. The supported range is 0 to 11, and 11 is the default.

The force parameter instructs Brotli to overwrite the output file if it already exists.

The verbose parameter instructs Brotli to display its compression speed in megabytes per second upon completion.

To decompress a file, specify the input and output filenames:

  • --in compressed_filename
  • --out filename

… and optionally specify the --verbose parameter to instruct Brotli to display its decompression speed in megabytes per second upon completion.

Brotli in 2016

After you’ve optimized your site’s caching and its image assets, enhancing your use of compression is the next logical step. Brotli is an exciting new algorithm that should prove straightforward to deploy and I can’t wait to see its adoption in the new year!

If you’d like to follow along, check out the following references:

 

1 While Brotli can use a 16mb window, for performance reasons it appears that constraining the window to 4mb is the plan for most scenarios.