Web Performance Calendar

On November 3 2010 Google announced the first release of an Apache module that can help speed up your site: mod_pagespeed. The mod will perform speed optimizations automatically. Currently the mod has 15 filters built in and we can all expect Google to add more in the future.
Mod_pagespeed is supposed to be the easy solution for making your site faster: install, configure and run. Easy as pie. Users will be happy, your traffic and conversion goes up. Free magic!

Up to 50% faster, really?

The mod_pagespeed announcement article by Google mentioned very promising speed gains: “We’ve seen mod_pagespeed reduce page load times by up to 50% (an average across a rough sample of sites we tried) — in other words, essentially speeding up websites by about 2x, and sometimes even faster”. Sounds great, but as a natural skeptic to these kind of percentages, I went on to read what the mod actually does.
Of the 15 filters, only 9 are part of the CoreFilters, which is turned on by default. The others have to be turned on manually. This is because those other filters are not considered safe enough: they may break page design/functionality. So, out of the box, mod_pagespeed runs in ‘Safe Mode’. I’ll shed some more light on these filters further down. Knowing a little bit about what mod_pagespeed does, I decided to run some tests on real world web pages, and compare page load times with mod_pagespeed on versus off.
The big question is: what speed gain does mod_pagespeed deliver in ‘Safe Mode’, for empty browser cache pageviews?

Test methodology

I installed mod_pagespeed on Nov 8, and didn’t upgrade, so I may not have been using the latest version.
My server is a VPS located in The Netherlands, running Apache 2.2.8 on Linux Ubuntu with KeepAlive On and mod_deflate doing compression.

The installation of mod_pagespeed went fine. I modified the pagespeed.conf as per the instructions (view it here).

I downloaded the homepage of 8 major Dutch travel sites and saved all page objects on my server.
Then, I fixed all the broken links to the objects and I made sure domain sharding was kept intact.
The third party stuff (trackers, widgets) was left untouched.

Testing was done on the awesome Webpagetest.org, using the Amsterdam test locations.
For each page I did 3 test runs, each run having 5 tests. The median of each run was logged and the page load time is the average of the 3 medians.

I did First View tests only, for two reasons:

1. Speeding up First View is more important than Repeat View (the browser cache will not be primed for most visitors)
2. It saved me some time 😉

Test results

Let’s look at data first.
Below are two tables, one for IE7 and one for IE8, showing the Doc Complete times (time untill all page objects finished downloading) for the 8 pages with mod_pagespeed on versus off.

Results for IE7 – Doc Complete

Results for IE8 – Doc Complete

Note:

Suprisingly, the average speed gain is 1% negative IE7 and 2% negative in IE8. In other words: practically nihil in both browsers.
Averages tend to hide interesting information, so let’s highlight a few data points:

• Max speed increase in IE7 is 6% (OAD)
• Max speed decrease in IE7 is 13% (Weekendjeweg)
• Max speed increase in IE8 is 13% (Sunweb)
• Max speed decrease in IE8 is 18% (Weekendjeweg)

The impact of mod_pagespeed varies quite a bit. The Weekendjeweg page is the big loser and Belvilla is the overall winner.
Let’s take a closer look at both of them.

Weekendjeweg

Mod_pagespeed reduced the number of HTTP requests for Weekendje weg by 1 in IE7 and by 3 in IE8, of the original 53, which cannot make a big positive impact. So why did the page get slower? I don’t know. The median of 2 of the 3 test runs is actually sightly better than with the mod turned off. The median of run 3 is a lot higher (+58% in IE7, +56% in IE8), so that number brings up the average quite a bit. Why was the median for run 3 so much higher in both browsers? I don’t know. The IE7 and IE8 test machines are in different locations on different networks. Maybe my server or the mod had hickups. Perhaps the best conclusion is to say that the number of tests was too low (do you agree?).

Belvilla

The number of HTTP requests for Belvilla went down by 12 in IE7 (131 > 119) and by a whopping 47 in IE8 (125 > 78). As expected that improved the Belvilla load times significantly.

Mod_pagespeed reduced the number of HTTP requests for all pages in both browsers, with an average of -4 in IE7 and -15 in IE8. But not for all pages did this result in better load times. Vrijuit for example got 7 less requests in IE7 and Doc Complete was not impacted. Number of requests for Cheaptickets dropped by 14 in IE8 and Doc Complete went up by 5%.

Conclusions

Mod_pagespeed may slow down your site, speed up your website significantly or anywhere in between, while running in Safe Mode. You have to test it yourself on your own site to find out if this mod is for you. Run many tests on many pages. Hopefully, you get that 50%, but that is not likely in Safe Mode. But hey, even with 15% you should be pretty happy. After all, that is worth the effort of installing the mod and doing the very minimal configuration.

If you use the non-Safe Mode filters the speed gain will very likely go up, but maybe not that much. Most of these filters will likely not have that big of an impact (e.g. Elide Attributes) and/or will not be applicable to many sites (e.g. Combine Heads). The Minify JavaScript filter is worth turning on and testing (it’s classified as High risk), because that may shave off up to 30% of the file size of the JavaScript code.

My wish list of mod_pagespeed improvements

Not taking into account how difficult it is to implement these filters, this is what I hope will be added to mod_pagespeed (just some things that came to mind):

• Crunch CSS background images (currently only images in the HTML are optimized)
• Even better: base64 encode CSS background images + generate one CSS file with MTHML (IE6, IE7) and one CSS file with data URIs for other browsers + reference these CSS files in the HTML using conditional comments
• Remove empty image/script/link tags
• Remove redundant favicon tags if favicon.ico is in root of the website
• Optimize *where* in the HTML the JavaScript is (aka: move down as far as possible)
• Apply a non-blocking JS loading technique where possible (this is probably very hard to implement)

Closing note: the mod_pagespeed team is very responsive and helpful via Google Groups. Kudos to them!

Mod_pagespeed links

• Mod_pagespeed description (start page)
• Download mod_pagespeed
• Mod_pagespeed source code (on Google Code)
• Mod_pagespeed discussion forum (on Google Groups)