Every year when it comes time to plan for what WebPageTest is missing it always feels like it’s basically “done” but then when the year closes out there’s always a fairly long list of things that were added or improved.

One of the main focus areas for 2015 was to continue improving the situation for mobile testing, particularly when using real devices. Probably the biggest improvement for 2015 was support for testing Safari on iOS with low-overhead 60fps video capture. The networking data still isn’t as good as what we can get out of Android but it’s orders of magnitude better than how things were previously (where we were stuck on iOS 5, WebView only and 1fps video with high overhead). As of this writing, there are 3 iOS 9 devices available in the Dulles location (iPhone 5c, 6 and iPad Mini 2).

Some other improvements to mobile testing include:

• The reliability of the mobile agents was a huge focus as well and they can now run for months at a time without human intervention.
• Traffic shaping support was rolled out to the public mobile devices, allowing for testing of arbitrary network conditions instead of a fixed 3G profile.
• Custom Metrics, Navigation Timing and User Timing are all now available from mobile device testing (and the work lays the foundation for bringing full scripting support to mobile).
• Content blocking and Single Point of Failure (SPOF) testing were added for mobile devices. The SPOF testing in particular is important because the timeouts on Android (60s) can be significantly longer than Windows (20s).

There were several new partnerships/integrations:

• Tim Kadlec launched the very-awesome “What Does My Site Cost” tool to provide information on what it costs users to load a given page over a mobile connection and it is now integrated directly into the WebPageTest test results.
• Akamai deployed a set of test agents running in all of the EC2 locations for users to use with the REST API. I used to assign API keys manually to keep testing under control but thanks to the new agents you can now self-register for an API key and start testing right away.
• HeadSpin deployed test agents on real mobile networks globally. With the huge variability in quality and behaviors of mobile networks it is great to be able to test on live carrier networks and validate the synthetic traffic shaping profiles. Look for a group of locations at the top of the location list labeled “Real Mobile Networks”.

Other notable additions:

• SpeedIndex without Video capture (desktop).
• Support for reporting HTTP/2, including server-pushed resources.
• Fixes to keep up with Chrome/Firefox updates.
• Updated Trace and Timeline viewers. The new flame charts in the Dev Tools timeline viewer are particularly helpful when tracking down JavaScript issues.
• Side by side screenshot UI for comparing several screen shots to each other.

2015 By the Numbers

It looks like web performance testing is still going strong.

• 14,526,767 tests (6,812,634 interactive, 7,714,133 API)
• 8.8 TB of new test results (~24 GB/day, 29 TB total going back to 2008)
• 565,135 new videos rendered
• 2,429,229 Unique visitors
• 21,908,984 Page Views
• 20 Outside Contributors (contributing code)

The Darker Side

I know it will come as a total shock, but offering a free service with no authentication requirements ends up getting abused from time to time, mostly by people trying to turn it into a free monitoring service either by automating the UI or registering for multiple API keys and rotating through them.

I try to keep the service as open as possible so people can generally run as many tests as they need to while working on a site but there is a bunch of monitoring in place that I use to identify suspicious traffic. When that happens I can block the testing (usually by IP or site being tested) and the user will get a message like this.

Unfortunately several domains have had to be blocked from testing so just seeing the message doesn’t mean you’ve done something wrong, it could just be that you tried testing a site that is not allowed. For the most part that is usually a result of testing YouTube (spammers were using WebPageTest to inflate video views) or an URL shortener (often used to try to hide testing of a blocked domain).

Here are the current stats:

• 173 IP addresses/subnets blocked from submitting tests.
• 255 Domains blocked from being tested.
• 11 IP addresses blocked at the firewall. This is usually only done in extreme cases where someone has a script that went out of control and is hammering on the web server.
• 7 EMail domains blocked from API key registration.

Coming Next Year

I can’t guarantee that all of these will get done but this is roughly the list of things on my radar to work on for WebPageTest in the coming year:

• Improved iOS device scaling. Right now it takes a dedicated Mac for every iOS device used for testing.
• Device selection for mobile emulation. WebPageTest allows for mobile emulation using desktop Chrome but is currently fixed to emulating a Nexus 5. I’d like to expand the list to include a mix of Android and iOS devices (phone and Tablet).
• Mobile agent WPT scripting. Adding support for multi-step tests on the mobile agents.
• Microsoft Edge agents. Hopefully Microsoft will release extension support for Edge in 2016 which will make creating an agent trivial but even if it doesn’t happen we can leverage the WebDriver interface it already supports.
• Chrome HTTPS decode. There are a bunch of limitations to the HTTPS (and HTTP/2) reporting for the Chrome desktop agents because we do not have access to the decrypted streams. Work is underway to fix that which will open up more accurate reporting, optimization checks and response bodies.
• Azure agents. In addition to the existing EC2 AMI’s, make public images available for deploying agents on Azure.
• Integrations with the Page Speed Insights API and SSL Labs API.
• Full-site spider. Instead of testing just a single page, crawl a site and provide an aggregate report.
• SPOF detection. Detect likely 3rd-party Single Points of Failure and allow for a one-click SPOF test.

If you have other things you’d like to see or want to weigh-in on any of the features above, please feel free to open a github issue (https://github.com/WPO-Foundation/webpagetest/issues). WebPageTest is only as useful as it is because of feedback and ideas from the community and for that I thank you.