Prefetching is a technique that can be used to achieve near-instant page loads. This technique has been used for decades to speed up hardware and compilers; but it’s only been recently, thanks to the introduction of the Network Information API, that it’s usage to speed up web applications has become practical. The Network Information API makes it possible to use different prefetching strategies for different connection types and this vastly improves the performance of prefetching. This article will explain how prefetching works and how developers can start using it.

Part 1: Understanding prefetching

What is prefetching?

Prefetching is the strategy of anticipating the resources that a user will need in the future and downloading them before they are needed. This eliminates waiting for resources to download and can result in very fast page loads.

Prefetching is based upon predictions about what resources a user will need next: The more accurate the prediction – the more effective the prefetching will be.

Prefetching fetches resources at the “Lowest” priority: This is done to give higher priority to the resources that are needed for the current page. If the resources for the current page and the resources for the next page were loaded with the same priority, they would compete with each other for resources (i.e. bandwidth), and this would hurt the page load performance of the current page.

Prefetching can eliminate waiting for resources to download: If a resource has been prefetched it will already be in the cache when it is needed. This can noticeably improve page load performance, particularly for large resources and/or slow connections. (Note: “prefetching” is just the process of fetching resources in advance. The browser will still need to parse, compile, and execute the resources before they can be used.)

Prefetching should be conscientious of network conditions: The biggest downside of prefetching is that unused prefetches waste resources – namely bandwidth. For this reason, it’s important to take network conditions into account when deciding whether to use prefetching.

Prefetching strategies

Prefetching is only as effective as the accuracy of its predictions about which resources to fetch in advance. These are some of the most common strategies for determining what to prefetch:

1. Interaction-driven prefetching

Interaction-driven prefetching uses a user’s mouse and scroll activities as signals to determine which resources to prefetch. For example, if a user slows down their scrolling when they come across a particular search result, the search engine could prefetch that page.

2. State-driven prefetching

State-driven prefetching uses an app’s current state (usually a page or URL) to determine which resource to prefetch. For example, a store’s “Shopping Cart” page could prefetch the “Checkout” page. Analytics data can be very helpful in determining which page a user is most likely to visit next from a given page. This approach to prefetching is a real-life application of Markov chains.

The Markov Chain for a checkout flow.

3. User-driven prefetching

User-driven prefetching uses signals from a particular user, rather than all users, to determine what to prefetch. These signals may include things like account information or past usage of a site. For example, if a social network notices that a user tends to click on a particular type of article, they could automatically prefetch these types of articles when the user logs in.

4. “Download everything” prefetching

This approach doesn’t try to predict the user’s next navigation – instead it downloads “everything” the user might need next. This can be applied in different ways depending on the site: for example, prefetching all the links on a page, or, prefetching all the bundles that make up an app.

Putting it all together

In practice, the boundaries between these types of prefetching can blend and there’s no reason why a site can’t apply more than one of these techniques. For example, this is how Nikkei, a Japanese newspaper publisher, uses prefetching:

  • Nikkei prefetches the “next section” for each of its sections. For instance, if the sections are ordered “Economy, Politics, & World”, the “Economy” section prefetches the “Politics” section and the “Politics” section prefetches the “World” section. Nikkei did this after noticing that many users only read the headlines of each section.
  • Nikkei automatically prefetches the search page when users move their mouse close to the search button.

By adding just these two optimizations they were able to achieve 75% faster page loads by using prefetch.

Part 2: Implementing Prefetching

The previous section discussed the strategy of prefetching. This section discusses how to implement prefetching by using the prefetch resource hint.

The Prefetch Resource Hint

The prefetch resource hint makes it easy to use prefetching. A prefetch resource hint looks like this:

<link rel="prefetch" href="/shoppingCart.html" as="document">

This resource hint would prefetch shoppingCart.html.

This is how the prefetch resource hint works:

  • The prefetch resource hint can only be used with a link element.
  • The href attribute is mandatory and specifies the resource that should be prefetched.
  • The as attribute is technically optional, but should be included whenever possible. This attribute tells the browser the content type of the resource (e.g. script, document, font, etc.). This helps the browser prioritize requests, set headers, and determine whether the resource already exists in the cache.
  • All major browsers except Safari support prefetch.

Patterns for using resource hints

The prefetch resource hint can be made more powerful by combining it with other techniques and APIs. These are two common usage patterns:

Use client-side scripts to dynamically add prefetching

JavaScript can be used to dynamically add tags to a document. Prefetching that is triggered by a user’s mouse or scroll actions is an example of prefetching that uses technique.

When executed, the script below will append a link tag to the document. This will result in the queueing of a new network request with a priority of “Lowest”. Like any other network request, it will be scheduled and executed based on it’s priority.

if (mouseclick) {
  const linkTag = document.createElement('link');
  linkTag.rel = 'prefetch';
  linkTag.href = 'https://www.example.com';
  linkTag.as = 'document';
  document.head.appendChild(linkTag);
}

Note: The fact that prefetched resources are fetched at the “Lowest” priority is a key difference between a script like the one above versus a script that uses XHR or the Fetch API to load resources. If a script uses XHR or Fetch, the resource will be fetched at the “High” or “Highest” priority. As a result, it is likely that the resource will be fetched as soon as the script is executed. If the current page is still loading, this can interfere page load and hurt performance. This is why the prefetch resource hint, and not XHR or Fetch, is the preferred method for loading resources needed for the next page navigation.

Use the Network Information API to adjust prefetching based on network conditions

The biggest drawback of prefetching is that unused prefetches waste user data; this has the greatest impact on users with slow connections. However, this can be mitigated by incorporating the user’s network conditions (via the Network Information API) into the logic that determines whether a resource should be prefetched.

A simple approach is to only prefetch on fast connections and/or for specific resources that are very likely to be used.

if (window.navigator.connection.effectiveType === '4g') {
  // prefetch
} else {
  // do nothing
}

A more complex approach is to calculate the “likelihood of usage” for each individual resource. This information, combined with the user’s connection speed, is used to decide whether a resource should be prefetched.

const connection = window.navigator.connection.effectiveType;
if ((connection === '4g' && likelihood > .25) ||
    (connection === '3g' && likelihood > .5) ||
    (connection === '2g' && likelihood > .75)) {
  // prefetch
} else {
  // do nothing
}

Prefetch Tooling

Humans

No special tooling is needed to use prefetching. Manually adding prefetch resource hints is a simple, no-dependencies way to use prefetching.

Build Tools

Webpack 4.6.0 added support for native prefetching. Native prefetching requires manually indicating the resources that should be prefetched, but this is very easy to do (it takes < 1 line of code / resource). Older versions of Webpack should use the preload-webpack-plugin instead.

Prefetch Tools

These tools automate many of the implementation details of using prefetching.

quicklink is a very small script (<1KB gzipped) that automatically prefetches links that are in the viewport. By default, it will not prefetch links on 2G or for users with data-saver enabled.

Guess.js

Guess.js consists of two tools that use Google Analytics data to implement prefetching:

  • GuessPlugin is a Webpack plugin that analyzes Google Analytics data to determine how an app should be broken into bundles. These bundles are prefetched as the user navigates the application.
  • guess-static-sites is a workflow for setting up a prefetching system that automatically adds and updates prefetch resource hints based on Google Analytics data.

Gatsby

Gatsby is a site generator for React. Gatsby automatically prefetches all the links on a page. React’s website is an example of a site built on Gatsby and a good way to experience firsthand what a prefetched experience “feels like” from a user’s perspective.

Thank you to Yoav Weiss, Jeff Posnick, Shunya Shishido, and Tarun Bansal for their input on this article.

ABOUT THE AUTHOR

Katie Hempenius

Katie Hempenius (@katiehempenius) is an engineer at Google where she works on web performance.

4 Responses to “All about prefetching”

  1. Jan

    What happens if you e.g. trigger the prefetching on hover of a link or when the mouse comes near a specific section, the user then clicks that link before the prefetch request finished? Will this just wait for the prefetch call to end, or will it drop the prefetch and create a new request?

  2. Ben

    Very handy article, Katie, thanks for sharing. I didn’t know about the `window.navigator.connection` object and looking forward to that having a greater browser coverage for it.

    Definitely bookmarking this article and will resort to it frequently.

  3. Dan

    Could prefetching skew analytics? For example, does a prefetched webpage trigger a pageview in Google Analytics?