Web Performance Calendar

Introduction

You are probably wondering: “What a user centric metric like INP has to do with Puppeteer? Isn’t Puppeteer a tool that lives in the synthetic world?”, or why “Tsvetan, who works on mPulse, a Real User Monitoring system, is messing with synthetic tools?”.

The truth is that mPulse and the other Real User Monitoring systems are great about giving us an early warning that something is going wrong but we still need to “zoom in” and understand the root cause.

In this article I would like to walk you through an approach that we developed in order to help a customer of mPulse. We wanted to estimate how a specific change in a JavaScript file would affect the INP score. The customer’s website was very complex and there were many factors that were affecting the INP. E.g. complex DOM, CSS animations, reflows, DOM updates on user interaction, a JavaScript library that does bot and threat detection and more.

I won’t name the customer but with the help of my imagination I will wrap a nice story.

As a result of our work the customer implemented a small fix for one of the problems. This happened in the middle of November and we noticed an improvement of INP by a few percent globally. It’s not a massive improvement but it’s a move in the right direction.

Together with our customer we identified an easy fix for one of the cases where a reflow was affecting the INP. We waited a few days in order to get over 10 000 page hits for each of the page types (Homepage, Flights Search and Flights Listing) and then we compared the “before vs. after” results.

Below is a table that shows the INP improvement of the top 5 countries for Homepage, Flights Search and Flights Listing.

Page Type	P10	P25	P50	P75	P95
Homepage	6%	12%	10%	8%	4%
Flights Search	7%	10%	11%	12%	9%
Flights Listing	10%	16%	17%	14%	15%

I quickly learned that the INP fixes seem to be complex and seem to require a lot of brainstorming and planning from the engineering team that maintains a website.

My prediction for 2024 is that the web developers will be spending more and more time in the Developer’s Tools – Performance tab and will be mastering the art of improving the responsiveness of their websites.

And finally before I dive in I hope that this article will inspire and will save time for other web performance geeks that are decoding the mysteries of INP 😉

The problem of the imaginary customer

Earlier this year we added monitoring of INP to Boomerang JS and we started collecting INP data together with a few customers that were interested. One convenient thing we do in mPulse when we work on such proof of concepts is to start sending data from Boomerang JS to mPulse even before we are ready to display the new data points in the mPulse dashboards. Then with the help of Data Science tools and specially crafted SQL we query the mPulse RAW data and we create statistics and analyze trends. This helps us mostly to understand the data and to get an idea how we should display the data later in the mPulse dashboards.

A customer that sells airplane tickets online (let’s pretend for the sake of the story telling 😉 ) and with whom I frequently meet noticed very high INP values in the CrUX data and asked us if we could find more details in the RAW mPulse data.

We ran a few reports with our Data Science tools and we found interesting things.

Clearly, there was something going on with the input#datepicker element.

Credits to my colleague James Bricknell for setting up the Data Science tools, helping me to generate the INP chart and brainstorming together with me.

The RAW data/Boomerang beacons

In the collected mPulse RAW data we could find the following:

• et.inp – the recorded INP duration.
• et.inp.t – a timestamp marking the INP start time.
• et.inp.e – CSS selector of the DOM element that a user interacted with.

An example of a RAW Boomerang JS beacon:

{
    ...
    et.inp: 128,
    et.inp.t: 8465
    et.inp.e: "input#datepicker"
    ...
}

Root cause analyses

We knew that the high INP values were caused when a visitor was clicking/tapping on the date picker input:

We could give the new LoAF API a try to further understand why this element was causing an issue, but initially it was more convenient to stick to the tools available in the browser as we were comfortable with those and felt they could identify the issue. Together with our customer we started recording and analyzing Chrome Performance traces.

A lot was going on the page when the date picker was being displayed. There were a bunch of AJAX calls, tracking pixels were sending events, CSS animations, new DOM elements were inserted and bot detection algorithms were running in the background.

The performance traces were really noisy and everything that was happening on the page was adding a few milliseconds here and there to the INP.

We decided to start reducing the noise:

• From Chrome Dev tools we blocked a few requests that were loading the Google Tag Manager and other telemetry related scripts.
• With Chrome Local overrides we commented out a few CSS animation rules.
• With Chrome Local overrides we commented out a few JS code paths that were causing DOM manipulations and reflows.
• We work on modern developer machines but the real users of our customer were mostly using Android phones. That’s why we artificially slowed down the CPU 4 times in order to see clearly in the performance traces what was affecting the INP.

The Pretty Print functionality of the Chrome Dev Tools was really helpful when modifying CSS and JS files because the CSS and JS files were minified and of course indentation was missing.

For example, this is how minified source code looks like:

And this is how the same source code looks after applying Pretty Print

Every time when we were changing something we were loading the page and we were clicking on the date picker input and then we were writing down the INP value that we were getting from the official Core Web Vitals Chrome extension.

Let’s measure (manually)

One of the performance bottleneck candidates was the function getBoundingClientRect() which was called on interaction with the date picker element.

A good list of things that cause reflows can be found here: https://gist.github.com/paulirish/5d52fb081b3570c81e3a

First we measured 10 times the INP on the original page without any modifications but later we measured 10 times a version where the getBoundingClientRect() call was removed.

Note: For the final fix the engineers of our customer substituted getBoundingClientRect() with something that doesn’t cause reflows but we just wanted to explore what is the maximum possible room for optimization here.

Our setup was

• 4x slowed down CPU.
• Removed getBoundingClientRect() with the help of Chrome Local Overrides.

We were repeating this manual check about 10 times in order to calculate the 75th percentile.

 * We all know that if we measured 100 times but not 10 times we would have more confidence in the numbers but in this case measuring 10 times was just fine.

Puppeteer for the win

We had a few more ideas that we wanted to validate but doing this manually was not convenient anymore. It was time consuming and we just wanted to try new ideas as quickly as possible.

So … why not using Puppeteer and scripting everything we were already doing manually?

I created and shared boilerplate Puppeteer project that automated everything we needed: https://github.com/ceckoslab/inp-measure-puppeteer

And here are a few of the challenges that had to be solved:

1. There was a Cookie Consent popup. We had to bypass it.
2. We had to slow down the CPU 4 times.
3. Just in case we had to simulate the User-Agent and viewport size of a popular Android mobile device.
4. We had to use an overwritten version of some of the customer’s assets.
5. We had to simulate interaction with the element in question in order to cause a slow INP.
6. We had to measure INP.

1. Bypassing the Cookie Consent popup

The Customer was using Optanon Consent and we figured out what cookies are being created on visitor consent. We identified 2 cookies: OptanonConsent and OptanonAlertBoxClosed and we created a simple helper function that was creating these cookies with Puppeteer:

module.exports = class Cookies {

  constructor() {};

  // TODO: Modify cookie values, domain and path

  static async setOptanonConsent(page) {
    await page.setCookie({
      "name": "OptanonConsent",
      "value": "ADD YOU COOKIE VALUE HERE",
      "domain": ".example.com",
      "path": "/",
      "secure": true,
      "sameSite": "None" // or 'Strict' or 'None'
    });

    await page.setCookie({
      "name": "OptanonAlertBoxClosed",
      "value": "2023-11-07T12:55:33.827Z",
      "domain": ".example.com",
      "path": "/",
      "secure": true,
      "sameSite": "None" // or 'Strict' or 'None'
    });
  }

};

2. Slow down the CPU 4 times

It’s self explanatory but we had to call the following function:

page.emulateCPUThrottling(4)

3. Simulating an User-Agent and screen size by a popular Android phone

We created the following helper function:

// Define the device properties for Samsung Galaxy A51
const GalaxyA51 = {
  name: "Galaxy A51",
  userAgent: "Mozilla/5.0 (Linux; Android 10; SAMSUNG SM-A515F) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/12.0 Chrome/79.0.3945.136 Mobile Safari/537.36",
  viewport: {
    width: 1080 / 2,   // The width of the viewport in pixels. Divide by device scale factor for actual pixels.
    height: 2400 / 2,  // The height of the viewport in pixels. Divide by device scale factor for actual pixels.
    deviceScaleFactor: 2, // The device scale factor.
    isMobile: true,    // Whether the meta viewport tag is set to mobile.
    hasTouch: true,    // Whether the device supports touch events.
    isLandscape: false // Whether the device is in landscape mode.
  }
};

module.exports = class DeviceEmulation {

  constructor() {};

  static async emulate(page) {
    await page.emulate(GalaxyA51);
    await page.emulateCPUThrottling(4);
  }

};

4. Override customer assets

Below is the snippet that allowed us to override the customer’s assets. In the example we are “waiting” for a JavaScript file with name example-js-of-interest.js . When such a request was intercepted we were serving not the original file content but the content of a file located in overrides/example-js-of-interest.js on our local file system.

 // Enable request interception
  await page.setRequestInterception(true);

  // Add event listener to intercept requests
  page.on("request", (interceptedRequest) => {
    // Check if the request is for the resource you want to override
    if (interceptedRequest.url().endsWith("example-js-of-interest.js")) {
      console.log("Intercepted and overriding: " + interceptedRequest.url());

      // Create a response from a local file
      const overrideContent = fs.readFileSync(path.join(__dirname, "overrides", "example-js-of-interest.js"), "utf8");
      interceptedRequest.respond({
        status: 200,
        contentType: "application/javascript; charset=utf-8",
        body: overrideContent
      });
      return;
    }

    // Allow all other requests to continue normally
    interceptedRequest.continue();
  });

5. We had to simulate interaction with the element in question in order to measure INP.

const elementToInteractWith = "input#datepicker";

 // Wait for the element to be present in the DOM
 await page.waitForSelector(elementToInteractWith);

  // Click the input input
  await page.click(elementToInteractWith);
  await page.focus(elementToInteractWith);

6. Measure the INP

We literally copied and reused the minified code of the of the Web Vitals library from here: https://unpkg.com/web-vitals@3.5.0/dist/web-vitals.iife.js

​​Instrumentation:

module.exports = class CWV {

  constructor() {};

  static async attachCWV_Lib(page) {

    await page.evaluateHandle(() => {
      // Including the CWV library
      window.webVitals = function(e){"use strict";var n,t,r,i,o,a=-1,c=function(e){addEventListener("pa..
    });
  }

}

Measure:

 // Execute JS code after the timeout
  await page.evaluateHandle(() => {
    window.webVitals.getINP(function(info) {
      if (info.value) {
        console.log("inp: " + info.value);
      }
      else {
        console.log("inp: not measured");
      }
    },
    {
      reportAllChanges: true
    }
    );
  });

The results

Run	Original	Removed getBoundingClientRect()
1	264	176
2	240	176
3	248	160
4	240	168
5	240	176
6	248	168
7	232	160
8	232	176
9	224	168
10	224	160
	P75	P75
	246	176

Great results I have to say. In the 75th percentile the INP was reduced with 70ms which is nearly ~29%.

It’s important to note that the simulated results don’t completely match the CrUX data for a few reasons:

• The real users CPU sometimes are way faster or way slower than the 4 times slowed CPU we used for our experiment.
• Also the suggested fix/improvement for INP was deployed in the middle of November where we see at the end November the P75 INP was reduced with 25 ms globally. This means that the fix was available only for the second half of November. I would expect even better results for a full month, so let’s see what will be the situation for December.
• The engineers substituted the getBoundingClientRect() with IntersectionObserver which improved the performance. However, in the simulation where we removed getBoundingClientRect() we also missed to run a few DOM manipulations which skewed the results a bit.

The END

Congratulations! You reached the end of this article. I hope that by reading it you have gotten inspiration and new ideas. I don’t have super powers and I can’t see in the future but something makes me think that 2024 will be the year of the INP and testing whether improvements to address INP issues are having the desired effect. Automating this through the likes of Puppeteer as we have done here, can hopefully help with this.

I would like again to mention where you can find the Puppeteer boilerplate project that automated the measuring of INP: https://github.com/ceckoslab/inp-measure-puppeteer

And finally, special thanks to Barry Pollard for proof reading this article and making great great suggestions and to my colleague James Bricknel for helping me with the Data Science tools.