Web Performance Calendar

Writing JavaScript in a modular way has been an interesting field for exploration for many years. Since modules were not defined in the specification, the community invented a few different modules systems, three of which deserve a special attention:

• AMD – this was the most popular format for many years, primary used for JavaScript, running in a browser

• CommonJS – mostly used on the server, but thanks to projects like Webpack and Browserify they can be successfully loaded to the browser

• ES2015 modules – the official module format for JavaScript. Unfortunately, no single browser supports it yet.

There were of course other module systems. For example, the modules system of YUI deserves some respect, because it influenced significantly the community and helped TC39 to define the official module system in ECMAScript language.

During the years, regardless of the used module format, the most popular approach for loading the JavaScript files on the page was to bundle them together. Another one was to require them on demand, on page load or after some user’s actions. In some cases, a combination of both approaches was used.

Nowadays the situation is similar – the developers wrap the files in AMD or CommonJS format and use Webpack, or some other build tool to create a bundle file. In most cases, the developers create only one bundle file, however, when there are modules, shared among multiple pages, they very often extract the common scripts in a separate bundle file and then load it on the corresponding pages. This in general is a good approach and leverages the browser’s caching system.

Creating a single bundle file on the server requires knowledge about the dependencies in advance, since they are being resolved during the build process. Also, creating the bundle file in this way often ends up with loading unnecessary code to the browser. For example, it is not uncommon some Promise library to be included to the bundle and the same bundle to be loaded in each browser, despite Promises are not supported natively only on IE. The same may happen with some other API’s.

What happens however when the dependencies just cannot be resolved on the server in advance? This may happen mainly because the exact list of scripts which should be loaded on the page is unknown during the build process. It becomes clear only after the page is being constructed and ready to be served to the browser. Also, sometimes it is unclear if some script will be needed by the browser, or not. Some polyfils might be needed only for IE and detecting the browser on the server might be not an option.
In this case, the only option is to resolve them on the fly – in the browser. The idea is simple and it worked very well for years – having the information about the modules and their dependencies, a module loading system resolves the dependencies in the browser, constructs an URL and loads them on the fly via a combo service. The downside of this approach is of course the additional time used to resolve the dependencies on the fly and to request the modules from the combo service.

For the flagman product of the company I previously worked (Liferay Inc.) – a huge and very complex Portal solution, the situation was exactly that – which modules had to be loaded to the page, was unknown in advance. Not only that, but the project had to support IE9+, Chrome, Firefox and Safari. We figured out that there might be some parts of the code, only needed for IE. Combining that with the fact that it was impossible to create a single bundle and to detect the browser on the server, we didn’t have other choice except to trigger the loading of the modules in the browser. In the same time, we also really wanted to go a step further and leverage the benefits of ES2015 modules but to load them by sending as less request as possible to the server, ideally only one. This would be a significant performance boost.

And, we did it.

First, for those of you who are still not fully aware of ES2015 modules, they consist from two parts:

• Declarative syntax (for importing and exporting)
• Programmatic Loader API

With the declarative syntax, you write your modules as if they were CommonJS modules:

import {foo, bar } from "my-module.js";

function myModule() {}

export default myModule;

With the Programmatic Loader API, you load the modules more or less as if they were old school AMD or YUI modules:

System.import("some_module")
    .then(some_module => {
        // Use some_module
    })
    .catch(error => {
        ...
    });

Loading the modules using the declarative syntax is easy, but there is an issue. Even if they were supported by today’s browsers (and they are not), the browser would normally send multiple requests to the server. The point is that if the first module contains some dependencies, the browser should resolve them, then to load the corresponding files, parse and execute them. Imagine what will happen if these dependencies have other dependencies too. We may end up with multiple requests, and it is common understanding that they are very expensive regarding to performance. Switching to a new protocol, like HTTP/2 may fix that (especially leveraging the push promise) but in some cases if might be not supported by the browser, by the server or simply to not work well for you, as it happened for Khan Academy.

Facing these issues, we discussed numerous possible solutions, dropped one by one all of them, and finally ended up with the following:

1. Write the code using ES2015 modules and their declarative syntax (for importing and exporting)
2. Transpile them to AMD during the build process. We use Babel, but you can use other tools too. Babel however is currently the most advanced tool, which transpiles code from the future to today’s browser, so when in doubt, go with it.
3. Create a common file, which described the modules and their dependencies. You can do that by hand, but we wrote a special NodeJS tool, which parses the JavaScript files, extracts the modules definitions and creates the list of dependencies automatically. It is Open Source and free to use.
4. Use an AMD loader, which is able to load the AMD modules using a combo service. We searched for some external one, but we didn’t discover any to fit our needs, so we wrote our own. It is also free and Open Source, supports conditional loading too.

Of course, you will needed a combo service on your server, but that is the easiest part.

After the process of transpilation, you end up with AMD modules under the hood and a config file, containing the description of the modules and their dependencies. On each page, this file is being included, so the Loader knows how to resolve the dependencies among modules. Then, once some modules are being requested, the Loader resolves their dependencies and usually only one request is being made to the combo service. It returns the needed modules and that is all.

Conditional loading of modules is another interesting topic. Imagine the following situation – you want to load a Promises library, but only if the current browser does not support them natively. This is called conditional loading and it is a very powerful technique – when some module is triggered, which requires Promises for example, you check if they are natively supported by the browser and if they are not, load a Promises library, so the first module will continue to work. There is still no easy way to do this using the declarative module syntax, but in our loader we managed to achieve it. Everything which should be done is to add a META label in the body of the function:

META: ({
    condition: {
        test: function() {
            var el = document.createElement("input");
            return ("placeholder" in el);
        },
        trigger: "my-module"
    },
    path: "my-module.js"
});

During the process of creation of the config file, this label will be parsed, recognized and included to module’s description.

Pros and cons of the loading of ES2015 modules via combo service:

1. When there is no way to resolve the dependencies on the server, this is the only option.
2. Polyfills will be loaded only when the browser really needs them.
3. There is no initial rendering blocking – if you are doing progressive enhancement (and you should), the markup and CSS are coming first and then the JavaScript loads and executes. No any blocking here.
4. As soon as browsers start to support the modules, and especially if you can leverage HTTP/2, you will drop the whole thing and everything will continue to work (except maybe the conditional loading part, which should be addressed in addition).

As a drawback, we can mention the additional time to resolve the dependencies and to make the request to the combo service.

Conclusion

The results for us were very promising. We were excited that we were able to write our code in ES2015 modules format using the declarative syntax without performance penalties.