Leon Fayer (@papa_fire) currently leads engineering at Teaching Strategies. Leon has over two decades of expertise concentrated on architecting and operating complex, web-based systems to withstand crushing traffic (often unexpectedly). Over the years, he's had a somewhat unique opportunity to design and build systems that run some of the most visited websites in the world and has the opinion that nothing really works until it works for at least a million people.
It’s not what you look at that matters, it’s what you see. — H.D. Thoreau
Most performance conversations, at one point or another, will get to the subject of caching. And for a good reason. Caching helps to hide some of the biggest performance inefficiency in many systems. That’s right, hide not fix. Whether it’s local storage of data, pre-calculated data set or a transient storage to reduce a number of server-side requests, all these methods are improving performance by limiting the frequency of expensive operations. Again, it is important to remember, cache does not improve the performance of the operations themselves, but rather reduces/delays/eliminates the need to execute those operations.
Don’t get me wrong, limiting a number of expensive connections or operations is important, this is why caching is a common component in most modern architectures. Even short TTL can yield you significant performance boosts. This is why it’s somewhat surprising to me that other good practice techniques that accomplish similar goals, but don’t fall under traditional definition of “caching”, often get ignored. Specifically, I am talking about scope caching™.
What is “scope caching”™?
Scope caching™ is something that most of you familiar with programming are likely already doing in some capacity. The simplest example of scope caching is a basic programming principle of not putting computational operations in a loop where possible. Running operations outside the loop, storing it in the variable and using a result variable in the following loop/code path is an example of reuse of pre-computed value. Which, in essence, what caching is trying to accomplish.
How many scopes are there?
Many. As many scopes as there are in a programming language and than some. That’s why it’s important to take time and understand the needed scope of cache requirements.
For example, scoping a variable globally is considered a bad practice. However, when working with a web language, global scope could be a webserver thread or webserver session. Loading something once and keeping it for the life of that scope for all the users can (and should) be completely acceptable solution. Pre-generated constants is a good example. If you (like many) store your countries and states in the database, loading them during a web server start and keeping them in global memory, instead of going to the database on every request, is a form of scope caching.
And speaking of web, REQUEST
and SESSION
are probably the two most common scopes that can be taken advantage of.
Consider a function that sits on top of auth service, that returns a current user object. It’s really simple. Useful. Widely used.
function currentUser() { const currentUser = user.getCurrentUser(); return currentUser; }
A lot of different modules and services take advantage of it to validate user’s access control, setting, etc. And because of that, it’s being called, dozens of times per page load, dozens of requests to your database/cache/user service, over the network, to retrieve the exact same information on a single page. Service design ftw.
Problem? Of course. But an easily solvable problem, without the need to re-engineer the service.
public function currentUser() { if ( !isDefined("REQUEST.currentUser") ) { REQUEST.currentUser = user.getCurrentUser(); } return REQUEST.currentUser; }
Storing a result of the first-retrieved user object in a REQUEST
variable, persistent for the duration of that, well, request, will make the information available to different consumers on the same page without a round trip to the source.
Scope caching™!
The list of scopes above is not exhaustive, although it (hopefully) provides a good illustration of the diversity of scope levels and techniques to limit operations. Remember, if you need to ask for the information more than once, make sure to cache the response. (hey, it rhymed!)