Hello, fellow web perf enthusiast! Contribute to the 2024 edition happening this December. Click.

Web Performance Calendar

The speed geek's favorite time of year
2012 Edition
ABOUT THE AUTHOR
Carlos Bueno

Carlos Bueno is an engineer at Facebook, where he helps paint on the go-faster stripes. He is also the author of Lauren Ipsum, a series of children's books about computer science and critical thinking.

The internet is the largest and most interesting artifact created by mankind. Every country, every town, every no-longer-lonely island contains a piece of it. The rest lies under the waves. But which waves, which islands, in what order? That’s the most interesting part of all. The patterns that shape the internet are both older and younger than you think.

Picture yourself in Sao Paulo, Brazil. You’re writing an email to a friend in New Zealand, about 7,500 miles to the left. Or maybe you’re calling her, or viewing a website she hosts. To the internet it’s all the same: a temporary caravan of data packets circulating between points A & B. One measure of the power of an idea is how many different things can be expressed in terms of it.

A packet is simply a string of data (ones and zeroes, if you wish) of a specified length. A handful of packets can contain a web page; a short phone call will need a few hundred in both directions. The first bits of each packet describe where the data is from and where it’s going, what number it is in the sequence of packets that make up the entire message, and so on. There are layers of this header information, similar in spirit to the barcodes and stickers on a UPS package.

Packets can exist in many forms, which is part of their charm. Over the course of their flight they will be voltage spikes on a copper wire, lost in the murmurs of thousands of frequencies of light in an optical fiber, ripples in microwave beams booming over valleys of sugar cane, and brief flashes in the RAM of a network device before warping off in another direction and another form.

You send the email. What happens next is counterintuitive. Your packets go on a trek 21,000 miles in the wrong direction. They pass through both hemispheres, five continents, a couple of dozen countries, and too many cables to count. This is by design.

A Series of Tubes

The packets are handed off from your computer to the WiFi router above your head. The router converts the wireless signals to digital pulses and hands them off to the cable modem, which hands them off to the upstream “head end”, owned by your ISP. From there they bounce around various networks in Sao Paulo over larger and larger tributaries until they gain entry to one of the fiber optic submarine cables that navigate the maze of the Caribbean islands. Your packets, finally in the fast lane and possibly out-of-order, come ashore in Miami and enter a grand caravanserai called the Network Access Point Of The Americas.

In an industry that favors anonymous warehouses tucked out of sight, the NAP of the Americas is about as inconspicuous as Epcot. It’s a brightly-painted fortress in the middle of downtown Miami, visible from two major highways. Around it are waterfront hotels, the financial district, sports stadiums, a mall, and the opera house. Before September 11 made everyone paranoid they even offered tours with interactive exhibits and a peek at the power room.

The paranoia is probably justified. Terribly Important Infrastructure lives inside this NAP. Part of the e911 system lurks in unlabeled cabinets. There’s a backup of one of the 13 root DNS nodes, a secure area belonging to the US Southern Command, and machines representing major telecoms and internet companies from all over. There are a lot of eggs in this basket. That’s why it exists. It’s in here and places like it, where cables intersect, that different networks exchange packets to form a global inter-network, aka the internet.

After a brief stay to switch from one network to another, your packets tack north up the coast over fiberoptic trenches to another exchange in New York. They kick east over one of the many, many cables crossing the Atlantic, then zip straight through southern England and out the other side. Another little-known internet nexus is Belgium, and that’s where your packets go next.

A look at the map shows that this isn’t a bad place to land cables. Boat traffic, specifically the anchors and fishing trawlers that come with it, are death to submarine cables. Belgium is out of the danger zone of the English Channel and centrally placed to branch off to all of Europe. Crossing the Mediterranean, your packets arrive at the ancient city of Alexandria then cut across the desert and re-enter the water just after the Suez Canal. After exiting the Red Sea they skip along the bottom of India, Indonesia, Australia, and finally to your friend in Aukland.

Switches Route And Routers Switch

If this meandering packet swarm seems rather involved, you’re right. Chopping your email or phone call into bite-sized chunks, throwing them into a great river of data, and reassembling them at the other end is very complicated, in the sense of having many steps. But all of the paperwork is done by computers and computers are cheap. And, believe it or not, it’s very efficient.

Consider the alternative. Fifty years ago you would have had to rent the use of thousands of miles of copper wire, hundreds of switches snapping closed in just the right way to form a temporary electrical circuit all the way to New Zealand. This system, called circuit-switching, dominated from about 1900 to 1960 and was a hack built on top of the telegraph network. Telephony won because talking is more user-friendly than TRADING MESSAGES LIKE THIS FULL STOP. But while I’m using all those wires to talk, you can’t use them, even if I’m just listening or breathing. The number of simultaneous circuits the system could support was inherently limited.

This combination of harsh limits on circuit-switching and high demand for what it could do eventually birthed a third way. Simplifying horribly, packet-switching is like a telegraph with the humans replaced by computers, more flexible routing, and sped up a billion times. It turns out that if you send little telegrams fast enough, you can create the *illusion* of a continuous voice circuit in the same way a film strip creates the illusion of motion.

Packet-switching is one of the most important ideas in modern history. Packets did for information what the standard shipping container did for cargo. A shipping container abstracts away everything about what’s inside except its weight. The same cranes, ships, trucks and rules can be used on a million different cargoes without a second thought. The packet abstracts away everything about information except its length.

Having a standard box to put things into and rules about how those boxes move simplifies global exchange, no matter whether it’s bits or atoms. Simplified exchange means you can do more of it, cheaper, faster, and more reliably. And so we have: you can use your old phone to track the progress of your new phone as it travels from a factory in Asia to your doorstep. In less than thirty years nearly all human telecommunications was converted to packetized routing. It happened so smoothly that hardly anyone noticed.

But why do the packets go this way instead of that? Because that’s where the cables go. So why do the cables go that way? And for that matter, why on earth are there so many cables connecting North and South America along the east coast, only one along the west coast, and none from the South to Asia? Why does that email go the long way round?

Topology Recapitulates Hegemony

The answer is a mix of geography, demography, and history. Let’s take geography: the Pacific Ocean is wide and treacherous. On top of that, the western half of both continents are dominated by mountain ranges, the Rockies and the Andes. Crossing them with a cable or road is very expensive. Second is demography: a lot more wealthy people live on the Eastern side of those mountains than the West. Third is history: New Zealanders didn’t colonize South America. Europeans did.

The internet is a physical expression of trade volume between cities. A long submarine cable takes hundreds of millions of dollars and endless planning. Their builders tend to follow rather than lead. There would be little point and enormous risk in laying a cable between two places that don’t do a lot of business already. So as a general principle, the more bandwidth there is between two points, the higher the volume of economic exchange. An extreme example is New York & London, which at last count enjoy ten direct links.

This logic applies to all forms of transport infrastructure. A trade route, once proven to be profitable, tends to attract more traffic and more infrastructure to itself. It’s no coincidence that internet nexuses are almost always established banking and shipping hubs, or that telegraph lines followed the rails. Cargo follows money follows data follows cargo, in a self-reinforcing cycle that can last for centuries. It’s the original network effect.

Of Sailing Ships and Sealing-Wax, Babbages and Pings

Look at a map of shipping routes from 1800 compared with a map the telegraph network of 1900, with the internet of 2000 on top of that. Notice anything?

They are nearly identical. The internet is just the latest upgrade to a trading network that was established three or four hundred years ago. Data and money move faster today, but they travel the same routes that carried Ben Franklin’s letters and Mark Twain’s telegrams.

One big difference between 1800 and 2000 is the Suez Canal, which cuts shipping times by skipping a trip round Africa. It’s used by everybody. Any attempt to stop traffic on this canal is a serious act of war. By now you shouldn’t be surprised to learn that nearby Alexandria is a major internet nexus. It lands no less than nine cables that carry a majority of the data traveling between Europe and Asia. The world’s cargo floats on top of the Red Sea while the world’s data flows along the bottom.

An incident during the Arab Spring showed just how strategically important these nexuses are. Whatever else he was, General Hosni Mubarak was not a fool. He cut internet access throughout Egypt during the revolution to cripple the opposition, but two data streams were left untouched. One was to the Egyptian Stock Exchange. The other was the torrent of bits passing *through* Egypt to somewhere else. Even for a cornered, murderous dictator clinging to power, damaging the internet was as unthinkable as damaging the canal. The international community would have destroyed him and he knew it.

Plus Ça Change

Internet connections are lagging indicators of trade and economic importance, and by their nature they boost trade even more the second they are lit up. The global pattern changes slowly, but it does change. It’s changing in a major way right now.

There is relatively little bandwidth between Brazil (pop 170M) and the United States (pop 320M). There is only one direct cable between their economic capitals, laid down in the heady dotcom days. Over the last fifteen years not much else has happened. Now several more cables are planned by upstart consortia, to be finished in a few years.

But here’s the real news: in recent years Africa has been wired up by loops all round the continent. And there are *three* cables being laid across the Atlantic from Africa to Sao Paulo. For the first time these two continents (and by extension, Asia) will be linked with serious bandwidth. The geography of the southern Atlantic route has always been straightforward. The demography and history are catching up. Brazil has become one of the largest economies in the world and is impatient to show it. China is now their #1 trading partner and Turkey is gunning for the #4 spot.

As time goes on, the cat’s cradle of cables between the US & Europe will no longer be unique. Your email from Brazil to New Zealand will still go right instead of left, but it will take a more direct route along the bottom of the globe instead of stomping through the northern hemisphere and centuries of old history.

The North/South trading routes, the old patterns we take for granted, are losing their primacy. Cargo follows money follows data follows cargo, and trade routes tend to reinforce themselves. The countries connected by this “Southern Loop” are increasingly talking to each other, if for no other reason because they can. Never underestimate the transformative power of SMS, low-lag video games, low-friction money transfers and cheap web hosting. It’s going to be an interesting ride.