When people talk about end-to-end performance, they usually mean one dimension of performance: performance observed by end-user (from sending a request until getting full response). But without considering other dimensions of performance – such as load or data – such view is rather limited (although, as any model, is useful in many cases). Another dimension of performance is from the life cycle point of view: from the moment the system is conceived until it is retired. Different groups of people are involved in business analysis, design, development, testing, support, and maintenance. When it concerns performance, they often use different terminology, tools, and sources of information.
Holistic View of Performance
It is amazing that, while performance is the result of every aspect of the system, performance as a discipline consists of multiple silos and different groups of performance-related specialists have very limited communication between themselves.
Performance involves many different highly technical areas of expertise, so existence of different groups of professionals is explainable and well-grounded (with today’s technology pace you hardly could be an expert in all areas). But the need in a holistic view of performance considering all its aspects is often overlooked. Unfortunately ensuring performance in separate silos is not a guarantee of overall performance – so there is a need for better communication between different groups of performance professionals and a holistic view of all dimensions of performance. While some companies may have elements of it, we don’t see it often and it doesn’t look like this notion is widely accepted.
You hardly find any book or community covering all aspects of performance. Performance-related responsibilities are usually spread across several groups and rarely there is a person responsible for end-to-end performance throughout the whole lifecycle of the system. Ian Molyneaux’s post The Case for the CPO (Chief Performance Officer) has a point – but how far it is from the reality. Many performance experts I talked to do believe that it should be a person responsible for performance – but you hardly see it anywhere, not to mention CPO.
New Trends and How They Impact Performance
I’d expect that the current devops trend should amplify that need in a unified view of performance: devops, in the essence, is about tearing down silo walls. And I believe that performance is the first area where silo walls should be torn down – but it looks like performance somehow was left out. In the best case developers gets some exposure to performance issues with other performance silos left mostly untouched. But, unfortunately, it looks like in some cases many performance-related activities were just dropped as if introduction of devops would magically fix performance problems.
Development of performance engineering as a discipline definitely have cycles – and it is a pity that next cycles often start from a scratch and forget about what was done before. Yes, traditionally performance engineering was mainly concentrated on back-end. Just because front-end was rather trivial and didn’t impact much the bottom line. Yes, now front-end is far from trivial and deserves the attention it has now. But back-end and scalability issues haven’t disappeared and they deserve their share of attention too. I like Andy Hawkes’ post When 80/20 Becomes 20/80 bringing back some balance. Every performance specialist should understand Performance vs. Scalability and, even being a specialist in specific area, be aware about other performance-related areas. Moreover, all these aspects of performance should be considered together on some level.
APM and Its Impact
Application Performance Management (or Monitoring, APM) tools are, in a way, supposed to break down some silos providing end-to-end visibility into the system (at least end-to-end across different tiers and components of the system including workload and data considerations). Even leaving aside the discussion how these tools live up to their promise to provide full visibility across all tiers with minimal overheads, most silos still remain. APM assumes that we have people who may work with the information provided by the tools across all tiers and work with all teams to ensure system’s performance. Whatever issues were identified by (or with help of) APM, with exception of trivial cases when they may be handled automatically (for example, auto-scaling), they need to be addressed by professionals as part of performance engineering efforts (even if they are not officially named so). Meanwhile we have practically no across-the-silos performance professionals, nor have positions in organization org charts for such professionals.
Moreover, we have different ways to mitigate performance risks and even when APM will live up to their promises, we need other approaches (and professionals skilled in these approaches) to mitigate the risks. With APM, we base our decisions on existing information only. While theoretically it is possible to do modeling and what-if scenarios based on the current situation (which sounds as a natural extension of APM – although, to my surprise, it doesn’t look like any APM vendor has it beyond trivial trending or has much interest in it), modeling has its limitations and provides rather the best case scenario. If we need, for example, to assure that the system would handle certain workload load testing would be the best approach (assuming, of course, that it is done properly). And while there are some connections, all these professional groups mostly remain in silos.
Performance Communities
Velocity is probably the most popular performance-related conference for the moment, but it limits itself to a few performance areas. It is centered around front-end and web operations. Some scalable architecture patterns are discussed (mainly when backend processing may be parallelized). Other topics, such as load testing, APM, capacity planning, are mainly ignored. All these topics are still relevant to web operations, especially if we speak about e-commerce. And we still have traditional corporations with all their performance problems. Well, I suppose focusing on web operations is intentional – but it isn’t a comprehensive view of performance. What I am concerned here is that with all Velocity popularity it may make impression that the topics that are not covered are not important – and I believe that it may harm the industry.
For those who are interested in other aspects of performance, I’d suggest to attend Performance and Capacity conference by CMG, which is probably the only vendor-neutral conference that is covering most aspects of performance. Unfortunately it is rather small now and its impact on the performance community is rather limited. Unfortunately for the performance community – for attendees it is probably rather an advantage when you may easily reach out to renowned performance gurus in a cozy environment. I don’t position it as an alternative to Velocity – front-end coverage is rather light there for those who specialize in it – but rather as a perfect complement and maybe a better choice for those who do not specialize in front-end.