It looks like there is great interest to quantifying performance impact on business, linking response time to income and customer satisfaction. A lot of information was published, for example, Aberdeen Group report Customers Are Won or Lost in One Second or Gomez whitepaper Why Web Performance Matters: Is Your Site Driving Customers Away? There is no doubt that there is a strong correlation between response times and business metrics and it is very good to have such documents to justify performance engineering efforts – and some simplification may be good from the practical point of view – but we should keep in mind that the relationship is not so simple and linear and it may be cases when it would matter.

Response times may be considered as usability requirements and are based on the basic principles of human-computer interaction. As long ago as 1968, Robert Miller’s paper Response Time in Man-Computer Conversational Transactions described three threshold levels of human attention. Jakob Nielsen believes that Miller’s guidelines are fundamental for human-computer interaction, so they are still valid and not likely to change with whatever technology comes next. These three thresholds are:

  • Users view response time as instantaneous (0.1-0.2 second)
  • Users feel they are interacting freely with the information (1-5 seconds)
  • Users are focused on the dialog (5-10 seconds)

Users view response time as instantaneous (0.1-0.2 second): Users feel that they directly manipulate objects in the user interface. For example, the time from the moment the user selects a column in a table until that column highlights or the time between typing a symbol and its appearance on the screen. Robert Miller reported that threshold as 0.1 seconds. According to Peter Bickford 0.2 second forms the mental boundary between events that seem to happen together and those that appear as echoes of each other.

Although it is a quite important threshold, it is often beyond the reach of application developers. That kind of interaction is provided by operating system, browser, or interface libraries, and usually happens on the client side, without interaction with servers (except for dumb terminals, that is rather an exception for business systems today). However new rich web interfaces may make this threshold important for consideration. For example, if there is logic processing user input so screen navigation or symbol typing becomes slow, it may cause user frustration even with relatively small response times.

Users feel they are interacting freely with the information (1-5 seconds) : They notice the delay, but feel that the computer is “working” on the command. The user’s flow of thought stays uninterrupted. Robert Miller reported this threshold as one-two seconds.

Peter Sevcik identified two key factors impacting this threshold: the number of elements viewed and the repetitiveness of the task. The number of elements viewed is, for example, the number of items, fields, or paragraphs the user looks at. The amount of time the user is willing to wait appears to be a function of the perceived complexity of the request.

Back in 1960s through 1980s the terminal interface was rather simple and a typical task was data entry, often one element at a time. So earlier researchers reported that one to two seconds was the threshold to keep maximal productivity. Modern complex user interfaces with many elements may have higher response times without adversely impacting user productivity. Users also interact with applications at a certain pace depending on how repetitive each task is. Some are highly repetitive; others require the user to think and make choices before proceeding to the next screen. The more repetitive the task is the better should be response time.

That is the threshold that gives us response time usability goals for most user-interactive applications. Response times above this threshold degrade productivity. Exact numbers depend on many difficult-to-formalize factors, such as the number and types of elements viewed or repetitiveness of the task, but a goal of two to five seconds is reasonable for most typical business applications.

There are researchers who suggest that response time expectations increase with time. Forrester research of 2009 suggests two second response time; in 2006 similar research suggested four seconds (both research efforts were sponsored by Akamai, a provider of web accelerating solutions). While the trend probably exists (at least for the Internet and mobile applications, where expectations changed a lot recently), the approach of this research was often questioned because they just asked users. It is known that user perception of time may be misleading. Also, as mentioned earlier, response time expectations depends on the number of elements viewed, the repetitiveness of the task, user assumptions of what the system is doing, and interface interactions with the user. Stating a standard without specification of what page we are talking about may be overgeneralization.

Users are focused on the dialog (5-10 seconds): They keep their attention on the task. Robert Miller reported threshold as 10 seconds. Users will probably need to reorient themselves when they return to the task after a delay above this threshold, so productivity suffers. Or, if we are talking about Web sites, it is the threshold when users start abandoning the site.

Peter Bickford investigated user reactions when, after 27 almost instantaneous responses, there was a two-minute wait loop for the 28th time for the same operation. It took only 8.5 seconds for half the subjects to either walk out or hit the reboot. Switching to a watch cursor during the wait delayed the subject’s departure for about 20 seconds. An animated watch cursor was good for more than a minute, and a progress bar kept users waiting until the end. Bickford’s results were widely used for setting response times requirements for web applications.

That is the threshold that gives us response time usability requirements for most user-interactive applications. Response times above this threshold cause users to lose focus and lead to frustration. Exact numbers vary significantly depending on the interface used, but it looks like response times should not be more than eight to 10 seconds in most cases. Still, the threshold shouldn’t be applied blindly; in many cases, significantly higher response times may be acceptable when appropriate user interface is implemented to alleviate the problem.

So while there is a strong correlation between response times and business metrics, it is definitely not a linear function. We are touching here the psychology of human-computer interaction and it is definitely not a single-dimension issue. It is very context-specific and published data should be used carefully with understanding what really stands behind them. The main practical conclusion is that you may have a point when further performance improvement won’t make much sense: you have increasing costs of performance improvement with diminishing business value. Although it looks like most existing systems haven’t reached this point yet.

ABOUT THE AUTHOR

The last fourteen years Alex Podelko (@apodelko) worked as a performance engineer and architect for several companies. Currently he is Consulting Member of Technical Staff at Oracle, responsible for performance testing and optimization of Hyperion products. Alex currently serves as a director for the Computer Measurement Group (CMG). He maintains a collection of performance-related links and documents.

7 Responses to “How Response Times Impact Business?”

  1. Scott Barber

    I can’t find the link to the reference right now, but it was an IBM study that classified “instant” as .25 seconds or less. Not a significant difference — in fact, a completely irrelevant difference for the kind of systems we’re talking about (at least for plenty of years to come). Both studies support my response to “How fast is fast enough?” — which is:

    “Until you get to 250ms, consistently, the only answer that matters is ‘faster’. Once you get there, we’ll talk.”


    Scott Barber
    System Performance Strategist & CTO, PerfTestPlus, Inc.
    Director (Marketing), Computer Measurement Group
    Author, Web Load Testing for Dummies
    Co-Author, Performance Testing Guidance for Web Applications

    http://www.perftestplus.com
    http://cmg.org
    http://about.me/scott.barber

    “If you can see it in your mind…
    you will find it in your life.”

  2. Alexsandro

    Very good, I would like get more and more topics about user experience under response time.

  3. Alex Podelko » How Response Times Impact Business?

    [...] my How Response Times Impact Business? post in 2011 Performance Calendar. Categories: Performance Requirements Tags: Performance [...]

  4. Alex Podelko

    Scott,

    It is not how I interpret the research data I mentioned in the post. 250 ms (or similar) is for the first threshold, instantaneous, and it is what we want for navigation inside the page. For navigation between complex pages we are talking about the second threshold and, according to quoted research data, it may be significantly larger (2-5 sec) without impacting user performance / satisfaction. Haven’t seen any real research proving the opposite yet.

    Alex

  5. Carl Kyonka

    I’m not sure the following reference is the same as the IBM paper mentioned previously, but it is at least similar. At the time, I am pretty sure we were getting TSO response times over a second, and it was definitely slowing us down.

    The Economic Value of Rapid Response Time by

    Walter J. Doherty, Manager of Systems Performance and Technology Transfer for the Computing Systems Department at IBM’s Thomas J. Watson Research Center
    • Ahrvind J. Thadani, Advisory Engineer at IBM’s General Products Division headquarters, San Jose, California.

    http://www.vm.ibm.com/devpages/jelliott/evrrt.html

  6. Alex Podelko » Response Times: Digesting the Latest Information

    [...] == "undefined"){ addthis_share = [];}Returning to the discussion around my post How Response Times Impact Business? and recent publications about the topic, like For Impatient Web Users, an Eye Blink Is Just Too [...]

  7. Email Deliverability

    I recently came across an interesting poll on CSS-Tricks about the ideal page weight/size for today’s modern web design. We worked really hard at optimizing the front end of this very site for page load time, using asynchronous assets, CSS sprites and tons of other things to create a happy user experience, but I’d never thought about the total page size ’til now!

    So I opened Firebug to check out Dyn.com, which clocked in at 195.9K and could change a few kilobytes per day depending on the featured images we have on our homepage. Chris Coyier suggests that very few sites today come in at under 200K which “used to be a common goal”.

    Used to be? Why can’t it still be?

    Check out the advantages of dyn dns

Leave a Reply

You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
And here's a tool to convert HTML entities