This being the Performance Calendar, you probably already know that slower websites lead to customer frustration, disengagement, and eventually loss in sales. You may also know that these effects are even higher amongst mobile users. Testing tools like WebPageTest (WPT) are regularly used to test website performance (desktop and mobile) to ensure that webpages are loading in an acceptable way. Further, there are tools that automate WPT for regular testing of your site for regular tracking of performance.
Native App Performance
But what about mobile apps? We know instinctively that mobile apps that are slow are going to face the same customer reactions as slow websites. The awareness of performance issues in the developer community has been helped by Colt McAnliss’ Android Performance Patterns series on YouTube. I also recently published a book on building High Performance Android Apps.
It goes without saying that mobile apps use the network as much as websites, and unoptimized mobile apps can also be slow. I’ve found that the issues that trip up mobile website performance are the same issues that impact the performance of native mobile applications. But what tools are out there to test with?
Performance Testing Tools
The tools that native app developers have for testing the performance of their app are getting better, but they are still in their infancy. In general, they suffer from requiring a lot of time to set up and learn to use before any meaningful results can come out.
To test the network performance of your native app, AT&T’s Application Resource Optimizer (ARO) is one of the best tools out there. I’ve written about ARO in previous Perf Calendar posts, and we are constantly working to improve the usage of ARO to make it easier to use. When you test a native app with ARO (iOS or Android), it collects a network trace on your device, and grades the data your app uses against 23 best practices – pointing you to regions of potential improvements.
Native App Continuous Integration
As I mentioned above, the biggest barrier to widespread adoption of ARO has been the manual testing requirement. Manual testing does not lend itself to repeatable, scheduled testing. It is hard to build in time to allow for manual testing in a regular manner (monthly, on every release, etc.). To keep an eye on your app’s performance, tools that integrate easily into a Continuous Integration framework are required.
So today, I am happy to announce the release of ARO 5.0 Beta. This release contains Continuous Integration APIs that allow remote startup and stopping of the ARO collector (and then automated analysis of the traces collected). The APIs run in both Java and using a script from the command line. With a device tethered to your computer, one command will start the collector:
sh arocli.sh --startcollector vpn_android --output /Users/demo/AROTraceAndroid/CommandLineTest --video yes
This will start the ARO collector (VPN method) on your tethered device. To top the collection, you simply enter:
stop
in the same terminal window. The file that is created can be analysed with a subsequent command:
sh arocli.sh analyze /Users/demo/AROTraceAndroid/CommandLineTest --output /Users/demo/AROTraceAndroid/CommandLineTest/CLT.json --format json
The output can either be a json file with a full data dump, or HTML – listing the pass/fail of each ARO Best Practice.
By wrapping the start and stop commands around existing automated tests, we hope to allow developers to quickly begin the process of automating their performance testing around test cases they have already automated.
Conclusion
Native application performance suffers from a lack of automated tools for testing. We are beginning the sea change to automated testing for native performance with ARO 5.0. While ARO is not integrated into any CI testing frameworks, with a little work, it should be pretty easy to set up.
This is a beta release, so the code is not yet ready for open sourcing, but this will occur with our full release in early 2016. On the ARO download page, look for the ARO 5.0 downloads. The install has all the required tools and examples to get you started. If you try the tools, please pass on your feedback. We’d love to hear how you might think of using these CI tools!