Why do I need to test the performance of mobile apps?

There are even multiple new businesses that operate and grow value on the basis of mobile applications – WhatsApp, Spotify, Tinder and even a few mobile wallets. What many people don’t know, is that 25% of the apps are deleted after their first use in 2019 (09/2020 Statista). One of the reasons is lacking performance, which is a no-go for many users in our fast-paced world. That’s why we at Greenliff strongly believe that performance is key and should be tested before every app release. This blog article describes the important aspects of mobile performance testing and shares some best practices. 

Areas that require performance testing

There are a few areas that have to be taken into account when testing the performance of apps.

• Validate the hardware performance of the mobile device. One of the most important aspects of performance testing is if low-performance CPUs or little storage will prevent the app from functioning according to the requirements. This evaluation can be used to identify the recommended devices that your app can support. 

• Evaluate the performance of an application during peak workloads. The performance of the system typically lacks as the number of concurrent users or actions increases. It is important to know response time and the amount of system resource consumption at the app’s peak points. Especially when your app has traffic spikes, the data that can be gathered during those tests can give important feedback if the backend hardware is sufficient in the current system setup.

• Determine the capacity of the application. You have to determine if the current software is capable of matching the KPIs that have been set (peak traffic load, server response time, and so on). This is very useful if your app requires a constant network connection. Network connections such as 3G, 4G or WLAN (with different bandwidth) need to be simulated.

• Performance under critical conditions (Stress Test). With performance testing, a developer will be able to determine the limits of the app’s performance. This way, you’ll be able to predict and prevent system crashes. 

Best practices 

Isolate the network.

This is done in order to eliminate timeout errors. However, if the bandwidth is wide enough to support both a tester and other users, it’s not always necessary. For most networks, supporting a few concurrent activities without affecting the performance of the app is impossible. 

Remove proxy servers from the network path.

A proxy between the client and the web server can put a strain on the performance of the app. The problem can be solved by conducting server transfers in a secluded environment. If a proxy is always required, it is actually necessary to include them into the test to simulate a realistic scenario. 

Isolate the test environment.

You need to ensure you’re the only one to use the system for testing and no other activity is performed on it simultaneously. Otherwise, you’ll jeopardize the test accuracy. If you can not isolate the system, another best practice is to schedule the tests when there is a time with little or no activity on the system

Use test data generators.

Most tests are reliant on database records. When doing performance testing, it’s crucial to pay attention to data writing, reading, and deletion as these actions are known to halt the app’s performance to the fullest extent. In order to match the database records with those used in the test environment system, QA specialists use data generation tools. 

Step-by-step guide 

Step 1. Define goals and test objectives 

In order to know what to test, you need to align the goals of performance testing with your business stakeholders. Once everyone has a clear understanding of the goals of these tests, they are typically described in non-functional requirements. At this point in time, the delivery team knows the architecture, its UI, business, and data levels of the product and will have valuable input to this process whereas the product owner can provide the priorities of the features that are most important from his perspective such as cart feature for e-commerce stores, the registration and contact form, and so on. 

Step 2. Identify test KPIs

The KPIs are measurable benchmarks for testing to enable you to decide if the execution was a success or a failure. For mobile app testing, these KPIs are commonly used:

• Error rate;
• Maximum response time;
• Average response time;
• The peak number of requests;
• Average throughput and the peak number of concurrent active users per device and OS.

Step 3. Prioritize scenarios

According to the input of your business representative (i.e. Product Owner) you set priorities for the scenarios. If your high priority performance requirements fail early, you might want to repeat the tests after the root cause was identified and fixed instead of blindly moving forward. This way you can ensure that the most important performance benchmarks are fulfilled. This can be time-consuming and could mean that you have to drop other lower priority performance requirements when the time before the release runs short.

Step 4. Simulate a real-life test environment

The testing environment is essential in order to understand the user experience of the app and should be as close to reality as possible. One approach to speed up the testing process is the use of system emulators. 

Such tools do a good job of emulating the basic parameters of an operating system and allow testers to get a sense of the look and feel of the interface. However, to assess the performance of features that require camera permission, GPS, and other device-specific functions, it’s better to use real hardware. 

Step 5. Align the testing approach with the development methodology

Testing should fit into the project development framework. When agile development is in place, it makes sense to implement the principles of continuous integration. If a company is using Waterfall, a tester should align his workflow accordingly. Adopting the same methodology for testing as for development allows both parties involved to follow a holistic technology approach, improve communication, and speed up the decision-making process. 

Step 6. Test the latency and the bandwidth of a carrier network 

Due to the fact that mobile devices are connected to the internet by third-party carriers, the latency and bandwidth of the networks can vary a lot. Keeping the carrier network differences in mind allows the developing team to optimize the app’s performance and improve user experience. Consequently, you also want to test how the system behaves when network connections change while the app is being used. Those are scenarios that are difficult to simulate.  

We hope you got a rough understanding of performance testing of mobile apps after reading this blog article. Greenliff is happy to guide you through this process with all the in-depth expertise that our QA experts have. Don’t hesitate to get in touch with us.