Beta + Load Testing for Mobile Applications

Unit, Beta, and Load Testing your Mobile App

Testing is vital in developing a successful mobile application, but testing takes many different forms. At Cadence Labs, we follow industry-defined standards and best practices that allow us to rapidly develop applications that look great and work well. Testing starts with driving your development process through unit tests and often ends with proper load testing of your production servers. For mobile apps in particular, there are a few special nuances to opening up your app to beta testers besides just giving them the URL like you would for a web application.

On Device Beta Testing for Android and iOS Apps

Both the Google Play and App Store have their own processes for distributing a mobile app for beta testing on their services. It is important to note that this is the last step in testing any application before putting it in the App Store or Google Play store for release. This means that if you’ve still got some failing unit tests or if your app is crashing, it is not ready for beta testing on either platform. First, let’s take a look at how Android (Google Play) handles alpha and beta testing.

Android

Android apps are distributed through the Google Play store and this is where app developers can choose groups of beta testers for their app. After uploading a test APK file, developers will be able to choose the type of testing they would like to do:

• Closed beta – only users that are invited through the Google Play console.
  • Note: These users must have Gmail or Google Accounts email addresses.
• Open beta – Google Play generates a link that can be shared anywhere on the web so that anyone can download the test app and start playing around.

iOS

While all iOS apps are distributed through the Apple App Store, beta testing is done through a different application called TestFlight. To get your application in to TestFlight, follow the steps on our guide (link to approval guide). Once the application is in TestFlight, you will be able to invite users to the application via email address. Alpha (truly internal testing) is limited to just 25 invites, while beta (truly external testing) allows up to 10,000 invites.

When you invite a user to alpha or beta test your application in TestFlight, they will be sent a link with a redemption code that they must enter in the TestFlight mobile app on their device. Note: invited users must download the TestFlight app from the App Store, only available for iOS devices.

Other Testing and Deployment Tools

While both platforms come with IDEs and developer tools to help push your app from development to testing to production, there are also third-party tools that can drastically reduce the time it takes to move through common tasks such as uploading a beta test build to TestFlight.

Fastlane

Fastlane is a great tool in that it can help you greatly simplify your Android and iOS deployment tool. With features such as:

• Screenshot Automation – generate screenshots for the app store
• Beta Deployment – build and deploy beta builds to testers
• App Store Deployment – publish a new release almost instantly
• Code Signing

And, with integrations with all top CI platforms, you can further automate your mobile application development with the help of a DevOps professional on our team here at Cadence.

Expo

Expo.io has been a revolutionary step in mobile app development with React Native. Now all you need to do to test a React Native app built on Expo is the Expo app on your phone. Send the user the QR code to open in the app and BAM, they are instantly transported to the latest version of the code that your developer is working on. Allowing your team to get instant feedback on new features! 

At Cadence Labs, we understand the importance of delivering high quality, bug-free software, and that is why we pride ourselves on the tools we use to battle-test our applications. We are always looking for new ways to improve our testing workflow and look forward to sharing these improvements with our clients.

Load Testing

When load testing a mobile app, it is very important to take into account some nuances in the difference of experience that users expect when using a mobile app versus a web app. Overall, users tend to expect a much snappier and faster experience when using a mobile app. At the same time, mobile connection speeds can vary greatly which can cause large gaps between the performance expected and the performance delivered given a bad connection. With these facts in mind, we like to use load impact testing as it can simulate different mobile networks and allows for testing of your app’s backend endpoints with a variable number of users.

At Cadence Labs, our preferred load testing tool is LoadImpact.

It is important to note that when load testing an mobile app, the only component that needs to scale is the backend integration that receives network requests (e.g., your server).