Software QA at WillowTree is not the kind of software testing that you might be used to. We put our code through the ringer to ensure exceptional native app quality for millions of users and to survive the harsh mobile world. Our goal as WillowTree QA is to maintain the quality consumers come to expect from some of our industry leading clients, while also providing the best mobile user experience. Testing for mobile users comes with its own set of unique challenges that sets it apart from traditional software QA - a reality that every tester should consider.
Mobile users encounter a wide variety of edge case scenarios simply because their connection to a stable network is not guaranteed. With the shift from desktop to mobile digital consumption, it is becoming more and more important to test for users in all environments. Users travel between different signal strengths at any given time and if a device loses network connection during a critical transaction it can send the user into an invalid state and risk losing important information. Testing network conditions is something that should be considered from the early phases of a project when UX is designing user flows. Plan on users hitting all of the different error states in the app, because they will.
Mobile testing in today’s world requires more than simply testing how mobile apps run on the latest and greatest devices. Why? Because of the reality and pain of mobile platform fragmentation. Whether it’s something as simple as a pesky performance lag or as dramatic as your new device catching on fire, the concept of fragmentation is one of the most unique challenges the mobile industry faces; especially for Android users. The largest base of Android users are on Lollipop, so mobile test approaches need to revolve around users on older OS versions and devices. Newer versions generally implement improvements with security, performance, and adaptation to newer hardware. The impact of these implementations on older devices can be as minor as a change in performance or as major as an inability to open and run an application. One of the most challenging topics on this issue is conceptualizing how older OS versions and devices will interact with our future-forward approach and innovations with regard to user experience.
This is where the plot thickens in mobile app development-- finding that sweet spot between taking advantage of the latest and greatest OS and hardware features, while still preserving functionality on older devices and OS versions. To find that sweet spot, QA needs to not only test on a wide range of devices, but also emphasize testing on the most problematic devices. A benefit with this approach is that blatant problems that are fixed for the sake of supporting older devices often result in a global improvement and consistency with all devices and OS versions.
Automated testing has been an essential tool for web applications, so it is no surprise that frameworks have emerged for mobile apps as well. iOS and Android both support native unit testing, allowing developers to write tests with access to the project’s codebase and ensure lower level functionality is still working as intended. Testing for different user flows using the apps interface, however, becomes trickier when keeping both platforms in mind. Despite having relatively similar UIs, Android and iOS have different ways of locating elements on the page and different levels of access to the build itself.
Automating UI tests has its own native solution, but doing so can arguably take QA testers more time to develop and support separately for QA. Fortunately, Appium provides a well documented framework that allows tests to be written for both platforms using the same language. When writing UI tests for a project, it is beneficial to collaborate with developers on UI elements from the beginning of feature development. Planning for UI testing can help with element naming conventions for both platforms and ensures your tests will be able to find and interact with all the areas of the app that need to be tested.