As mentioned in an earlier post, UI tests are a great way to make sure your app meets essential functions and the build has no obvious regressions. These tests make sure that the app is still intact by interacting with visible elements on the page in the way a user would.

There are a few drawbacks to UI automation, though. Long run times, difficult debugging, and flaky tests impede some of the efficiency gains automation is intended to achieve. Even though these characteristics come with the territory of UI testing, there are ways of mitigating some of these risks and getting the most out of UI tests. divider 600

Need for speed

UI tests naturally take a long time to run because they are performing end-to-end testing. These tests are dependent on both the time it takes for data to be returned and the speed at which the device can render the elements on the screen. There is also limited visibility into the app so there is a noticeable delay with processing the automated commands.

You can work around sluggish test runs by prioritizing and grouping your tests and getting a quicker feedback loop from failed test runs.

Group tests efficiently

Just because you have a lot of tests doesn’t mean you need to run all of them all the time. Running a suite of automated UI tests can take a long time and the team shouldn’t have to wait to see if it passes every single test.

Instead, group tests so you can have a set of smoke tests that run automatically and a set of regression tests that run nightly.

Fail fast

Take advantage of test frameworks that allow you to skip tests after initial failure is reported. Prioritizing the most critical tests will save time by stopping the test run as soon as the app fails one of the most basic checks.

For instance: if the user can’t sign in, then there is no need to test other features that an authenticated user can do because those will eventually fail as well.

@Test (groups = UNAUTHENTICATED)
    public void testUserLogin() {
        SavedPage homePage = new BasePage(this)
 	    .login();

        Assert.assertTrue(homePage.contentDisplayed());
    }


@Test  (groups = AUTHENTICATED, dependsonGroups = UNAUTHENTICATED)
    public void testUserSaveArticle() {
        SavedPage savedPage = new BasePage(this)
 	    .login()
                .selectMenu()
                .selectSaved();

        Assert.assertTrue(savedPage.contentDisplayed());
    }

divider 600

Making sure you’re getting reliable results

Flaky tests are tests that occasionally fail and are often a talking point in UI automation. The reasons for failure aren’t always clear and investigation is often required to determine if the test is actually failing or not.

It’s important to stay vigilant and keep these flaky tests at bay because they can reduce the reliability of the test results. Fluctuating test results can occur for a variety of reasons, but there are some tried and true ways of hardening an automated test and give more confidence in its result.

Implement waits

Timing is huge in a test run. If a test types or swipes at the wrong time, then it could be taken down a wrong path. By implementing “waits”, you can be sure that the test waits until the UI element is ready to be interacted with. Always make sure that the app is in the correct state before proceeding. It’s especially important to be patient in areas that are dependent on external services and data.

Be efficient about reruns

Lots of test frameworks support a feature to rerun failed tests. Any test that fails two or three times in a row is a test that needs immediate attention and is either a poorly written test or represents a bug in the app. This can be a good way to build credibility in your test results, but be sure to make note of any tests that are still inconsistent as they will increase run time significantly.

Standardize your test environment

Automated testing needs a stable and static environment to run on. It’s hard to verify that the user is on the right page if the content on the page is constantly changing. It can be tempting to work around a changing environment and write tests to be flexible and handle unexpected data, but this can lead to costly test maintenance.

By testing against the same content each run, you can be sure that the test is failing because of changes made in the app.

Use element IDs

Make sure to work with developers to get IDs for page elements. Any element that the test uses should have an associated ID. This will help get your tests more precise and speedy when trying to navigate the app. Using lengthy Xpaths can take a while for tests to identify and will cause tests to break with the slightest change to the app’s code. divider 600

Making sense of test results

One of the biggest headaches with UI tests is debugging a failed test. Unless you are using a native automation framework, like Espresso or XCTest, then the test will not be able to report much on the defect in the app’s code. But there are still tools available to identify defects when performing black box testing.

Get good screenshots

Appium supports screenshot captures and they are going to be your biggest clue as to why things went wrong in your tests. It is always a good idea to take screenshots at the end of a failed test, but it can be just as useful to take screenshots at the end of all of your tests. Capturing the finished state on all of your tests helps verify that the test ran successfully and there weren’t any false positives in your run.

Don’t skimp on logs

Logging as much information as possible will help you step through the failed tests and determine where it all went wrong. By logging areas that the test didn’t anticipate, you can figure out if the test is behaving as expected, or why a test is generating unexpected results. Capturing information not directly related to the test, like device logs and network traffic, is also important and will provide beneficial information for a bug report after the test discovers a defect. divider 600 UI tests are an essential sanity check to make sure that your app isn’t on fire. If these tests can’t be trusted or are taking incredibly long, they begin to lost their value. Implementing these tips can go along way to help gain confidence in your tests and your builds as well! divider 600