Lessons Learned on Automation Episode II: Slow and Steady Wins the Race

In a previous blog post about automation, I explained that End-to-End (E2E) tests are inherently slow and that you need to focus on parallelism if you want throughput. With that in mind, you should strive to build your tests as reliably as possible in all possible environments; just because it works on your local machine does not mean that it will work consistently in your staging environment or when the tests are run on SauceLabs.

That leads us to the second lesson of the saga:

Slow and steady wins the race

In the Selenium world, many intermittent failures are caused by elements not showing up in time or not being enabled when the test tries to access it. In general, this is because the element has an unexpected transient state when the action is being performed.

The quick and dirty solution is to wait for an arbitrary amount of time, hoping that the element has the final expected state. This is such a terrible idea that I plan to dedicate an entire blog post to it in the future. A common scenario would be:

private final AugmentedDriver driver;

public void muchBetter(By identifier) throws InterruptedException {
    driver
        .augmented()
        .findElementClickable(identifier)
        .click();
}

1. What if the app was slow and the element was not there yet? The test fails.
2. What if the element was there but was not enabled, so it was still not clickable? The test fails.
3. What if the element was part of an animation and it was moving while you tried to click it? The test fails.

Explicit Waits

To address these issues, Selenium introduced the concept of Explicit Waits, which allows you to wait until the element has a particular state (“clickable” or “visible” for example). In AugmentedDriver, the driver has some extra functionality bundled in the augmented() library that helps you in those cases.

The previous example could be written:

private final WebDriver driver;

public void ohWhy(By identifier) throws InterruptedException {
    Thread.sleep(3000);
    WebElement element = driver.findElement(identifier);
    element.click();
}

With this approach, the click() will be performed only when the element is actually clickable. Augmented() has many other functionalities, such as findElementNotMoving(by) that waits until the animation has finished and the element is static, or findElementContain(by, text) that will wait until the element contains some text (Here is the interface where all that handy functionality is defined for Web and Mobile, here is the extra functionality that applies only to IOS and Android).

Summary

Whenever you are writing E2E automation you should be defensively coding so the tests are resilient to some degree of performance variance that are typical on Web or Mobile Applications. Adding extra checks on your tests might slow down your suites, but it’s a small price to pay for reliability and at the end that will make you win the race.

Get Split Certified

Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.

Switch It On With Split

The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Switch on a free account today, schedule a demo, or contact us for further questions.