Lessons Learned on Automation Episode II: Slow and Steady Wins the Race

Fernando Martin on June 2, 2016

Throughput in UI Automation comes through parallelism. If you can accomplish that, then your next goal is to make each test as reliable as possible.

In a previous blog post about automation, I explained that End-to-End (E2E) tests are inherently slow and that you need to focus on parallelism if you want throughput. With that in mind, you should strive to build your tests as reliably as possible in all possible environments; just because it works on your local machine does not mean that it will work consistently in your staging environment or when the tests are run on SauceLabs.

That leads us to the second lesson of the saga:

Slow and steady wins the race.

In the Selenium world, many intermittent failures are caused by elements not showing up in time or not being enabled when the test tries to access it. In general, this is because the element has an unexpected transient state when the action is being performed.

The quick and dirty solution is to wait for an arbitrary amount of time, hoping that the element has the final expected state. This is such a terrible idea that I plan to dedicate an entire blog post to it in the future.
A common scenario would be:

In this scenario, the test is waiting for 3 seconds, hoping that the element will be there by that time. The test then searches for the element and, finally, performs a click. This approach fails in several scenarios:

What if the app was slow and the element was not there yet? The test fails.
What if the element was there but was not enabled, so it was still not clickable? The test fails.
What if the element was part of an animation and it was moving while you tried to click it? The test fails.

To address these issues, Selenium introduced the concept of Explicit Waits, which allows you to wait until the element has a particular state (“clickable” or “visible” for example). In AugmentedDriver, the driver has some extra functionality bundled in the augmented() library that helps you in those cases.

The previous example could be written:

With this approach, the click() will be performed only when the element is actually clickable. Augmented() has many other functionalities, such as findElementNotMoving(by) that waits until the animation has finished and the element is static, or findElementContain(by, text) that will wait until the element contains some text (Here is the interface where all that handy functionality is defined for Web and Mobile, here is the extra functionality that applies only to IOS and Android).

In Summary, whenever you are writing E2E automation you should be defensively coding so the tests are resilient to some degree of performance variance that are typical on Web or Mobile Applications. Adding extra checks on your tests might slow down your suites, but it’s a small price to pay for reliability and at the end that will make you win the race.

We're Hiring!

Join the Team