End to End (E2E) WebDriver Testing is a two-edged sword; it can be the hero, saving you a lot of time during regression testing and preventing faulty releases, or it can be the villain, slowing down the development and release process with flakey, hard to maintain and time-consuming suites that you don’t trust or want. I have seen both.
In these series of blog posts, I will share my experience developing
AugmentedDriver, an open source framework that allowed my team to run 80,000 tests per month and more than 275 tests in parallel in less than 15 minutes.
Data Independent Tests
Selenium tests are inherently slow. If you want throughput, you need to focus on parallelism, leading us to the first lesson in this saga:
If each test creates its own data that is not shared with anyone else, then all of your tests can run in parallel.
With parallelism, your throughput will no longer be constrained to the length of the test. Instead, it will rely on how many tests you can run at the same time. I personally like using SauceLabs, but there are other commercial options to help with that.
You can always play with your own Selenium Farm, but I caution against it because it has been a nightmare for me to maintain.
Here is a simple example of how to achieve this in a rest-based application using Java + JUnit:
First I create a TestResource with an endpoint that cannot be executed in production. This creates and cleans the basic data that every test needs.
Then I create a JUnit
TestRule that calls those resource endpoints:
With that rule in place, each test creates its own data at the beginning. If the test does not fail, it also cleans afterward.
Achieving Data Independence is the foundation on which a Selenium framework is built. You should strive to not relying on existing data (except when that data is immutable). Following that simple rule will avoid nasty situations when data gets corrupt, or worse—when one test modifies data that another test needs (leading to one test depending on another test—yikes!). Finally, it will also open the gates for parallelism, significantly decreasing the time spent on the testing phase.