In this world of continuous delivery and high volume of feature releases, it’s important to have a testing infrastructure in place to ensure your existing functionality does not break because of current changes. Smoke tests are a type of regression test that makes sure that your critical functional flows work. These smoke tests should not encompass your entire testing suite, rather they are a subset of tests that cover your top prioritized user flows that run in your continuous delivery pipeline.
Smoke Test Your Pipes?
The term “smoke testing” comes from plumbing, if you can believe it. In plumbing, smoke is created and fed into pipes to ensure there are no leaks before they are installed. If we apply this to software development, smoke testing makes sure that the main functionality of your software is properly working so that you have a solid foundation for your full test suite.
Smoke Tests Give you Fast Feedback
Smoke tests should be put in place to ensure that your new code has not broken any existing functionality, and ideally, they should be run in production. In software testing, you don’t want to be reactive – you don’t want to wait until a feature breaks, have a user report it to you, and then push a fix. You want to know if something is wrong before your users ever experience anything wrong. As a developer, smoke testing gives you that confidence you need to know you are releasing new features without breaking existing functionality, and they give you this feedback fast so that you can make changes quickly when necessary.
What Should You Smoke Test?
The best way to figure out which tests to implement in your smoke testing suite is two-fold. The first is to talk to your product owner to understand the user flows with the most critical business value. Then, talk to your data analyst to get a list of the flows that users perform the most in your system. Between these two lists, you should have a good idea of what to test, and what is the highest priority. For example, if you are a developer for a retail application, and the most important business flow is that the checkout works, you should definitely add that to your smoke test suite. On the same application, if you look at the data and you see that people like to zoom in on the items and that functionality gets the most clicks on the site, you also want to add that to your smoke suite as well.
Write Your Tests
Smoke tests should be a combination of functional end-to-end tests, unit tests, integration tests, and API tests. Once you get the data you need from your product owner and data analyst, you can categorize the tests into one of these buckets.
Your functional end-to-end tests should cover user flows that are high level. Unit tests should cover one specific component at a time, where integration tests should test two components working together. Smoke testing your APIs should be a quick call to the API and validating its response. It’s also important that any tools you use for writing your tests can easily be integrated into your build pipeline.
Insert the Smoke Tests into Your Pipeline
Once you have your tests, you need to insert them into your pipeline. The easiest way to do this is to create a new `run` step in your YAML file. This can be done easily with any CI/CD solution, such as CircleCI, Travis, Jenkins, or Maven.
Smoke Tests Should Be Blocking
Smoke tests should be blocking, which means if any of them fail in a pull request, they will block the pull request from being merged. This blocking aspect of smoke tests is important because if the code you’re trying to push breaks a critical flow, you either need to update the test if the requirements have changed, or fix your code to ensure proper functionality of the feature. This is by far the most important aspect of smoke testing.
I cannot stress this enough – whether you need to fix a selector, or the logic of the test, or the functional code itself, figure out what’s going on and fix it!
Learn More about Testing and the CI/CD Pipeline
Smoke testing is an essential part of having a complete CI/CD pipeline. Without them, you are just expecting your software to work without any validation beforehand.
If you’re interested in learning more about testing in your CI/CD pipeline, check out these posts:
Stay up to date
Don’t miss out! Subscribe to our digest to get the latest about feature flags, continuous delivery, experimentation, and more.
Build a simple Spring Boot web app and see how you can easily use feature flags to change functionality.
There comes a time in every developer’s life when they need to do a database migration. Because business requirements change frequently, database schemas need to be updated from time to time. When you think of making changes to your code, it’s easy to roll back with git or feature flags,…
To understand how businesses can begin using feature flags, let’s look at a typical case of an organization evolving a continuous delivery system using feature flagging from the ground up. By analyzing a system taking shape, we can identify some of the common mistakes that software development teams make and…