Reduce Risk in Your Database Migration

What comes to your mind when you think of feature flags? If you’ve heard of them, you probably think of them in the context of turning on and off new features. But feature flags have back-end use cases too – including but not limited to reducing risk in database migrations. In this post, we’ll explain how.

What is a Feature Flag?

In case you haven’t heard of feature flags before, here’s the quick definition: a feature flag (also a “feature toggle”) is a piece of conditional code that separates code deployment from release by toggling new code on or off. Their implementation can be simple, with some boolean variables and a configuration file, or more comprehensive, with a complete feature flagging interface to promote cross-team feature flag management.

Use cases for feature flags in software development include changing the layout of a UI, promoting continuous delivery, creating canary releases, and safely releasing new features. They make developers’ lives easier by removing the merge problems from long-lived feature branches, they make other teams’ lives easier by lessening the development effort required to do feature experimentation, and finally, they make users’ lives easier by helping to improve the quality of the software they use.

Using Feature Flags for Database Migrations

One disturbingly common way of doing database migrations is what is sometimes called a “forklift upgrade”: everything is built up-front, the resources are invested to create the new database, and then all at once, a master switch flips all the users to the new version. If something is broken, or not migrated appropriately, one bad outcome is that everything crashes. But this is, in fact, far from the worst possibility. If everything crashes, no new user data can be written, so if you have a good backup, this can be recovered from without too much difficulty.

However, imagine a scenario in which most things work out, but one key functionality is missing; for example, imagine a working ecommerce site where for some reason, receipt data isn’t being written. In that case, the development team is caught between a rock and a hard place: a rollback would require aborting, and refunding customers for, all the transactions that have happened since the upgrade, and rolling forward would require scrambling to manually replace the missing data while also fixing the problem.

In short, forklift upgrades are extremely risky. There are many ways to mitigate this risk; among the most effective, feature flags.

There are five steps for using feature flags to migrate your database. They are:

  1. Canary release writes to the new database
  2. Fully release writes
  3. Canary release reads from the new database
  4. Fully release reads
  5. Remove all feature flags from your codebase

Let’s go through these in slightly more detail.

Canary Release Writes to the New Database

Canary releasing is the process of rolling out a functionality or new feature to a small subset of your userbase – between 1% and 10% as a rule – to see how it’s working. In the first step of the feature flag database migration process, we roll out writes to the new database for that subset of users, using a feature flag wrapped around whatever part of the code writes to the database. There should also be a feature flag wrapped around the part of the code that reads from the database, but it should be turned off for now, so that the reads are still all coming from the old database.

After this is released, monitor your user data and your error logs, and ensure that everything is running smoothly. If it isn’t, rolling back just the write functionality will be a snap. If it is, move along with a full rollout.

Fully Release Writes

Assuming everything has gone well with the canary release of writes to your new database, roll out writing to the new database for all users. It can be done all at once, or gradually with increasing percentages over time. At the end of this step, all new data will be written to both databases, but reads will still be coming from the old database exclusively.

Repeat to Roll Out Reads From the New Database

At this point we begin using the read toggle. Begin with a similarly small percentage, as before, and monitor it for both outright errors and performance decreases. If there are none, begin to roll out reads from the new database for all users.

Remove all Feature Flags from Your Codebase

This final step is critical to ensure that feature flagging has not increased your organization’s technical debt. Old feature flags, if not removed, can pile up and make for a cluttered, nigh-unreadable codebase. Once your database migration is complete, make sure that all feature flag code, references to the old database, and the database itself (once any still-needed older records are migrated over, a simple process of inserting) are deleted.

The Best Way to Migrate a Database with Feature Flags

In order to make your database migration as smooth as possible, the functionality for creating the feature flags, monitoring the impact of new stages of the process, and removing unused flags from the code should all be available in one place, with a single UI. Split can make this happen. With SDKs available in all your favorite languages, our comprehensive feature flag management platform can help you make the most of your feature flags.