GoodRx

Reducing risk and driving innovation for millions of users on the GoodRx platform

The GoodRx website and mobile application leverages their real-time market-intelligence platform to gather retail prices and coupons for every FDA-approved medication at more than 70,000 pharmacies across the US, providing savings of up to 80%. More than 10 million consumers use GoodRx each month.

Challenge

Several of the GoodRx applications had the user segmentation logic in the code base, and as a result required a full deploy to execute feature flags, experiments, or feature kills. Other applications used an in-house database-driven feature flagging system, but extending this system to support increasing scale and scope was outside their expertise and siphoned resources away from delivering customer value.

Solution

The Split SDK sits within the application so the decision engine is local, providing a fast and reliable platform to handle feature flagging and experimentation logic. GoodRx uses Split for frontend A/B testing that focuses on focus on conversions and user interaction, backend testing and migrations, and feature iteration.

Results

  • 95% reduction in execution time by eliminating the need for a full deploy of their mobile web application at each phase of a rollout
  • Immediately kill problematic features without manual intervention or code deploy
  • Safely migrate backend systems without impacting user experience

The Situation

GoodRx had been using a homegrown feature flag solution for their desktop application, leveraging a database to track feature variants for phased rollouts and A/B testing of the user experience. The in-code solution segmented customers based on user session IDs, and an admin control panel provided a central location to kill a rollout or experiment.

Applications that weren’t using the in-house feature flag solution had the segmentation logic in the code base, which meant every change in the percent allocation for a phased rollout required a full deploy of the site. With every deploy of their mobile web app for example, QA resources were also necessary to ensure functionality and stability of the segmentation logic.

Another challenge of their in-house system was performance. The database memory cache could sit for several hours before being pushed live. To propagate or kill a flag immediately required manual intervention by an engineer.

Their in-house solution worked well enough until a growing number of application teams also wanted to conduct more sophisticated user tests, which increased the volume of flags in play and started to stress the system. There was no centralized source of truth that tracked which features were flagged and why, making monitoring and clean-up of old flags increasingly tricky. In addition, they wanted to apply robust targeting rules to their expanding experimentation practice, but continuing to add feature flagging logic into the code was becoming a burden to engineering and had already proved inefficient.

Expanding the in-house system to support the full application stack required significant infrastructure considerations to build out a framework in-house. For example, the engineering team wanted to implement feature flags for backend system testing and migrations but there was no API exposed for easy expansion. Their mobile web React application would have also required more work on the in-house system, making it difficult to achieve a unified feature flag and experimentation pipeline.

Choosing Split

Extending the in-house system to support increasing scale and scope was outside their expertise and siphoned resources away from delivering customer value. Continuing to build in-house was becoming a burden, and the engineering team appreciated the opportunity to offload the non-core application to a commercial solution. Several vital capabilities drove the decision to choose Split.

Sophisticated user segmentation

The GoodRx team wanted to implement more sophisticated product experiments, and they liked Split’s advanced segmentation capabilities such as assigning custom attributes for more granular user targeting. For example, GoodRx can segment users based on dimensions such as the user’s city and state so they can serve up feature variants of an experiment based on user location or IP address. GoodRx also takes advantage of Split’s whitelist functionality to make internal testing of backend systems much more manageable.

Platform performance and stability

Split provides the ability to change treatment rules or initiate a feature kill within 60 seconds, without having to re-deploy their desktop or mobile application. Stability and performance was a concern with the in-house solution, as segmentation and rollout logic lived in the codebase. Now the teams rely on Split for user segmentation and phasing the rollout. The Split SDK sits within the application so the decision engine is local, providing a fast and reliable platform to handle feature flagging and experimentation logic.

“We no longer have to worry about the accuracy of user bucketing logic. We offload this work now. We trust Split is doing the right thing and the logic is correct, which increases the developer’s quality of life.”

Eric Wang, Director of Engineering, GoodRx

Using Split

Feature iteration

In addition to A/B testing, GoodRx increasingly leverages Split to support feature iteration. Split provides a platform for phasing rollouts, giving the team a way to gradually measure the impact of new features without risking exposure to the entire user base. If the minimum viable feature doesn’t test well with users, they will work to make it better, then execute another user test. The rapid feedback loop helps engineers refine functionality and speed up the iteration of ideas.

Supporting advanced customer analytics

GoodRx uses a data warehouse to run advanced customer analytics focused on front end changes to their desktop and mobile applications. With Split’s outgoing webhook, the team integrates impression data captured by Split into their existing data pipeline, providing insights on the impact of feature changes.

Backend migration

Backend changes for GoodRx have far-reaching impact to the user experience. Gating functionality and phasing out a rollout with Split provides runtime control to quickly revert to previous behavior if needed, without waiting for a long deploy cycle. When GoodRx initiated a cutover to a new URL shortening service, they put the functionality behind a flag. From here they monitored the phased rollout to ensure the new service could support their API call volumes.

“If we build it we have to test it to make sure it’s doing the right thing. When we conduct backend experiments, we have to know the system will remain reliable. We have to make sure the backend behaves in a way that doesn’t impact the user experience.”

Eric Wang, Director of Engineering, GoodRx

Future usage

The team at GoodRx found the Split user interface to be intuitive and straightforward, making it easy to get up and running quickly. As they realize the benefits of feature flags, they increasingly gate features behind a split. For example, the engineering team is working on integrating feature flags into development workflows. They plan to incrementally test and deploy smaller pieces of multi-faceted features, keeping them dark until full functionality is available. Separating code deploy from feature release helps engineers identify problems early to reduce risk and improve productivity.

“Split is a straightforward product that gets stuff done.”

Eric Wang, Director of Engineering, GoodRx