Experimentation

We had an amazing time at this year’s AWS re:Invent in Vegas. We were able to chat with more than 2,000 developers, operators, and cloud architects that stopped by. A few observations: Feature flags were a hot topic. We chatted with quite a few organizations that are already getting value…

Great teams don’t run experiments to prove they are right; they run them to answer questions. Guard against wishful thinking and hidden biases with this shortlist of core principles for productive online controlled experiments. (Video, transcript and screenshots from talk given at Pinterest HQ on September 13, 2019)

Here’s the video of my talk, “Experimentation for Speed, Safety & Learning in CD” delivered at QCon New York 2019. I’ve edited it down to 39 minutes, so you can easily fit this into a lunch hour or possibly a commute. Text and screenshots follow the video below. Transcript and…

Feature launches in leading engineering teams increasingly look like a ramp rather than a one time switch, going through dogfooding, debugging, max power ramp, scalability and learning phases.

In this post, we will talk about key experimentation concepts including how to choose your Overall Evaluation Criteria (OEC) for your experiments and how to increase the sensitivity of those metrics through metric filtering and metric capping.

Learn how experimentation platforms help development teams test ideas directly with customers and enable continuous delivery with lower risk.

The effects of burnout can be significant for any software development organization, large or small. In this blog post, we discuss how burnout can be alleviated with an agile workflow powered by feature flags, a robust feature experimentation platform and incremental feature rollouts using continuous delivery.

We built Split’s feature experimentation platform with this fundamental assumption: the data you capture to measure and understand your customer experience is collected across many touch points, and any tool you use to release feature flags and measure impact must be able to capture data from all of them.

As adoption of feature flags has spread, product teams have learned to release new features incrementally, exposing new functionality slowly to the user base with a careful eye towards ensuring the right customer experience. Feature flags have made automating the process of measuring product metrics much easier.