Meet with Split at AWS re:Invent! Book Now


Fixed Horizon

Fixed Horizon is a statistical testing method where sample sizes and your experiment goals are defined in advance.

With fixed horizon, you can do a single statistical test and draw conclusions only once your review period has completed. Since sample sizes are defined in advance, this is the perfect testing method when you’re looking to make small improvements to a relatively low traffic site or page.

Why Implement Fixed Horizon?

Fixed Horizon is perfect for those teams that are looking to make tiny optimization “tweaks” within their product. Results are only valid once your experiment has reached statistical significance and your review period has been met. This enables the ability to track even the slightest movement in your data (i.e. 0.10% increase in your page load times).

Fixed Horizon vs. Sequential Testing

With Sequential Testing, results are always valid the minute you launch an experiment. However, since you don’t need to define your sample size in advance, this method is optimal when you’re trying to detect big notable changes in your data. With Fixed Horizons, you have to wait for your experiment review period to complete before peeking at your data to avoid interpreting any false positives. However, since you define your sample size in advance, this method is great when making slight tweaks within your KPIs. If you were to use the Sequential Testing method when making small improvements, it would take you a longer time to reach a conclusion in comparison to the Fixed Horizon approach. Both methods should be used interchangeably, depending on your team’s needs. Learn more about setting up Fixed Horizons here.

Want to Dive Deeper?

We have a lot to explore that can help you understand feature flags. Learn more about benefits, use cases, and real world applications that you can try.

Create Impact With Everything You Build

We’re excited to accompany you on your journey as you build faster, release safer, and launch impactful products.