O'Reilly eBook
Understanding Experimentation Platforms
Drive Smarter Product Decisions Through Online Controlled Product Experiments
A software developer’s playbook for mitigating unintended consequences in financial services, and how feature flags can help.

Learn Why Agile Businesses Use Split
Deliver software features that matter, fast!
renderContent(deleteTreatment) { const allowDelete = deleteTreatment.treatment === "on"; return ( <div className="todoListMain"> <div className="header"> <form onSubmit={this.addItem}> <input ref={(a) => (this._inputElement = a)} placeholder="Enter Task" ></input> <button type="submit">Add</button> </form> </div> <TodoItems entries={this.state.items} allowDelete={allowDelete} delete={this.deleteItem} /> <TodoItems entries={this.state.items} allowDelete={allowDelete} delete={this.deleteItem} /> </div> ); }
Building a Product Experimentation Platform
Product experimentation platforms consist of a robust targeting engine, a telemetry system, a statistics engine, and a management console.
- The targeting engine is responsible for dividing users across variants.
- Telemetry is the automatic capture of user interactions within the system.
- A statistics engine determines what feature caused a change in your metrics.
- The management console is where experiments are configured, metrics are created, and results of the statistics engine are consumed and visualized.

DevOps Best Practices
Whether you’re merging conflicts, managing silos, or working on the weekend, Engineering Managers can make life better for their teams.
Running A/A Tests
In an A/A Test, both the treatment and control variants are served the same feature, confirming that the engine is statistically fair and that the implementation of the targeting and telemetry systems are unbiased.Understanding Power Dynamics
Power Dynamics measures an experiment’s ability to detect an effect when there is an effect there to be detected.Executing an Optimal DevOps Ramp Strategy
During a ramp up process, taking too many steps or taking too long at any step can slow down innovation. But taking big jumps or not spending enough time at each step can lead to suboptimal outcomes.Building Alerts and Automation
By building in metrics thresholds, you can set limits within which the experimentation platform will detect anomalies and alert key stakeholders, not only identifying issues but attributing them to their source.
Designing Metrics
Four types of metrics are important to experiments:
- Overall Evaluation Criteria measures business value or user satisfaction
- Feature Metrics are important to the team in charge of the experiment
- Guardrail Metrics should be directional and sensitive
- Debugging Metrics must be sensitive but need not be directional
Schedule a Split Demo Tailored to Your Needs
Speed up development cycles, reduce release risk, and focus your team on DevOps best practices that create maximum impact.
Create Impact With Everything You Build
We’re excited to accompany you on your journey as you build faster, release safer, and launch impactful products.