Big News! Split to be acquired by Harness. Read More

Reducing Risk by Testing Every Change in Production

Split - Blog-New_Testing-Every-Change-Where-It-Matters

Webinar Summary Highlights

Feature flags are on/off switches in code that control different versions of a feature. Split is a feature flagging platform that enables remote control and targeting rules for feature flags. Staging environments differ from production environments in terms of setup, configurations, user behaviors, and monitoring. Pre-production testing is not enough because production is the first time a system is truly tested by users. Testing in production is inevitable and crucial for understanding real user behaviors and system integrations.

Production testing involves deployment, release, and post-release testing. Different testing methods in production include integration tests, tap compare tests, load tests, shadowing, configuration tests, and dark launches. Various techniques like canarying can be used during the release process. Continuously testing production systems is important to identify and address failures over time. Testing in production allows deliberate testing in a real environment

Alpha stage is when people outside the team start testing and interacting with the feature. Beta phase involves real customers who opt in to test features that are not fully developed. Gradual rollout from 5% to 50% of users helps identify and debug issues. Feature flags and control rollout enable slow and controlled deployment. Dark launches involve deploying new features without enabling them for customers. Canary releases gradually roll out features to assess system reactions. Configuration testing allows testing different combinations of software and hardware. Tap compare test compares results of new and old versions to detect changes and issues. Monitoring and alerting tools help identify anomalies and their impact on features. Experimentation measures the impact of changes and determines their effectiveness. Guardrails, including business and operational metrics, ensure changes don’t have negative impacts.

  • The architecture of the software allows for increased security and privacy protection by not capturing personally identifiable information (PII) during flag evaluation.
  • The software enables targeting based on specific attributes within an app without sharing sensitive information with the provider.
  • Guardrails, such as monitoring throughput, errors, and latency, are important metrics to track for every change in production.
  • Additional technical metrics may vary depending on the business and implementation, such as crash rates for mobile apps or memory and CPU utilization for other systems.
  • Business-related metrics include tracking user engagement, revenue impact, and the impact on active users.
  • Split, the software being discussed, provides various methods to feed data into the system, including emitting track events from the SDK, using REST APIs, or integrating with other analytics tools and systems.
  • Split allows for the creation of custom metrics based on specific events or behaviors to track the desired outcomes of feature changes.
  • Split automatically tracks all metrics and helps identify unexpected anomalies that may impact different parts of the company or product.
  • The software’s ability to identify and address production errors quickly and accurately was demonstrated through a customer example involving a major bank rolling out a new feature in controlled stages, monitoring metrics, and promptly addressing issues.
  • The decision to kill or roll back a change depends on the severity of the problem and the impact on users, revenue, and other factors.
  • In experimentation, statistical techniques and monitoring enable users to trust the data and make informed decisions about reverting or addressing issues in a timely manner.

Get Split Certified

Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.

Switch It On With Split

The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. 

Switch on a free account today, schedule a demo to learn more, or contact us for further questions and support.

Want to Dive Deeper?

We have a lot to explore that can help you understand feature flags. Learn more about benefits, use cases, and real world applications that you can try.

Create Impact With Everything You Build

We’re excited to accompany you on your journey as you build faster, release safer, and launch impactful products.