Background on Skillshare:
Skillshare is an online community for creatives with a subscription business model accessible via desktop, mobile web, and apps. They offer various classes with free trials available.
Experimentation at Skillshare:
The experimentation process at Skillshare is detailed, highlighting the 12-stage experiment lifecycle and the importance of a thorough checklist. Their initial process lacked experimentation culture. By partnering with Split they were able to implement experiments and created a 38-point checklist for experiment execution. For example, they created a sticky mobile web banner experiment that identified the need for a mobile web banner similar to desktop.
Developed hypothesis:
Their hypothesis behind implementing a sticky mobile web banner will drive signups and subscriptions. They launched the experiment with control and treatment groups. They look at the stages of launching the experiment, analyzing the results, and addressing unexpected findings. Despite initially promising results, further investigation reveals issues with traffic distribution and negative user experiences.
Investigated potential causes:
Their team decides to iterate rather than celebrate or obliterate, because of chance, random number generation, or code bugs, ending the experiment and defaulting to the control due to upcoming website rebranding. Subsequent experiments are conducted on different user flows, with varying outcomes—some successful, others not. They ended up running three separate experiments for different user flows.
Results varied:
They emphasize the importance of experimentation in detecting and addressing issues that may not be apparent otherwise. They highlight the role of experimentation in improving user experiences and business outcomes while stressing the need for a strong team culture, trust, and rigorous experimentation processes. The significance of rigorous experimentation processes and a collaborative team culture is key.
Switch It On With Split
The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Schedule a demo or explore our feature flag solution at your own pace to learn more.