Join us for Flagship 2024: April 16-17 – Register Now.


Flagship 2021: A Sticky Mobile Web Banner – It Makes the Case for Experimentation

Put on your data detective hat as Jean Steiner delves into a series of sticky mobile web banner experiments that she ran at Skillshare. Through these experiments, we’ll share best practices and demonstrate why experimentation was critical.

Background on Skillshare:

Skillshare is an online community for creatives with a subscription business model accessible via desktop, mobile web, and apps. They offer various classes with free trials available.

Experimentation at Skillshare:

The experimentation process at Skillshare is detailed, highlighting the 12-stage experiment lifecycle and the importance of a thorough checklist. Their initial process lacked experimentation culture. By partnering with Split they were able to implement experiments and created a 38-point checklist for experiment execution. For example, they created a sticky mobile web banner experiment that identified the need for a mobile web banner similar to desktop.

Developed hypothesis:

Their hypothesis behind implementing a sticky mobile web banner will drive signups and subscriptions. They launched the experiment with control and treatment groups. They look at the stages of launching the experiment, analyzing the results, and addressing unexpected findings. Despite initially promising results, further investigation reveals issues with traffic distribution and negative user experiences.

Investigated potential causes: 

Their team decides to iterate rather than celebrate or obliterate, because of chance, random number generation, or code bugs, ending the experiment and defaulting to the control due to upcoming website rebranding. Subsequent experiments are conducted on different user flows, with varying outcomes—some successful, others not. They ended up running three separate experiments for different user flows.

Results varied: 

They emphasize the importance of experimentation in detecting and addressing issues that may not be apparent otherwise. They highlight the role of experimentation in improving user experiences and business outcomes while stressing the need for a strong team culture, trust, and rigorous experimentation processes. The significance of rigorous experimentation processes and a collaborative team culture is key.

Switch It On With Split

The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Schedule a demo or explore our feature flag solution at your own pace to learn more.

Want to Dive Deeper?

We have a lot to explore that can help you understand feature flags. Learn more about benefits, use cases, and real world applications that you can try.

Create Impact With Everything You Build

We’re excited to accompany you on your journey as you build faster, release safer, and launch impactful products.