thredUP is a massive web app with hundreds of thousands of visitors a day. They realized the potential benefits of controlled rollout early on: so much so that they built their own rudimentary feature targeting system to slowly roll out features. But it relied heavily on manually whitelisting targeted accounts and other userID-focused methods. Plus, with a large distributed system, changes to rollout plans wouldn’t take effect immediately. Instead, they’d move from one service to the next over the course of minutes, achieving the end-goal but causing hiccups in the continuity of customers’ experience along the way.
“What’s interesting is that we’re actually able to move faster with Split. In the past there were a lot of questions like ‘if something goes wrong how will we roll it back?’. Now we just put it behind Split, and then we have the flexibility to make changes on-the-fly once it’s in production.”
Dan DeMeyere, VP Engineering, thredUP
And when they do decide to make changes to a rollout, Split handles it much quicker than their old system:
“If we want to roll something out to 100% of our customers, that change is almost instantaneous. If we want to roll back, same thing.”
Dan DeMeyere, VP Engineering, thredUP
Executing Real-Time Deals on Black Friday
thredUP’s deep implementation of Split has allowed them to get creative, not only with how new features are delivered to customers, but with how their web app itself handles deal calculations. During Black Friday 2016, thredUP used Split in an innovative way—processing unique offer codes in real time from destination URLs to serve unique experiences to customers.
We used Split to control what customer promotions were being shown on the website and in emails. In years past we approached Black Friday with a new promotion every day: different on-site messaging and different email messaging. This year we used Split to determine what everyone should be seeing.
“You can pass attributes into Split for targeting, so we passed in some of our machine-learning scores, and then that would dictate what promotion a customer would see. You can also set the value that is returned to clients with Split, so in this case we returned not ‘on’ or ‘off’, but a promotion code. That way the clients could just dynamically handle what to show to the customers, and then we could also pass in the current date, and based on conditions within Split, for each day, naturally deliver new content and promotions.”
Dan DeMeyere, VP Engineering, thredUP
Full-Stack A/B Tests
Like nearly every e-commerce company, A/B tests are an essential part of they way thredUP does business. But after reaching the limits of what existing A/B services could do, the company turned to Split. By providing back-end a/b functionality out of the box and native SDKs in the backend languages thredUP uses (Ruby), Split was easy for the team to try and test. But what really set Split apart from the competition was its ability to decouple data and analytics gathering from serving features and treatments to users:
“On any given day we have hundreds of thousands of people coming to the website. That’s a lot of impression data. We needed to be able to determine what variation of a feature customers were going to get or when an anonymous person becomes a customer, and Split just takes care of all of that for you. Now our web apps just care about what customer is in what treatment for what feature, and all of the data can just be its own separate beast. Our data team can integrate with the Split webhook, get all the data they need, and don’t have to worry about tracking anything. With Split, that just happens for free.”
Dan DeMeyere, VP Engineering, thredUP
thredUP’s data team is able to continue their work in the analytics architecture they prefer (Redshift and Looker), and the thredUP app is able to quickly and efficiently serve feature variations to customers via locally-installed SDKs.
“Every time you visit thredUP.com, you’re fetching Split treatments.”
Dan DeMeyere, VP Engineering, thredUP
Sticking to Core Competencies
thredUP knew they wanted a singular platform to be able to handle feature delivery to customers, but knew it needed myriad features itself to be successful:
“We wanted something that was easy to work with, developer friendly, had an intuitive UI, had everything we needed from a data standpoint. Split just had everything we needed out of the box.”
Dan DeMeyere, VP Engineering, thredUP
But beyond initial accessibility and ease-of-use, they also realized that creating something from scratch and constantly maintaining it would distract them from their core mission:
If we built this in-house, most engineers would look at this and say ‘feature rollout, it’s not that hard’. But when you think about having multiple environments, having a UI, conditional filters, an impression data pipeline, all that—it’s a whole team working for a very long time to be able to do all of that.
Dan DeMeyere, VP Engineering, thredUP
Quickening QA
thredUP does QA in-house, within the engineering team: engineers check each others’ work, in staging and production environments. Split gives them the confidence to ship those features to prod, then make them accessible only to internal employees for testing purposes. That allows not only engineers, but anyone at the company—customer service, product, even the front desk—to actively QA together.
Deliver software features that matter, fast
The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Switch on a free account today or schedule a demo to learn more.