Imagine you are part of an hunter tribe in the early days of mankind, and you realize that you need to hunt something big to keep the village fed during the winter months. Back in the day when when did not have robots, the chief of the tribe would select one of their stronger men to go and hunt alone. The validation if the person came back alive with the hunt it was a successful expedition. Imagine you start realizing that the male population of the tribe is getting thinned out because of the lack of predictability of your hunt ( you realize sending one person for a hunt is your control group), this time you decide you are going to send 2 together with the explicit instruction that they work together and try to get a bigger animal. As luck would have it the coordination works now you figure out that sending one person alone does not make sense because sending 2 people for the hunt gets a better outcome for the tribe.. Next time you try sending out a larger hunting party to see what they can get. Hence you see humans have been at this A/B testing stuff for a long time… it is just that now we can do it a greater scale and with lot less loss of life.
This one is an example of not only A/B testing but also of Big Data versus Smart Data. Imagine the small army of Alexander taking on the might of Persia and yet Alexander’s small army bought the behemoth Persian empire to its knees. Prior to the confrontation with the Persians, Alexander had a lot chance to experiment with other armies that the Greeks took down, he was able to implement strategic plays and work out ways to make his small army into a effective execution machine. I am not saying that the Persian Army lack discipline, they actually fought by the book (i.e. something that worked before should work the same way except now they have the bigger size). Persian Army did not challenge their conventional norms and just assumed that their army would just function as well with the larger contingent of soldiers (they did not A/B test) and in the context of Alexander they were so drunk with the size of their army that they did not take into account the terrain and it’s advantages offered to Alexander’s army. Plus the motivation that Alexander had made it very clear that they were moving forward not backwards actually helped the army to be more creative.
Another example of A/B testing at work and with some significant constraints too (actually all good A/B tests work with constraint, this is no different). Although it helped Alexander and team to win over the Persians … they eventually got as lazy with their power just like the Persians became
I think starbucks is a great technical company. We all joke that there is a Starbucks every two blocks in the big cities. But what is going on here? Yes I know their menu across the all the a starbucks is the same but there is a subtlety. Have you noticed how starbucks promo’s are different in their different stores (even if the stores are just blocks away). Starbuck A/B tests so many different variants of coffee drink (just of the non-fat, soy based, vanilla bean latte that you drink next time). They know exactly which store has a demand for what type of variant of their coffee and it’s volume. Which helps them manage their inventory better…. which is why I believe this company may have it’s rough patches but it will come through …because they are always testing what take the Starbucks experience to the next level!
Let me know what you think and if you have any stories to share feel free to share your thoughts on with me on Twitter..
BTW A big shout out to my readers in Tanzania ! I hope you find my content useful !