Thrive with Opportunity of A/B Testing at DevOps Day

“The risk for innovation is high if we don’t have the right technology in place.”  

This is one (of many) statements I highlight in my notes from Nicole Forsgren, CEO of DORA (now a part of Google Cloud) and author of the annual State of DevOps report and Accelerate, at our first DevOps day at NetApp Insight. In the time she spent with us Nicole revisited an old but relatable story of the risk a developer took at Amazon.  

In 2006 Greg Linden had a great idea – make customized recommendations to customers on last minute additions to their shopping cart based off past purchase history. The intent was to deliver an experience much like what you get checking out at the grocery store. Greg took the idea to his SVP who said “no”. Online customers could abandon cart much easily than they do in the store – he wasn’t willing to take the risk.  

But Greg did it anyway. 

After writing up a A/B test and sneaking it into production Greg gathered data over the next few weeks and presented it back to his SVP. His experiment worked. His feature resulted in a few percentage points of incremental revenue – millions of dollars at Amazon’s scale. But would you have the guts to defy your VP and push an experiment like this into production?  

In order to successfully experiment Greg had to have the right tools in place do easily push to production AND be able to “undo” any work should something go drastically wrong. Would your infrastructure enable you to do this? As Nicole put it, “if it takes you six months to push code, it’s also going to take you six months to undo a mistake. But if you can push code and experiments it then reduces the barrier for experimentation.”  

So, is your infrastructure up to the task? Last year we talked about NetApp’s framework for the six core capabilities IT has to deliver to enable DevOps success and behind these tools needs to lie the right platform to enable each developer and toolset. To learn more about enabling and measuring DevOps success from Dr. Nicole Forsgren make sure to check out her full talk from NetApp Insight 2018.