Tag Archives: Analytics

The value of the media that airs Super Bowl commercials will go the way of the Dodo bird

Image

 

Yes I know it is New years Eve and we all have festivities planned for the clock to midnight and ring in the New Year. Well as part of my goodbye to 2013 … I hope one day the media that charges astronomical amounts to air commercials goes away like 2013. 

Just for the record I am not against commercials! If a commercial encourages someone to do something whether emotional or otherwise is a great thing. The problem I have about the medium airing traditional commercials is that the attribution model really sucks!  Yes I have heard the excuse that certain commercials are “branding” or “awareness” campaigns hence are not measurable! I think that is just a bogus argument! For a “branding” or “awareness” campaign to truly work, there needs to be action that attributed to the campaign, such as more visitors, increase in sales etc. The problem with a commercial airing and the ensuing increase in sale is not directly implied it is best a proxy … which is ridiculous in this day and age.  Why is it that the traditional media companies are charging millions of dollars for mere seconds ? 

The reason I believe is because of something called value to media versus value to customer.  Advertisers in general focus on impressions but there is very little ROI  to show for (there is a fix for that but apparently a lot of folks have not made the leap yet).  Traditional media companies thrive on impressions (whether they convert for business, that is a different matter) and because they own rights to certain events that guarantee eyeballs (i.e. impressions) to advertisers they charge for the medium not for the customer utility.  I understand that Super Bowl commercials are an institution but really! What is the effectiveness of that medium (besides the proxy attribution world we have been living with)? How do you know that some one bought something because they saw a particular Ad on TV or because of a bill board on Times Square? 

This is why I believe the web and mobile will change all this … we are gradually working towards a model where we can attribute what actions caused our clients to do anything whether buy, visit our site, interact with our mobile app etc. Some of the more effective Super Bowl Ads are by Coca Cola (because you get text something back the company). So if a customer found something valuable out a commercial they are willing to text something back … which then feeds  into brand loyalty and potentially future acquisition of a Coca Cola Product. 

In the past commercials relied a lot on the “Halo” effect of their commercials and the traditional media companies made a mint by charging an arm and leg.  In the future traditional media companies need to revisit their business model in order to survive in the commercial world and figure out how they plan to incorporate the “value to customer” angle,  “value of media” is going away  just like 2013.

 

Happy New Year everyone !!  Looking forward to a great 2014!

A/B testing is not new!!! We are just realizing the importance of it and Starbucks a technical company

Lately I have been hearing a lot about how great A/B testing is and that it is  the best thing since slice bread!  Please understand I am a big fan of A/B testing  but let us get real, this is not something new. The web like everything has an amplifying factor of a million, which is why it is like some hot stuff … just like BigData … it just did not happen over night and someone had some sort of epiphany and lo behold a phenomenon was born!  Everything takes time to take hold and become a phenomenon ….  and while it is taking time to take hold there are a series of A/B tests that are going on to make sure what takes hold and what does not. Practice that learn form the results survive and those who don’t well they perish!

I am from a software development back ground so oh BTW agile development, continuous delivery, DevOps or as I like calling it BizOps are all forms A/B testing.  
A/B testing is so ingrained in our lives we just don’t realize it (just ask the hospitals and the Pharmaceutical folks, they have doing it for quite some time).  I believe one of the best A/B tests that marketed well was the “Pepsi Challenge” by John Scully… to bad he messed it up for Apple though. 
Why are A/B tests important ? Well because they help us build relationships (again us not realizing it). Everyday there are several A/B tests happening my house…. how can I make daughter do her home work ? or what can I do to improve my wife’s experience in being my wife everyday … (BTW sometimes I do get lazy … and pay the price). I would go even further to say A/B tests are so ingrained in our DNA, we just don’t realize it. I am going to share 2 old stories and an example of how A/B testing working even today outside of the web context … there are so many examples to choose from which makes it even harder….
 Early human hunters
         Imagine you are part of an hunter tribe in the early days of mankind, and you realize that you need to hunt something big to keep the village fed during the winter months. Back in the day when when did not have robots, the chief of the tribe would select one of their stronger men to go and hunt alone. The validation if the person came back alive with the hunt it was a successful expedition. Imagine you start realizing that the male population of the tribe is getting thinned out because of the lack of predictability of your hunt ( you realize sending one person for a hunt is your control group), this time you decide you are going to send 2 together with the explicit instruction that they work together and try to get a bigger animal. As luck would have it the coordination works now you figure out that sending one person alone does not make sense because sending 2 people for the hunt gets a better outcome for the tribe.. Next time you try sending out a larger hunting party to see what they can get. Hence you see humans have been at this A/B testing stuff for a long time… it is just that now we can do it a greater scale and with lot less loss of life.

Alexander the great and the Persians
This one is an example of not only A/B testing but also of Big Data versus Smart Data. Imagine the small army of Alexander taking on the might of Persia and yet Alexander’s small army bought the behemoth Persian empire to its knees.  Prior to the confrontation with the Persians, Alexander had a lot chance to experiment with other armies that the Greeks took down, he was able to implement strategic plays and work out ways to make his small army into a effective execution machine. I am not saying that the Persian Army lack discipline, they actually fought by the book (i.e. something that worked before should work the same way except now they have the bigger size).  Persian Army did not challenge their conventional norms and just assumed that their army would just function as well with the larger contingent of soldiers (they did not A/B test) and in the context of Alexander they  were so drunk with the size of their army that they did not take into account the terrain and it’s advantages offered to Alexander’s army. Plus the motivation that Alexander had made it very clear that they were moving forward not backwards actually helped the army to be more creative.
Another example of A/B testing at work and with some significant constraints too (actually all good A/B tests work with constraint, this is no different). Although it helped Alexander and team to win over the Persians … they eventually got as lazy with their power just like the Persians became

StarBucks !
I think starbucks is a great technical company. We all joke that there is a Starbucks every two blocks in the big cities. But what is going on here? Yes I know their menu across the all the a starbucks is the same but there is a subtlety. Have you noticed how starbucks promo’s are different in their different stores (even if the stores are just blocks away). Starbuck A/B tests so many different variants of coffee drink (just of the non-fat, soy based, vanilla bean latte that you drink next time). They know exactly which store has a demand for what type of variant of their coffee and it’s volume. Which helps them manage their inventory better…. which is why I believe this company may have it’s rough patches but it will come through …because they are always testing what take the Starbucks experience to the next level!

I know this was not one my regular types of Blog post…. But I wanted to make a point about A/B testing, DevOps all these new “Buzz Words” are great to push us forward. But we need to take a step back and look at this is in the grand scheme of things. What we are witnessing is the extreme scale of what we have always done without thinking twice and now we are applying in a more conscientious and effective ways to enrich our lives.

Let me know what you think and if you have any stories to share feel free to share your thoughts on with me on Twitter..

BTW A big shout out to my readers in Tanzania ! I hope you find my content useful !

5 Web Metrics to track for online commerce for Newbies

After my April blog on vanity metrics and real metrics (http://goo.gl/7KYj5) I have had a chance to meet several people,who are curious about web metrics but do not know where to start. I agree someof us a very adept at reading the googleanalytics blog and come up to speed to read Avinash Kaushik’s blog Occam’s Razor. Very few blogs cover how getstarted and what to focus on, over the last few years I have narrowed 5 keymetrics that work for me (BTW there is a certain level of comfort-ability oneneeds to have in order get these metrics). These metrics are multi-purpose aswell, they not only show you how your business is doing but also highlight theweb experience that a potential client might be going through via your website
The 5 metrics that have worked for me (especially if I want to track the buying funnel) are as follows:
  • · Exit Rates
  • · Churn Rates
  • · Cart Abandonment rates (this is needed if youhave a e-commerce site)
  • · Average Days to Purchase
  • · Average visits to Purchase

I am not inferring that metrics like Unique Visitors, Pageviews, Bounce Rates etc. are not relevant. They are,it just depends on the goal you want to accomplish. In the case of this blog the assumption is that there is a web site on the internet that is just starting and wants to do some level of commerce

Exit Rates:

I am big fan of exit rates because to me they can tell you which page of your web presence is leaking. Please do not confuse this with Bounce rates. Bounce rates are metrics that anyone who lands on your page and then just leaves. If your page has a clear call to action (CTA) and you have a high bounce rate then that is something to look into. Exit rates focus more on the page flow of the session i.e. which people did not just land on the page but more about how people navigate their way during the session
Here is an example of an exit rate
Monday
Page 1 -> Page 2-> Page 3
Tuesday
Page 2->Page 1-> Page 3
Wednesday
Page 2 -> Exit
Thursday
Page 1-> Page 3-> Exit
Friday
Page 2 -> Page 3 -> Exit
In the case above the exit rates would be calculated asfollows
· Page 1 would be 33% (3 of 5 sessions included Page 1)
· Page 2 would be 50% (4 of the 5 sessionsincluded Page 2)
· Page 3 would be 50% (4 of the 4 sessionsincluded Page 3)
Why are exit rates important? They give an idea of how youlead generation activities are performing especially in the context of thefunnel. In the case shown above I would want so see why Page 2 and Page 3 havehigh exit rates? You should ask yourself the question like are you call toactions not compelling enough?

Churn Rates

Churn rates deal with how are you retaining the people thatyou already have. This metric is borrowed from the Mobile operator industry. Ifyou are not seeing people renew (as inthe case of SaaS offerings) or even if you see that people that sign up for atrial once and do not return again (prospective customers). For the latterthere is a stronger content play especially if you selling downloadable onpremise solution. What you want is that after they download they come back toview more content and other key things back at your site. You can complementthis metric with some of the social media metrics especially for prospectingclients as in which user decided to comeback and contribute to forum or posteda comment on a blog post etc. You canalso track based on the cookies you have placed for a particular user etc.There are various ways to track churn via the web.

Abandonment Rate

If you have an ecommerce site then you should track thenumber people filled their shopping cart and then decided not to followthrough. If you are using Inside sales for closing your deals, then you need totrack the inside sales metric of loss percentages for every lead that came inthe funnel and the opportunity was lost.

Average Days to Purchase

This is about how long it takes some to purchase. This theweb term for the average sales cycle i.e. how long does it take to closebusiness. This metric can help you identify whether your offering hasself-evident value. Products are just tosimple and convert easily to sale others usually take a long time and that youwant to fix.

Average visits to purchase

This metric is about how many times a user visits beforethey purchase. This metric gives you an idea of how frequently someone iscoming to your site before they buy. This metric can help in making sure theright content is available to provide the right level of information to assistin the buying decision. This can also help optimize the web experience and helpyour clients go down the funnel you are guiding them too.
Like I said in the beginning of the blog, these are the metricsthat will help you get started. Not only will these metrics help you understandyour web business but also improve your web experience as well. I look forward to hearing your thoughts aboutmetrics that help you in your web business. You can contact me @ kkanakas on twitter with your comments

Web Strategy: What is a good bounce rate?

I don’t know how many times I have heard that question, that is like asking how many times do get acknowledged when you say hello?  Sorry to sound to presumptuous here but it would be a wasted conversation because the person asking the question does not understand the web or even does not know what outcome they want to drive and hence ask for a “prescriptive” set of numbers so that they can measure against.

 Why is it hard for people to be more outcome driven? Because bounce rate although they are an ok metric, it is contingent upon other factors such as :
Location: If you are targeting a particular geography and you have high bounce rates from sources outside of the geography should you care ?
New vs Returning: This is another factor that you need to consider on the bounce rates. New users are great but if you have a higher rate of people returning then that is a good thing
Device: What are the devices people using to come to your website? If you find a high bounce rate because your site is not mobile optimized.Then you should do something about that
Medium: How are people coming to your site? SEO, Email campaigns, Social media, Paid Search, organic etc. If you break up your traffic this way you will be able to see what are some of the more effective mediums to reach out to your clientele.
All these divisions/ segments are interrelated I guess the net of this blog is that know you what your outcomes are and segment accordingly.  Bounce rates are great start if you don’t know web metrics but what you should be really looking at are exit rates (thanks to Avinash Kaushik: http://goo.gl/qjpn0). To me exit rates are more revealing about the leaks in your conversion funnel and what you need to fix. 
Another more important metric would be churn rates, similar to exit rates except churn rates focus on the customers you already have. What are you doing to take care of your existing customers and at what rate are they leaving (http://goo.gl/t1ADj).

So when some asks you about what is the ideal bounce rate? Please take the time to educate them and let them understand what outcomes are they trying to drive ( Increase registrations, downloads, donations, buy products etc,)

Caveat: If your company is obsessed  about bounce rates, then you have a good inkling that the organization has never thought of the web as a viable channel and you have long road to educate people.

You can contact me @ kkanakas on twitter with your comments

3 ways to research an Opportunity

I know I am not an entrepreneur but I do know a thing or 2 about intra-preneurship. Working in technology company, every now and then you come across new cool things. Being product manager in my prior incarnation, I have been called a buzz kill  when it came to taking out the “coolness”  from the shiny new thing (Although I must confess I do enjoy working with cool things too). 
          When I am look at cool technology especially since I am in the business of selling software. I have to ask myself the following questions:

  • Does this new solution or cool technology solve a pressing problem?
  • Is the problem big enough that a potential customer is willing to pay for it ?
  • Does the opportunity match with my ability or my teams ability to deliver a solution?
  • Are the profits worth taking the risks?
Granted that one cannot be a complete expert in everything but that is where your ability to research comes in  and I am not talking about going on the internet and searching on the topic.  The internet is one of the few tools in your quiver that you use. 
The first thing you need to do is a frame the problem by asking yourself the following question:
What is the problem you are intending to solve ?
By answering that question you have identified the problem area/s you are going to research (It is critical for any venture actually_ .  Once you have identified the problem then  these are the following 3 areas that I would look at to evaluate the opportunity further:

1 – Meeting with Clients about a potential problem to be solved

Clients will tell you what their specific problems they are facing without even prompting and also the jobs they want to get done. Some might go to the extent on how you should help them to get their job done.  To me clients are a great source innovation. Now if you believe the Henry Ford’s old adage (which I do) “If I had asked my clients what did they want ?  They would have said faster horses”. But when working with your client base, the framing of the question needs to be different   instead of asking them the opportunity problem ask them the kind of work they are trying to accomplish. You may find some surprising inputs that might validate the opportunity at hand. Too many people expect their clients to just get it and understand the problem space they are going after or intend to go after.

2 – Meet with your Business Partners and Sales force

Both of these constituents are hungry to look at new opportunities and they are usually one of the few sources of validation. Business partners are useful because their business depends on you being successful and they also have access to first hand knowledge of why they are being employed by your clientele. 
Your sales force is the other constituent (if you don’t have a sales force then you should go back to doing #1). Sales always wants to win and as they should. They are one of the strongest advocates of  your products/solutions and they would be first one to let you know why they are losing on certain opportunities

3 – Internet based 3rd party research

The internet is truly a boon to doing tertiary research. You can get access to papers written by some well established analyst at a large analyst firm or you can take the simple route of trolling websites where your potential user base gets together to exchange and share ideas. I would go as far to say that trolling forums is just as effective as reading up on analyst papers (there is an art to it, but it can be done). It is called social listening. What do I mean by social listening ?  Well in the “internet of things” there are congregations of people in the virtual world building relationships especially on forums like “Linkedn” or even “Twitter” for that matter. Sometimes, all you have to do is join the right group or follow the right hashtag and you can get your answer/validation of the opportunity at hand. The analyst paper will probably give you a better idea of the market size but these two efforts combined provide a winning combination.
Let me assure you these are not fool proof activities and sometimes no matter how much research you do you may deliver a dud sometimes (just do it fast enough, so that you can course correct early on).
What are some of your ways to research an opportunity ? Appreciate any insight you can provide.
You can contact me @ kkanakas on twitter with your comments

How to get your business idea going

I am big a fan of Spreecast, but this topic is something that is close to my heart. Some insights on how to implement a business idea and get it going.

Product Management: 5 Ways statistics can help product managers

Statistics was never one my favorite topics but I was reintroduced to it when I went back to school  and now I cannot stop talking about. It also helps that BigData is now affordable to a lot people, cheaply I might add and the fact the predictive analytics is “in” thing right now.
                          I usually get a chance to talk to few new product managers and I am always surprised how little value they give to statistics in general. Yes doing regression analysis on variables may not sound cool but if you master a few techniques you can actually go far. In this blog I document 5 areas where a little bit of statistics can help

Segmentation

If you are product marketer or product manager this is one activity you have to do. Segmentation is critical activity even in the context of a startup.  If you need to create a new niche in the marketplace or focus on a particular type of client archetype or experience this is key. One statistical technique you can use effectively is regression analysis to see which independent variables influence the dependent variable. If you don’t know what I am talking about I would recommend the following books as a great primer on statistics
Heads first Statistics or Statistics in a nutshell  both book happen to be from O’Reilly media because they are actually useful books

Value Analysis  

Every once in a while you are asked what is value or how do you know what is valuable to your client base. In statistics there is a tool you can use called Conjoint Analysis.  Conjoint analysis let you look at different aspects or features and figure out how to maximize and identify the right features and function to deliver by looking at the data (which you should have after meeting your clients). One of the best explanations of Conjoint Analysis is given in the book Marketing Metrics. Conjoint Analysis is very powerful tool and can also give you broad insight into managing your requirements better for the various products you bring to market

 

Analyzing trends 

Trends are the anathema of product managers especially if  the trend has already taken a foothold in the marketplace.  Obviously the most simple way to spot a trend is a to plot on a graph and see the trend (if you are doing that, it means the trend has already taken hold and you are late in the game). In order to stay ahead of trend and if you are constantly engaging with your clients, you should be able to see what variables are important to them. There are some tools like binomial distribution that can help in identify a trend manifesting in a sample survey you with your client base. Binomial distribution can always provide a good proxy for a full blown research effort but they can offer a quick an dirty way to get an idea of what is going on.

Client Satisfaction analysis 

I am big fan of Pareto Charts or most commonly known at the 80/20 rule. You can  identify the top issues that matter to clients and focus your efforts in remediating those top concerns in your overall user experience. There are plenty of credible examples of how to develop a pareto chart but the simplest explanation that I have read is on a blog written by blogger Duncan Haughey. Please check out his blog at the following URL:  http://goo.gl/4E2VV

Quality

The definitive book in this context is Katrina Maxwell’s book called Applied Statistics for Software Managers 
There simple regression analysis techniques that Katrina highlights that can be used not only by software managers but these techniques are applicable to anyone launching a new product whether it be software or hardware manufacturing
As you can statistics can be applied very effectively in product management activities and just like everything you can provide context. I do want to emphasize just looking at raw data would not do, you actually need to get out of the office and talk to clients and get the data.  You can use other 3rd party research to do the same.  The emphasis on using statistics will start to happen more and more in every one’s jobs as Big Data infrastructure continues to be accessible to people.

You can contact me @ kkanakas on twitter with your comments