Occam's Razor
by Avinash Kaushik

5 + 4 Actionable Tips To Kick Web Data Analysis Up A Notch, Or Two

38 Comments | Print | PDF

We lovingly craft reports every day. We try to make sense of what they are saying. When we hear nothing we try to bludgeon them, hoping for the best.
My hope in this post is to share some simple tips with you that might make your reports and analysis speak to you a bit more. Suggestions that might increase the probability that you'll bump into things that might be insightful, and communicate data more effectively.
None of them are very hard to do, but I think they make a world of difference.
Excited? Here we go. . .
#1: Go as deep as you can. Then, a little bit more.
Far too often in our daily lives we let our job titles limit how deep we go in our analysis.
For example let's say I work at a delightful car / health / spaceship insurance company. Naturally all of my analysis is focused on the efficiency of the website in moving the Visitors quickly from the landing page to click on that delightful Submit Quote button.
I am focused on what the site does because that is what my job title says: Web Analyst
I am analyzing campaigns (which ones convert better and which worse), I am looking a little bit at the bounce rates, and of course I am totally obsessing about my seven step quote submission funnel (and how to reduce abandonment).
Bottom-line: Quote, quotes, quotes.
And that is fine.
The data is easily available in the web analytics tool so why not.
Here's my advice: You should kick things up a notch. Don't focus just on the quote (the part the site does), include the final conversion to a paying customer (even if that data is offline).
The picture you get from stopping at Quotes might be very different from stopping at Policies Purchased.
Here's what you are focusing on (and it is good):

All my experience in these things suggests that it is dangerous to think that the Conversions column is representative of the final outcome.
Here is what it probably looks like (and this is going from good to great):

See how the ranking changed?
You would make different recommendations right? Would it save your company money? Would it make you refocus your efforts on where improvements are needed?
You betcha!
For straight ecommerce websites the first picture is what you use every day. But for most other types of businesses the final success does not exist in web analytics tool. So what? Get the data out of the crm / erp / "backend" system. . . dump it into excel. . . write a simple formula!
Usually you don't need a complicated multi year data warehousing effort with expensive business intelligence tools to buy. At least for this scenario you just need a column and a short movie data with your online IT person and a longish coffee break with your "backend" IT person to get the right primary keys set up. Then you can bring your sexy back!
Go deep.
You are paid to find real bottom-line impacting insights (remember line of sight to net income?). Do that.
If you are a purely ecommerce business then you can go a bit deeper too. Consider doing quarterly analysis that focuses on calculating customer lifetime value. Up a notch.
If today you are a content site that is only focused on measuring content consumed try to go deeper to understanding CPA of the ads or Visitor Loyalty. Once again going one step deeper, up a notch.
And so on and so forth.
Make it a point to pause every Friday at 0900 hrs. Look at your most important work / report / dashboard. Then ask yourself this: "How can I take my view of the data one step deeper?"
Now figure out how to do that. That'll impress me, your boss and your mom.
#2: Join the PALM club. [PALM: People Against Lonely Metrics]
This rule actually comes from my second book, Web Analytics 2.0. [Page 318, Principles for Becoming an Analysis Ninja, if you have the book already.]
The rationale for this rule, joining the PALM club, is quite simple.
You need a someone in your life. I need someone. Everyone needs someone else. A boy friend. A girl friend. A cat. A "you complete me" person.
So why not your metrics?
We do reports / dashboards like this one all the time:

Ok great.
I know the top referrers sending traffic to my site in a month. Maybe I can appreciate more the power of Twitter or google.co.in or whatever.
You might even impress me next month with a updated version of this where some of these might have shifted a bit up or a bit down.
I might not do anything with the data… but you surely hypnotized me for a few seconds.
This is the problem with lonely metrics.
They don't have any context. They fail to communicate if 841 visits from Twitter were any good. In fact is any of the above good or bad? How do you know?
Why not find a BFF for your lonely metric and present something like this. . . .

Much better right?
I found a "you complete me" for my Visits metric, Bounce Rate.
Now in an instant I can not only see which referrers are big or small, I can see which ones are "good" or "bad".
I could have picked conversion rate as the bff. I could have picked % new visits. I could have picked connection speed or mobile platform or underwear size.
Whatever makes most sense for my business. But putting two minutes of thought into my metric would help make my report a little bit more useful.
Kick it up a notch. Right?
Never ever never never never ever present any metric all by itself.
If you want a cop out then at least trend it over time. If you actually want love then join PALM and don't let your metric be lonely.
Let me close with one of my favorite examples of this rule, this one's to inspire you if you have a pure content (non-ecommerce) website. . . .

Good to know what content's being consumed. Column: Pageviews.
Much much much better to know what the $ index value is for each.
See that crazy blue line that's literally off the chart? You would want to know that about the 1,414 pageviews right?
Now go find your dashboards, your reports, your data pukes (sorry!) and make sure that for every dimension you are not reporting success or failure using just one metric. Join PALM!
[Tip: Not that you are trying to but if you want to impress me but if you are then make sure the second metric you pick is as close to an outcome metric as possible. Or an actual outcome metric. I. Love. Outcomes.]
#3: Measure complete site success. Measure everyone's success.
One of my greatest passions when doing analysis is to look at the complete view of things. Rather than just the obvious.
An application of that passion is to look at all the jobs the website is doing, representing all the work that is being done by people in your company who touch the site.
Ecommerce is too easy an example of this so let me use a non profit example.
San Francisco Aids Foundation is a charity I support. It does incredible work to prevent new HIV infections.

The only way SFAF stays in business is if you and I make donations. As an Analyst I would focus all my energies on trying to figure out how many donations we are getting and where those people come from and what they are doing on the site etc.
But donations is just one measure of success ("macro conversion"). There are other jobs that the site is trying to do, and people who work on those jobs. So why not measure those?
For example. . . .

* SFAF helps prevention through information sharing and providing services. One key way of doing this is providing forms and information as downloads. Example see all the downloads on the Science & Public Policy page. Or the Bulletin of Experimental Treatment for AIDS.
I can track downloads easily (using event tracking or "fake" pageviews) and help quantify those micro conversions.
* There are a ton of micro conversions on the Advocacy Action Center page. Sign ups. Successful searches for elected officials. Tell-a-friend's.
* On the How You Can page, and other places on the site, there are links to other websites. Why not track these through outbound link tracking to see if we are sending people to the right place.
* Oh and of course the important micro conversion of signing up Volunteers!

Measure the above four micro conversions, in addition to the macro conversion of donation, helps give a complete view of success. And what to do better.
Maybe Google is really good at Volunteers and not optimal for attracting people who donate. If you focus only on donations you'll devalue Google. Or maybe facebook is the best source for sharing information (downloads). And more such things.
Not only are you measuring all that matters. . . . you are validating the jobs of people who put together all that content.

Most of the time we don't do this. We, web analysts, just focus on one thing and then we wonder why we don't have the impact we want to, or why everyone does not pay attention to us.
Broaden your view!
If I were analyzing Amazon I would measure sales AND affiliate signups, signups for amazon prime, credit cards, wish lists, "like's" on reviews, self publish inquiries, free downloads….
If I were analyzing L'Oreal Paris it would be sales AND reviews, coupons downloaded, successful completion of "Profile My Skin", videos watched, sign ups for mobile alerts….
In both cases a complete view of the website and success of every person who works on the site.
Ninjas do that. You should too.
[UPDATE: A key next step, post micro conversions identification, is to identify the Economic Value. See this post for specific ideas about how to do that: Excellent Analytics Tips #19: Identify Website Goal Values & Win!]
#4: Be smart about using time. Move beyond MoM.
This is one of the most common view of data presented in web analysis…

The picture illustrates the performance of a metric over two consecutive months.
This is of course better than just showing data for June.
The problem occurs when you proceed to look at six such graphs on your dashboard and then proceed to use the trends to deliver insights. You are reading too much into the ups and downs, you are inferring things that might not even exist.
Two months do not a trend make. Important lesson.
Not even for the world's most flat line no seasonality business.
So here is a best practice. . . . at least add three months. . . . if the data looks like below you'll think one thing (and every different from above pic)…

But if the data looks like the image below. . . . you'll think something else. . . .

Worry in one case. Jubilation for the temporary awesomeness for May in the other.
The more time you put into this graph (and if you have as much space as above you can easily add at least six months and it will still look pretty) the better.
But if I can only have three I love using current, prior, same month last year.

Better context right? Will take you off on a completely different line of inquiry, all from adding June 2009 to look at June 2010.
If June is the last month of your quarter and you have a cyclical business then maybe you want to compare Apr, May, June 2010 and have the first column be March 2010 because you want to see how the last month of this quarter did vs last month of the last quarter (because Apr and May don't really show if the trend ended as high or low as it should have ended).
So on and so forth.
Remember:

1. Don't look at just one month or just two consecutive months.
2. Understand your business and its cycles of up and down. Use that understanding to pick the right comparative time period / time horizon.
3. If you do present your data as a trend it does not hurt to include some "tribal knowledge" and throw in some annotations! Like this…

Sweet momma that is awesome!
Kick it up a notch, ok?
#5: Present data better, make insights obvious.
There are so many ways to present data that a small section of a blog post is insufficient. And of course there are so many people who are better at this than I am.
Let me just say that the way you present data matters, a lot. I'm not saying you should make it pretty. I could not care less if it is pretty or not. I'm saying present it in a way that the insights you think exist in the data become more obvious.
Here is a "data element", from an actual dashboard, that I really like. It might not be sexy but it is extremely functional and it is super awesome at communicating the smarts of the Analyst.
Three month trend for one very important business metric…

First note that rather than just showing one column for the performance of this metric it shows four. One for each key segment of the customer that the company has.
This would require you to know the business (good thing), know its customers (great thing) and track the segmented data. IE have your act together.
Second note that the data is for three months. You could show more but in this case you don't want to overwhelm the Executive. If you go more months, shrink the segments.
Third, really important, note that the overall goal is clearly indicated in the picture. 80. And to get that number you would have to talk to Finance and Marketing and HiPPO's and get an agreement up front. This is absolutely magnificent, key to you building relationships and finding insights.
The nice thing about our picture above is that the overall metric would get averaged out and show a trend similar those we showed in tip #4 above.
But would it be insightful enough? A single metric trend would hide insights.
In this case it is pretty clear that Blue, Red, Green segments are doing fine. In fact the one that is absolutely most important, Green, we are doing ok.
The stink bomb in the pile is Purple. It has been dragging the overall metric down (and you know that even if the overall metric is not even shown!).
And you know how much gap you need to overcome for each segment, and were to prioritize your work (PURPLE!!).
This is just one tiny, I call it "functional", way of presenting data.
The presentation is ok, could be made more pretty.
What's precious is the process that went into creating the element – talking to leaders, meeting with Finance and Marketing, identifying the key metrics, finalizing customer segments, and establishing goals.
We often don't do all the above work (the things that are mandatory for data driven organizations) and even if we do it we don't show it because we show lame single line graphs.
Don't do that.
Kick it up a notch. You are working very hard at your job, make sure your work shows up and helps identify better insights.
Those were the five simple things you can do every day to make the most of your daily data analysis. They are not very hard to do, and they'll pay outsized dividends.
I am not someone who leaves the good enough alone. No sirree bob!
With love and affection here are 4 more bonus tips on improving your analysis and truly kicking things up a few notches:
#6: Leverage segmentation, daily.
All said and done the number one way to move from being a Reporting Squirrel to an Analysis Ninja is to leverage segmentation. Every tool has on the fly current and historical segmentation feature set. Use it.
I'll honestly not respect anyone is not applying at least some primitive segmentation to their data.

Learn how to:

~ Web Analytics Segmentation: Do Or Die, There Is No Try!
~ Google Analytics Releases Advanced Segmentation: Now Be A Ninja!

#7: Move beyond the top ten rows of data, seriously.
The "head" of your data will sustain finding insights for a month or two. You might even action something. The real gold lies in your ability to analyze tens of thousands of rows of data at one time. It is harder to do, and hence the rewards are bigger and your competitors will eat your dust more.

Learn how to:

~ Analysis Ninjas: Move Beyond The Top Ten. Find Love (/Insights)
~ Make Web Analytics Actionable: Focus On "What's Changed"

#8: Perform "pan-session" analysis, and win big.
One of the absolute criminal behaviors in web analytics (and indeed online marketing) is that we are so obsessed about Visits, and visits based analysis.
Few people sleep with you on the first date. So why is that your mental model?
Every true Analysis Ninja focuses on measuring customer behavior of one person (or in our case Unique Visitor) over the entire span of that person's interaction one our website.

Hence my devotion to measuring Days and Visits to Purchase. Truly analyzing how people buy. Or analyzing Visitor Recency and Visitor Loyalty to understand now just the first Visit (and conversion) but rather subsequent Visits (and conversions).
I tell you this is honestly kicking your web analysis up five notches, not just one.

Learn how to:

~ Measure Latent Conversions & Visitor Behavior
~ Measure Days & Visits to Purchase

#9: Evolve to multichannel analytics, achieve analytics nirvana.
There is perhaps nothing harder and nothing more impactful than getting good at multi-channel analytics.
Measuring the offline impact of your online activities gives your business a view of success that is truly orgasmic. If you get good at measuring the impact on your website of your offline activities (television, catalogs, billboards etc) then you have truly accomplished the rarest of the rate.

Learn how to: Multichannel Analytics:

~ Tracking Online Impact Of Offline Campaigns
~ Tracking Offline Conversions. 7 Best Practices.

Feeling like an Analysis Ninja already?
Of course not, you have to go do all these things! :)
Remember that tips 1 through 5 you should be able to do quite easily, just need to remember them and remember to use them. Tips 6 through 9 take time, they take a lifetime. Remember them, practice when you have time and slowly evolve over time.
Ok?
Good luck.
As usual it's your turn now.
What are your favorite tips for data analysis? When you present data what is the "trick" that you use most often to be awesome? Have you used any of the tips above? Got any favorites? What do you think it takes to morph from a Reporting Squirrel into an Analysis Ninja?
Please share your feedback / critique / tips and all via comments.
Thanks.

Like this post? Share it: