We
lovingly craft reports every day. We try to make sense of what they are
saying. When we hear nothing we try to bludgeon them, hoping for the
best.
My hope in this post is to share some simple tips with you that might
make your reports and analysis speak to you a bit more. Suggestions
that might increase the probability that you'll bump into things that
might be insightful, and communicate data more effectively.
None of them are very hard to do, but I think they make a world of difference.
Excited? Here we go. . .
#1: Go as deep as you can. Then, a little bit more.
Far too often in our daily lives we let our job titles limit how deep we go in our analysis.
For example let's say I work at a delightful car / health / spaceship
insurance company. Naturally all of my analysis is focused on the
efficiency of the website in moving the Visitors quickly from the
landing page to click on that delightful Submit Quote button.
I am focused on what the site does because that is what my job title says: Web Analyst
I am analyzing campaigns (which ones convert better and which worse),
I am looking a little bit at the bounce rates, and of course I am
totally obsessing about my seven step quote submission funnel (and how
to reduce abandonment).
Bottom-line: Quote, quotes, quotes.
And that is fine.
The data is easily available in the web analytics tool so why not.
Here's my advice: You should kick things up a notch. Don't focus just
on the quote (the part the site does), include the final conversion to a
paying customer (even if that data is offline).
The picture you get from stopping at Quotes might be very different from stopping at Policies Purchased.
Here's what you are focusing on (and it is good):
All my experience in these things suggests that it is dangerous to
think that the Conversions column is representative of the final
outcome.
Here is what it probably looks like (and this is going from good to great):
See how the ranking changed?
You would make different recommendations right? Would it save your
company money? Would it make you refocus your efforts on where
improvements are needed?
You betcha!
For straight ecommerce websites the first picture is what you use
every day. But for most other types of businesses the final success does
not exist in web analytics tool. So what? Get the data out of the crm /
erp / "backend" system. . . dump it into excel. . . write a simple
formula!
Usually you don't need a complicated multi year data warehousing
effort with expensive business intelligence tools to buy. At least for
this scenario you just need a column and a short movie data with your
online IT person and a longish coffee break with your "backend" IT
person to get the right primary keys set up. Then you can bring your
sexy back!
Go deep.
You are paid to find real bottom-line impacting insights (remember
line of sight to net income?). Do that.
If you are a purely ecommerce business then you can go a bit deeper
too. Consider doing quarterly analysis that focuses on calculating
customer lifetime value. Up a notch.
If today you are a content site that is only focused on measuring
content consumed try to go deeper to understanding CPA of the ads or
Visitor Loyalty. Once again going one step deeper, up a notch.
And so on and so forth.
Make it a point to pause every Friday at 0900 hrs. Look at your most
important work / report / dashboard. Then ask yourself this: "How can I
take my view of the data one step deeper?"
Now figure out how to do that. That'll impress me, your boss and your mom.
#2: Join the PALM club. [PALM: People Against Lonely Metrics]
This rule actually comes from my second book,
Web Analytics 2.0. [Page 318, Principles for Becoming an Analysis Ninja, if you have the book already.]
The rationale for this rule, joining the PALM club, is quite simple.
You need a someone in your life. I need someone. Everyone needs someone else. A boy friend. A girl friend. A cat. A "
you complete me" person.
So why not your metrics?
We do reports / dashboards like this one all the time:
Ok great.
I know the top referrers sending traffic to my site in a month. Maybe
I can appreciate more the power of Twitter or google.co.in or whatever.
You might even impress me next month with a updated version of this
where some of these might have shifted a bit up or a bit down.
I might not do anything with the data… but you surely hypnotized me for a few seconds.
This is the problem with lonely metrics.
They don't have any context. They fail to communicate if 841 visits
from Twitter were any good. In fact is any of the above good or bad? How
do you know?
Why not find a BFF for your lonely metric and present something like this. . . .
Much better right?
I found a "you complete me" for my Visits metric,
Bounce Rate.
Now in an instant I can not only see which referrers are big or small, I can see which ones are "good" or "bad".
I could have picked conversion rate as the bff. I could have picked %
new visits. I could have picked connection speed or mobile platform or
underwear size.
Whatever makes most sense for my business. But putting two
minutes of thought into my metric would help make my report a little bit
more useful.
Kick it up a notch. Right?
Never ever never never never ever present any metric all by itself.
If you want a cop out then at least trend it over time. If you
actually want love then join PALM and don't let your metric be lonely.
Let me close with one of my favorite examples of this rule, this
one's to inspire you if you have a pure content (non-ecommerce) website.
. . .
Good to know what content's being consumed. Column: Pageviews.
Much much much better to know what the
$ index value is for each.
See that crazy blue line that's literally off the chart? You would want to know that about the 1,414 pageviews right?
Now go find your dashboards, your reports, your data pukes (sorry!)
and make sure that for every dimension you are not reporting success or
failure using just one metric. Join PALM!
[Tip: Not that you are trying to but if you want to impress me but if
you are then make sure the second metric you pick is as close to an
outcome metric as possible. Or an actual outcome metric. I. Love.
Outcomes.]
#3: Measure complete site success. Measure everyone's success.
One of my greatest passions when doing analysis is to look at the complete view of things. Rather than just the obvious.
An application of that passion is to look at all the jobs the website
is doing, representing all the work that is being done by people in
your company who touch the site.
Ecommerce is too easy an example of this so let me use a non profit example.
San Francisco Aids Foundation is a charity I support. It does incredible work to prevent new HIV infections.
The only way SFAF stays in business is if you and I
make donations.
As an Analyst I would focus all my energies on trying to figure out how
many donations we are getting and where those people come from and what
they are doing on the site etc.
But donations is just one measure of success ("
macro conversion"). There are other jobs that the site is trying to do, and people who work on those jobs. So why not measure those?
For example. . . .
* SFAF helps prevention through information
sharing and providing services. One key way of doing this is providing
forms and information as downloads. Example see all the downloads on the
Science & Public Policy page. Or the
Bulletin of Experimental Treatment for AIDS.
I can track downloads easily (
using event tracking or "fake" pageviews) and help quantify those micro conversions.
* There are a ton of micro conversions on the
Advocacy Action Center page. Sign ups. Successful searches for elected officials. Tell-a-friend's.
* On the How You Can page, and other places
on the site, there are links to other websites. Why not track these
through outbound link tracking to see if we are sending people to the
right place.
* Oh and of course the important micro conversion of
signing up Volunteers!
Measure the above four micro conversions, in addition to the macro
conversion of donation, helps give a complete view of success. And what
to do better.
Maybe Google is really good at Volunteers and not optimal for
attracting people who donate. If you focus only on donations you'll
devalue Google. Or maybe facebook is the best source for sharing
information (downloads). And more such things.
Not only are you measuring all that matters. . . . you are validating the jobs of people who put together all that content.
Most of the time we don't do this. We, web analysts, just focus on
one thing and then we wonder why we don't have the impact we want to, or
why everyone does not pay attention to us.
Broaden your view!
If I were analyzing
Amazon
I would measure sales AND affiliate signups, signups for amazon prime,
credit cards, wish lists, "like's" on reviews, self publish inquiries,
free downloads….
If I were analyzing
L'Oreal Paris
it would be sales AND reviews, coupons downloaded, successful
completion of "Profile My Skin", videos watched, sign ups for mobile
alerts….
In both cases a
complete view of the website and
success of every person who works on the site.
Ninjas do that. You should too.
[UPDATE: A key next step, post micro conversions identification, is
to identify the Economic Value. See this post for specific ideas about
how to do that:
Excellent Analytics Tips #19: Identify Website Goal Values & Win!]
#4: Be smart about using time. Move beyond MoM.
This is one of the most common view of data presented in web analysis…
The picture illustrates the performance of a metric over two consecutive months.
This is of course better than just showing data for June.
The problem occurs when you proceed to look at six such graphs on
your dashboard and then proceed to use the trends to deliver insights.
You are reading too much into the ups and downs, you are inferring
things that might not even exist.
Two months do not a trend make. Important lesson.
Not even for the world's most flat line no seasonality business.
So here is a best practice. . . . at least add three months. . . . if
the data looks like below you'll think one thing (and every different
from above pic)…
But if the data looks like the image below. . . . you'll think something else. . . .
Worry in one case. Jubilation for the temporary awesomeness for May in the other.
The more time you put into this graph (and if you have as much space
as above you can easily add at least six months and it will still look
pretty) the better.
But if I can only have three I love using current, prior, same month last year.
Better context right? Will take you off on a completely different
line of inquiry, all from adding June 2009 to look at June 2010.
If June is the last month of your quarter and you have a cyclical
business then maybe you want to compare Apr, May, June 2010 and have the
first column be March 2010 because you want to see how the last month
of this quarter did vs last month of the last quarter (because Apr and
May don't really show if the trend ended as high or low as it should
have ended).
So on and so forth.
Remember:
1. Don't look at just one month or just two consecutive months.
2. Understand your business and its cycles
of up and down. Use that understanding to pick the right comparative
time period / time horizon.
3. If you do present your data as a trend it does not hurt to include some "
tribal knowledge" and throw in some annotations! Like this…
Sweet momma that is awesome!
Kick it up a notch, ok?
#5: Present data better, make insights obvious.
There are so many ways to present data that a small section of a blog
post is insufficient. And of course there are so many people who are
better at this than I am.
Let me just say that the way you present data matters, a lot. I'm not
saying you should make it pretty. I could not care less if it is pretty
or not. I'm saying present it in a way that the insights you think
exist in the data become more obvious.
Here is a "data element", from an actual dashboard, that I really
like. It might not be sexy but it is extremely functional and it is
super awesome at communicating the smarts of the Analyst.
Three month trend for one very important business metric…
First note that rather than just showing one column
for the performance of this metric it shows four. One for each key
segment of the customer that the company has.
This would require you to know the business (good thing), know its
customers (great thing) and track the segmented data. IE have your act
together.
Second note that the data is for three months. You
could show more but in this case you don't want to overwhelm the
Executive. If you go more months, shrink the segments.
Third, really important, note that the overall goal
is clearly indicated in the picture. 80. And to get that number you
would have to talk to Finance and Marketing and HiPPO's and get an
agreement up front. This is absolutely magnificent, key to you building
relationships and finding insights.
The nice thing about our picture above is that the overall metric
would get averaged out and show a trend similar those we showed in tip
#4 above.
But would it be insightful enough? A single metric trend would
hide insights.
In this case it is pretty clear that Blue, Red, Green segments are
doing fine. In fact the one that is absolutely most important, Green, we
are doing ok.
The stink bomb in the pile is Purple. It has been dragging the
overall metric down (and you know that even if the overall metric is not
even shown!).
And you know how much gap you need to overcome for each segment, and were to prioritize your work (PURPLE!!).
This is just one tiny, I call it "functional", way of presenting data.
The presentation is ok, could be made more pretty.
What's precious is the process that went into creating the
element – talking to leaders, meeting with Finance and Marketing,
identifying the key metrics, finalizing customer segments, and
establishing goals.
We often don't do all the above work (the things that are mandatory for
data driven organizations) and even if we do it we don't show it because we show lame single line graphs.
Don't do that.
Kick it up a notch. You are working very hard at your job, make sure your work shows up and helps identify better insights.
Those were the five simple things you can do every day to make the
most of your daily data analysis. They are not very hard to do, and
they'll pay outsized dividends.
I am not someone who leaves the good enough alone. No sirree bob!
With love and affection here are 4 more bonus tips on improving your analysis and truly kicking things up a few notches:
#6: Leverage segmentation, daily.
All said and done the number one way to move from being a Reporting
Squirrel to an Analysis Ninja is to leverage segmentation. Every tool
has on the fly current and historical segmentation feature set. Use it.
I'll honestly not respect anyone is not applying at least some primitive segmentation to their data.
Learn how to:
#7: Move beyond the top ten rows of data, seriously.
The "head" of your data will sustain finding insights for a month or
two. You might even action something. The real gold lies in your ability
to analyze tens of thousands of rows of data at one time. It is harder
to do, and hence the rewards are bigger and your competitors will eat
your dust more.
Learn how to:
#8: Perform "pan-session" analysis, and win big.
One of the absolute criminal behaviors in web analytics (and indeed
online marketing) is that we are so obsessed about Visits, and visits
based analysis.
Few people sleep with you on the first date. So why is that your mental model?
Every true Analysis Ninja focuses on measuring customer behavior of
one person (or in our case Unique Visitor) over the entire span of that
person's interaction one our website.
Hence my devotion to measuring Days and Visits to Purchase. Truly
analyzing how people buy. Or analyzing Visitor Recency and Visitor
Loyalty to understand now just the first Visit (and conversion) but
rather subsequent Visits (and conversions).
I tell you this is honestly kicking your web analysis up five notches, not just one.
Learn how to:
#9: Evolve to multichannel analytics, achieve analytics nirvana.
There is perhaps nothing harder and nothing more impactful than getting good at multi-channel analytics.
Measuring the offline impact of your online activities gives your
business a view of success that is truly orgasmic. If you get good at
measuring the impact on your website of your offline activities
(television, catalogs, billboards etc) then you have truly accomplished
the rarest of the rate.
Learn how to: Multichannel Analytics:
Feeling like an Analysis Ninja already?
Of course not, you have to go do all these things! :)
Remember that tips 1 through 5 you should be able to do quite easily,
just need to remember them and remember to use them. Tips 6 through 9
take time, they take a lifetime. Remember them, practice when you have
time and slowly evolve over time.
Ok?
Good luck.
As usual it's your turn now.
What are your favorite tips for data analysis? When you present data
what is the "trick" that you use most often to be awesome? Have you used
any of the tips above? Got any favorites? What do you think it takes to
morph from a Reporting Squirrel into an Analysis Ninja?
Please share your feedback / critique / tips and all via comments.
Thanks.
752 comments. Share your thoughts »