Occam's Razor 
by Avinash Kuashik

Magnificent Mobile Website And App Analytics: Reports, Metrics, How-to!

42 Comments | Print | PDF

Nothing I can tell you about the importance of having an incredible mobile strategy will surprise you. Mobile devices (phones, tablets, wearables) are transforming how we behave, how we buy, how we consume content, and dare I say how we become happy or we become sad.
You after all have all of the aforementioned devices, and it is likely that at some level you are looking at traffic to your company's digital existence.
Still, let me try to surprise you.
Here's a graph that shows how US adults consume media, it shows time in hours. In blue is how much time we spent in 2010 and in red the time spent in 2014.

Surely you are not surprised that digital finally beats TV. (Yea!!!) I was not surprised, we could see that coming quite clearly.
What was surprising, even to me (!), was the dramatic shift between 2010 to 2014 to mobile content consumption. The light gray line is desktops plus laptops, the dark gray line is mobile devices.
The only reason good old digital is beating TV is mobile.
Amazing, right? And we see this trend all around the world.
Yet, if you look at the mobile experiences of the Fortune 1000, you will feel sad. If you look at the mobile marketing strategies, you will see they don't reflect this shift to mobile. A majority of YouTube consumption is on mobile, yet if there is an advertising or content strategy inside a company for YouTube it rarely accommodates for this reality.
Why?
Many reasons. CEOs still don't get it. CMOs don't grasp the implication of this shift in consumer behavior. Company UX leaders are happy to stink less by taking the sub-optimal path of responsive design, rather than create a mobile-unique experience (your customers tend to do different things on your desktop site than your mobile site!).
But why blame others, in this post let's focus on one important reason whose responsibility can be squarely put on your shoulders and mine: Measurement. Let's get the information we need to get our CEOs, CMOs and CUXOs to unleash a tsunami of creativity and awesomeness when it comes to mobile.
Framing the Opportunity.
There are two types of experiences we need to worry about.
Our mobile websites. Our mobile applications.
They are two completely different beasts.
Consider their purpose. Mobile websites have to do many jobs, because people with many different purposes will come there. You have to solve for all of them. Mobile apps are much more focused (or they stink), and people use them for specific purposes. They use different technologies of course, and different distribution mechanisms.
To use my See-Think-Do-Care framework , mobile websites have to solve for See, Think and Do while mobile apps usually have to solve for Care.

Create a distinct mobile website and mobile app measurement strategies. They will need two different implementations, it is quite likely that you will end up with two sets of metrics (more people focused for mobile apps, more visit focused for sites).
Remember, when someone says mobile analytics, first ask the clarifying question: Do you mean mobile application or mobile website? Then approach each separately (even though there are tools like Google Analytics that will do both). It will save you hours and hours of time, effort and focus.
In this post we will look mobile sites first, both data collection and analysis, and then mobile applications.
The post is written in these six steps:

Step 1: Tag every shared link with campaign tracking parameters.
Step 2. Tag your mobile website. GTM FTW!
Step 3. Dive into Mobile Reporting and Analysis.
Step 4. Dive into Mobile Reporting and Analysis.
Step 5. Implement Cross-Device Tracking.
Step 32. Media-Mix Modeling/Experimentation.

Taking inspiration from my favorite quote, it's not the ink, it's the think, in each section I'll share the think and then the ink, which includes screenshots and specific guidance on what you should do.
Ready?
Step 1: Tag every shared link with campaign tracking parameters.
This might seem slightly bizarre, but before you do anything email every person you know in your company, take your agency out for an expensive lunch, massage the right egos in your IT team, do whatever it takes to get every link shared by your company digitally to use campaign tracking parameters.
The reason for my recommendation is that mobile browsers are almost there in terms of their behavior when compared to desktop browsers, but they do sometimes exhibit weirdness when it comes to passing referrers. For mobile apps, there are no referrers (everything's Direct, hurray!). When you tag your email, paid search, display, organic and paid socially shared urls, you are going to ensure that your analytics solution will capture the source accurately (even in all those cases where there is no referrer).
So, be a dear. Always, always, tag every link (owned, earned or paid). It will have a material impact on the quality of your mobile analysis.

In Google Analytics there are five parameters: Source, Medium, Campaign, Term and Content. You should always use the first three, the last two are bonus. Check with the web analytics tool you use, every single tool allows you to do this, though each has a nuance you have to absorb.
Campaign tracking links have some differences depending on if they are going to your mobile website or your mobile app. Please see the detailed campaign tracking page for more info for websites, Android and iOS. Check out the helpful GA site URL builder and the Google Play URL builder.
When you analyze the data in Google Analytics (or Adobe or WebTrends or Webtrekk), this data will be in your Campaigns folder waiting for you to some pretty magnificent analysis.
Remember. Tag. Everything.
Step 2. Tag your mobile website. GTM FTW!
If anyone tells you that mobile analytics is hard, browsers are terrible, cookies are crumbling, phones use WAP (remember that?) etc., look at that person, give them a hug, then mark them in your book as enemy using red ink.
I kid a little of course. Use blue ink. : )
You can track mobile websites just fine with your standard web analytics tool.
According to Pew, while cell phones are nearly ubiquitous around the world (95% China, 91% Chile, 82% Kenya), penetration of smartphones are still getting there (37% China, 39% Chile, 19% Kenya). In the US, depending on who you ask, smartphones form around 65% of the cell phones.
It is important to realize that almost all monetizable behavior on mobile devices is on smartphones (everywhere in the world). So if with a few minutes of work you can track that, you have captured what's most important to your business. If you are in a feature phone majority market, and that market has most of the monetizable behavior, please invest in unique tracking there. If you are not… KISS.
Let me modify my earlier sentence: You can track smartphone usage of your mobile websites just fine with your standard analytics tool.
If you are starting new, or going to touch the code on your website for any reason, please, in the name of Thor, Superman and Dilbert, use a tag management solution. Enterprise tag management solutions are extremely feature rich now (they can even make you coffee), and hold the incredible promise of just having to touch your mobile site/app once and then being able to add more data capture sexiness remotely via the cloud.
Google Tag Manager is one such solution. It is free. And you don't even have to use Google Analytics to use it. It supports GA of course, along with AdWords, DoubleClick, GDN and many other Google tags. What is cool is that it is also pre-integrated with comScore, Turn, Media6 and other measurement and tracking solutions.

Sign up for your free account, set up a container tag, add that to a global element on your site and as soon as you add your GA Web Property ID, you are in business.
This will just give you the standard tagging. You would get the cool and sexy events, timers, rules and triggers, cross-domain tracking and all that. But for day one, standard will do.
One last request for you. Once you start collecting the data, please take half a day to identify all the micro-outcomes on your mobile website (remember this step for apps too), and ensure you configure them as Goals in your Google Analytics account's admin interface.
Mobile experiences are unique, to understand that one key strategy for us will be understanding the micro-outcomes.
So, use GTM, implement one container tag, turn on the standard GA tag, configure goals in GA admin, you are ready for a lot of mobile data analysis!
Step 3. Dive into Mobile Reporting and Analysis.
Analytics > Mobile > Overview > Devices.
That is all it takes for you to get a real solid feel for mobile's impact on your business. The very first thing you'll see will be the distribution between desktop, mobile (smartphones) and tablets. The team at Google has done a great job of having the end-to-end view there across Acquisition, Behavior and Outcomes.

What do you learn from this report? Mobile content consumption, behavior along key metrics (time, bounces etc.) and a sense for how much value you are getting from your mobile traffic. It will likely answer all the top level questions your boss has relating to mobile.
If you have ecommerce you will see key metrics related to money making. Transactions, Revenue and Ecommerce Conversion Rate.

What do you learn from this report? Mobile has much lower conversions and conversion rate than tablets or desktop. It does not matter if you are a B2B or B2C or A2K, you will always see this. This should give you two thoughts to frame to your boss. 1. Don't frame the value of Mobile in context of last-click on-device conversion. 2. Leverage Mobile (smartphone and tablets) to inform, assist and form deepen relationships with your visitors. (Remember my stress earlier on measuring micro-outcomes?)
Now that you have done God's work by making those two points clear, time to dive deeper. After all 50% of the traffic to the above site is on mobile devices, to get 10 times fewer conversions means there is something else of value that can be done.
Your boss will likely ask you which phones and tablets are used to visit your website, or other such value-deficient questions. For the first couple of interactions, give her/him that data. It is easy to find.

What do you learn from this report? Almost nothing. You will look at it only to please your boss for a couple of days. They will soon see that the values in the dimension (column one) rarely change, they will get bored and move on. Thank your lucky stars and stop reporting this.
(I've cropped off some of the not useful metrics in the standard GA report, at the minimum you see above what I think will be most useful in a not really useful report.)
One of the powerful reason for not using a dedicated mobile website analysis tool is that you can use the full suite of reports and analytical features inside your comprehensive digital analytics tool. You don't need anything special for this.
The best way to take advantage of this opportunity is to leverage the Analytics advanced segmentation feature.
You can choose the built in Mobile and Tablet Traffic segment, and apply it to any standard or custom report you want to analyze inside Google Analytics.

For example, I've invested an enormous amount of my budget in creating See and Think content, which I've properly tagged using the lovely event tracking feature in Analytics.
Apply the mobile segment to the event tracking report, and boom!

What do you learn from this report? In my case the interactive elements which are useful are clearly displayed above. In your case these might include videos people watch (See), car configurators people use (Think), handy guides they download (Think/Care), social amplification (Do) or follows (Care), and so on and so forth. This report is most useful in understanding if people are interacting with, usually, your most expensive investment in content and delivering some micro-outcomes.
Speaking of micro-outcomes (or micro-conversion if you insist), apply the same segment to your Goals Overview report in Analytics for some truly deep insights that you boss will love you for.

What do you learn from this report? How mobile traffic is adding value to your business. You've looked at ecommerce already and noticed some ecommerce is there, but not enough. So what else is going on? The stuff you see above. This report will help you identify where your company should invest in when it comes to mobile content and what kind of people to hire for the mobile awesomization team.
And how hard was it to get to the above? Standard GA tag > Goals in admin interface > Advanced segment. That, really, is all it takes.
You can, and likely should, do a lot more analysis of your mobile performance. Rather than having you jump around the many Analytics reports, I've created a destination where you can find pretty much all you need for your mobile report.
Let me walk you through the custom report, and I'll also share a link below to allow you to download the report directly into your GA account so that you can get jiggy with it right away!
The mobile custom report collects three key areas of analysis. Mobile device performance, search performance, and page performance serve as the three tabs on top of the report. In each case I've chosen the best fit metrics for that dimension (Users, Sessions, Bounce Rate, Time and Goal CR for devices) and added a custom set of drilldowns for each tab. This allows you to, for example start exploring performance at an operating system level, drill down to the devices for that operating system and then which landing pages perform best for that mobile device.

Here's the report for operating system. It shows clearly how you should at least be rooting for Windows Phone even as 40 awful pundits declare the platform a failure. For you, it delivers two percentage point higher conversion rate!

What do you learn from this tab? This is a better view than the standard report, but this tab is mostly for your boss (with mildly interesting data around engagement) and for your IT/Dev team as they decide which platforms to optimize the experience for or prioritize first.
The next tab is more fun/important, the search performance report. The drilldowns are Keyword, Source (engine) and Mobile Device Branding. You could do Source and then Keyword if you want, but I felt in this case the other way works much better. And again, a custom set of metrics.

First, a quick techie lesson. You are seeing (not set) above simply to indicate that 84,894 Users did not come via Search. Good contextual bit of data to have. You are seeing (not provided) above because Google, Yahoo! and others do not provide keyword data for users who do secure searches. [For more: Search: Not Provided: What Remains, Keyword Data Options]
What do you learn from this tab? How your search strategy is working for your company and what keywords and search engines you need to focus on in terms of improving your future mobile outcomes.
Finally, our third tab contains deeper insights around content. There are only four key metrics on this tab, and the drilldowns take us from the landing page to which source the traffic came from to aforementioned page and lastly did the type of user (new vs. returning) have an impact on performance.

What do you learn from this tab? Since mobile sites will have higher than average bounce rate, the landing page view is incredibly helpful in identifying both pages you need to improve (Bounce Rate, Time) and the pages you need to love a lot more (Page Value column). Once you find the most precious (or loser) pages, say #3 above, you can drill down to see which sources are delivering this valuable traffic – and you know what to do then. : )
Three simple tabs, your go to mobile analytics data without having to run around the tool. You can also download this report directly into you GA account: Mobile Performance Analysis Report.
I hope you can see that we don't have to go very far and wide to do very deep analysis of the smartphone and tablet users of our mobile website (or, eeek (!), desktop website on mobile platforms). Grab the report above, or apply the segment highlighted above and you will be more than on your way to creating a more data driven company.
Step 4. Dive into Mobile Reporting and Analysis.
Even if your business is mostly mobile app related and you barely have a five page mobile website, I recommend doing steps one through three first. There is enough complexity and enough simplicity above in implementation, reporting and segmentation that you will be a lot more familiar with what to do when it comes to ensure glorious glory for your mobile app analytics.
The cool part of app analytics with Google Analytics is that the tracking, reporting and analysis were re-thought from the ground up just for a mobile application environment. The downside is that this awesomeness comes at the cost of having to learn new things on top of my recommendation above. You do get really good stuff back, just look at it…

Real-time! OMG! (Remember, right-time is way more important than real-time.) Deep analysis of actual users, their location, their engagement with your app (how many open your app only once and never again, the answer awaits!), of course devices and crash reports, but more fun events and conversions and in-app purchases, and the pièce de résistance what traffic sources and campaigns actually drive people to your app page in the Play store! (Sorry, no iOS. Purely because of iTunes data sharing policies.)
Could you possibly want anything else?
Let's pull this a bit. How do you actually implement Google Analytics mobile app tracking?
Here's the world's greatest Mobile App Implementation Guide. Just follow the instructions. There are helpful links to the Android SDK or the iOS SDK along with the specific getting started guides.
When you are ready to grow beyond the default tracking (do that first!), you can implement event tracking in Android or event tracking iOS. Your next stop along the analytics ladder of awesomeness will be enhanced ecommerce tracking in Android or iOS . There is more. But should be enough to kill with smart insights.
If you have a mobile app, I also highly recommend using the Google Tag Manager because of the additional complexity of having to submit all changes to apps to the various stores, getting them to review and the added lag that comes with that love fest.
Once you leverage GTM… life becomes a little more simpler…

…and you have added flexibility in terms of making changes to measurement possibilities and having them propagate faster and bring your insights faster.
How can you get to this glorious stage?
Happy birthday: GTM Setup and Workflow: Mobile Apps

As with any analytics tool, your data capture journey can be never ending. Remember to stop at various stages (1. default 2. events 3. ecommerce 4. more 5. more more), and shift to data reporting, enjoy the data, realize you don't get what to do, shift to the data analysis phase (advanced segmentation, custom reporting).
Most people get stuck in DC, some make it through to DR, the bests spend time in DA because it is the only effort that yields the answer to: So what should we do?

There is a lot you can do (proof: details on all mobile app analytics reports). Here's my cheat sheet for you for initial focus of your mobile app analysis efforts.

1. Understand how many Active Users are interacting with your app. What does the 30-day active number look like?
2. Zero in on how Users discover your app, what sources and campaigns contribute to downloads and installs. Google Play Sources report, Play Referral Flow report.
3. Engagement, or death! Zero into Loyalty and Recency reports. You will see interesting patterns in the Goal Conversion column. Segment these reports as relevant to you.
4. Money, money, money. Dive into the outcomes reports to measure in-app revenue by day or a time period that makes sense. How many people are buying shiny swords and blunt gems?

That should get you going, and keep you with your first six months worth of analysis. Yes, there is a lot more, just scroll up three screenshots. Don't get lost… DC, DR, DA, money, DC, DR, DA, happiness, DC, DR, DA, glory.
If for any reason you don't want to use the custom mobile app analytics feature in Google Analytics, you also have other options.
Upsight (nee Kontagent) provides mobile app analytics, with a pinch of advanced segmentation (including sweet cohort analysis) and big data mining thrown in for good measure.
You can also look into Mobile App Tracking – it has a special emphasis on tracking marketing with a little less on detailed app behavior, segmentation and analysis.
Countly's claim to fame is that it is open source (a free version is available along with paid ones). You can also play with their mobile app tracking and data using their live dashboard/solution set. (Repeat, click on that link to play with actual data!)

Like other tools such as Analytics, they track events in great detail.
Here's an example of their tracking in-app purchases…

Their User Retention report is well worth taking a look at.
There are mobile app analytics solutions that also provide built in A/B testing and/or surveying capabilities and/or push notifications to individual users and/or capturing and analysis of personally identifiable information (PII) etc. Analyze your needs carefully, buy for now and the near future rather than for 2017, lest you spend all the time until 2017 on implementation, customization, reporting and re-implementation.
A new entrant worth checking out is Introspex, my friend Ali Hedayati is the Chairman.

It focuses on complete mobile customer experience management, with elements of the platform supporting developers, marketers and your tech/operations folks. The thing you might like the most is the no code instrumentation.
You have many choices. Free or paid. The one thing you don't have is an excuse to ever release an app into the world without the ability to collect the data you need to understand mobile marketing and consumer behavior.
Bonus: If you fall in the developer category and you would like to learn how to implement Google Analytics in an iOS app, please checkout Scott Sherwood's wonderful tutorial. Not only does he walk you through all the steps in detail, he also provides a clock app that you and download to practice!
Step 5. Implement Cross-Device Tracking.
Close to the holy grail, the ability to track and understand the behavior of the same person across devices, browsers, mobile and desktop applications and sites.
Holy grail is step five because it requires a great deal of thought – business and technology -, it requires business process re-engineering quite often, and time. If you are using Google Analytics, I encourage you to work with one of the Google Analytics Certified Partners who can help you do this quickly and efficiently in exchange for reasonable sums of money. If you are using a different analytics solution (this is where having the same analytics solution across your mobile, desktop, site, app, is incredibly valuable), please seek out their excellent consultants.
The Google Analytics team has made it easier for you to understand multi-device user behavior in a single view. With Universal Analytics, you can use the fantastic User ID feature to assign web behavior and outcomes to a single user across existences. [Since you are an over achiever, you are using Google Tag Manager for all this, Tracking User ID in GTM!]
The benefits of implementing real cross-device tracking is truly immense. I encourage you to use a consultant to help. If you want to go it alone, get a Red Bull and download this handy-dandy 62 slide Cross Devices Optimization presentation.
Step 32. Media-Mix Modeling/Experimentation.
This is going to be a drive-by (with a promise of a more detailed post in the future).
I've covered the value of media-mix modeling (nee. controlled experiments) in the reality check section of my detailed post on multi-channel attribution modeling.
Briefly, we don't quite understand how to allocate our media dollars optimally because often (if you don't do step 5 above), we don't know which people (as in People and not Users) were exposed to which media activity (AdWords, DoubleClick, Email, Facebook, yada, yada, yada). One way to overcome this issue is to use media-mix modeling to run tests and measure incrementality in results and attribute it to the optimal channel.

I love media-mix modeling. For advertisers with huge budgets it absolutely yields incredibly valuable insights.
It also involves insane complexity, huge costs to execute the experiments and a big commitment in time and people. Unless you are a massive site spending massive amounts on marketing and getting massive conversions, is rarely worth the revenue attribution you get.
But it is great for consultants, your internal people, and me.
So. If someone comes to delivering nirvana on the back of media-mix modeling (for mobile only or mobile and desktop and tv), take it with a grain of salt. At the very minimum, check that you have completed steps one through five above. If you have, and you are a big advertiser, embrace the consultant/internal person/me, cut a check and wait for glorious results.
I'm sorry for being a bit of a downer on something I love. It is important to be real though.
Ok, back to the happy zone.
Mobile devices – smartphones, tablets, wearables – present an amazing opportunity to engage our consumers at more times during the day, in more interesting contexts, and solve not just for a transient connection (BUY NOW!) but also a persistent connection. You can inform, entertain, and provide utility that delivers relevance, joy and delight. Guess what happens when it is time to monetize. You win first.
In this quest mobile data is your bff, and if you follow the five steps above you'll be ready to set your company apart from the wannabes.
I wish you all all the very best!
As always, it is your turn now.
At what step is your company's current mobile analytics effort? Is there one challenge that is most holding your efforts back? Which mobile site analytics tool do you use? How about mobile application analytics tool? Do you have a favorite mobile report – app or site? Have you had success implementing cross-device unique user tracking for your company? Got any tips to share with the rest of us on how we can do it faster?
Please take a moment to share to your delightful comments, insightful suggestions, and valuable examples via the comment form below.
Thank you.
P.S: Two additional posts you might find to be of value:

~ Mobile Analytics: Tracking Click-to-Call Mobile Ad Campaigns
~ Mobile Marketing 2015: Rethink Customer Acquisition, Intent Targeting

Diving into Data (Web Analytics) looking for key Insights / Advice of Avinash.

Diving into Data (Web Analytics) looking for insights.
Advice of Avinash:
Segmentation:

New Nirvana Rule: Never report a metric (even God’s favorite KPI) without segmenting it to give deep insights into what that metric is really hiding behind it.
The power of segmenting a metric is that you peek behind the curtain and find out more about the metric. These are the benefits that you will gain:
1.    It is impossible to segment any metric without putting in the effort to understand what we are reporting and the business value that the metric represents. This is hard work but what does not kill you makes you stronger. :~)
2.    Segmenting allows you to quickly hone in on areas of deeper dive from which will emerge key insights that will drive real and meaningful action.
3.    Our senior executives and decision makers don’t understand all the complexity and magic that is a web experience, showing them segmented trends is a extremely effective communication tool (and the best part is you barely have to talk, the picture will tell the story).
4.    You will earn a big fat bonus and promotion.

KPI Attributes:
1) What is good for Jane it might not be good for John. Every web business is unique !!
2) Do not go for perfection: Good is enough !!
3) Critical few baby, critical few !! Few KPIs:Most common: 3. Maximum: 4 KPI for each website analysis.
4) KPI Lifecycle (Half of KPI will die in one year), because your competitor become stronger. So apply Lean Six Sigma Process (Define, Measure, Analyse, Action, Improve or Eliminate). If your KPI do not give you insight, kill it, be brutal and kill it, and then define a new better KPI .

Each KPI must be:
1) Uncomplex
2) Relevant
3) Timely
4) Instantly useful

Do Not forget of Conversion Trinity
1) Relevant
2) Value
3) Call to action

Occam's Razor
by Avinash Kaushik

Best Web Metrics / KPIs for a Small, Medium or Large Sized Business

106 Comments | Print | PDF

We have access to more data than God wants anyone to have. Thus it is not surprising that we feel overwhelmed, and rather than being data driven we just get paralyzed. Life does not have to be that scary. In fact a data driven life is sexiest digital life you can imagine.
In this blog post we are going to bring the sexyback. I am going to attempt to significantly simply your life by recommending the critical few metrics you should use to analyze performance of your digital marketing campaigns and website. You'll be able to quickly go from "omg what can I do!" to "omg what am I going to do with all the money and fame I'm earning!"
The approach I'm going to use is to 1. Use my Acquisition, Behavior and Outcomes framework to ensure an end-to-end view of important activity and 2. Recommend metrics / KPIs you can use based on the size of your company.
Each recommendation comes with hints on what analysis to perform once you have the data, and what changes you could make to your campaigns, content and overall digital strategy. [A summary in pictorial format is at the end of this post.]
Excited? Let's do this!
Best Metrics / KPIs for Small Business Websites
Small business websites are a very fragile ecosystem. People working hard to do the best they can on the smallest possible budgets. But not to worry. They have to start with just four simple metrics to start rocking!
Acquisition:
Clicks? Visits? Backlinks? Impressions? No. We have something magnificent.
Cost Per Acquisition.
Obsess about this metric. You have very little money. You need to know, obsessively, what you get for it. This metric delivers that insight. Oh, and everything has a CPA (not just your paid search or display/banner ads). If you are doing SEO then you are likely paying for someone. That's the cost.

Kill things that don't have an optimum CPA. Invest more in ones that do. Simple enough, right?
Tip: Remember this is just cost, not profit. If your product costs you $15 to make then, in the above scenario, you are shipping a crisp $5 bill along with every Social Media order!
Where is it? Most likely in Excel. For Search it is in your Google Analytics or Omniture Site Catalyst reports. But for most other programs (Affiliate, Email, Social, Display) your Cost is likely sitting outside your web analytics tool. So extract the # of conversions, import into Excel, add a column for Cost, do the math, sing or weep (based on what the data says!:)).
If you are paying someone to do web analytics and this metric is not on top of the dashboard they've created for you, it might be time to say sayonara to them.
Behavior:
Page Views? Time on Site? No. You can do so much better!
Bounce Rate.
I continue to be a believer in trying to prompt love at first sight. Okay, okay, I'll settle for delivering relevance. :) Bounce Rate helps you identify campaigns where you might be targeting wrong people (who then come to your site and leave right away) or sending relevant traffic to irrelevant (and often flash-filled hideous) landing pages.
Bounce rate helps you find campaigns and landing pages that need to be killed / improved. Everyday.
Where is it? Standard metric in every web analytics tool worth anything. Look at your All Traffic Sources report and your Landing Pages report.
Checkout Abandonment Rate.
I find the fastest way to make money is to take it from the people who have already decided to give it to you. Obsess about checkout abandonment rate (the percentage of people who click Start Checkout to those who complete that process).

Focus on checkout steps with the highest abandonment. Tweak like crazy. A/B & Multivariate tests are a good option. But you are a small business… so just take away as many fields as you can, play with where to show shipping cost (I vote for way up front), reduce the number of checkout steps if you can, ask for account creation at the end of the process rather than at the start. Try, test, measure, be rich.
Where is it? In Excel. Or if you use Google Analytics: In Paditrack for free. (Google Analytics' native funnels are pretty sub optimal, ignore that entire feature.) For other tools: In KissMetrics. Create a funnel just for the checkout process (from clicking Start Checkout to Thanks for your Order) and both these tools will give you the metric automatically. They also allow you to segment the data! Make love to it.
[Bonus: What is abandonment rate?]
Outcomes:
My favorite Economic Value? No. As a small business I recommend…
Macro Conversion Rate.
You are a small business. Obsess about conversion rates, and everything connected to improving them. What products are people buying? Every single day (okay week) look at the All Traffic Sources report and seek out the Conversion Rate metric. Ruthlessly punish sources that are not working well and reward the pretty babies. Be they Earned, Owned and Paid media – oh and have a marketing strategy that has each of those elements or as a small business owner you are not going to win a lot.

I love creating an advanced segment with just the people who buy twice the average order size. I call them the Whales. Look at sources, locations, product bundles purchased, keywords and campaigns and all that to learn where/how you can find more Whales.
Where is it? Standard metric in all analytics tools. Remember to look at both the rate and the raw number of conversions for context. People make silly decisions when they don't do that.
That's it!
You are a small sized business and these four simple key performance indicators will literally rock your world as soon as you start measuring them. Cost Per Acquisition. Bounce Rate. Checkout Abandonment Rate. Macro Conversion Rate. Don't look at any other metric until you feel you've mastered them.
Tip: If you've hired the right analytics talent/consultant to help you, they'll be measuring these fabulous four.
Best Metrics / KPIs for Medium Sized Business Websites
What if you are a medium sized business? What key performance indicators are optimal for you?
First, you are going to measure the KPIs mentioned above. But because you are running a bigger and more complex business you'll also measure…
Acquisition:
CPA
+ Click-through Rate
While CPA is a macro metric about your campaigns' bottom-line performance, Click-thru Rate (CTR) is a deeper dive into analyzing the creativity and relevance of your affiliate deals / search listing / blinky banner ads.
In the context of Search (Paid or Organic), the text in your ads, the number at which your listing is ranked, the match between the user query and your ad's intent all help you receive a higher CTR. And if someone comes to your site (and does not bounce!) then you get an opportunity to convince them of your product or service's glory.

Small tweaks to the subject line of your email campaigns can have dramatic improvement in CTR. Recency and Frequency capping of your display remarketing campaigns can have a huge impact. Changing demographic targeting options in your Facebook ads can work wonders. Etc., etc., etc.
Put another way… CTR helps you understand if you showed up at the right place for your first date. Are you dressed okay. And if you are smiling the right smile. Helpful to know, right?
Where is it?
Everywhere. Start at a campaign level. Drill down to individual creatives. Kill badness. Promote goodness. Rinse. Repeat.
Behavior:
Bounce Rate
Checkout Abandonment Rate
+ Page Depth
A very tiny percentage of visitors to your site will see more than a couple pages. That's the internet for you. As you improve the user experience, information architecture and relevancy of content on your site, it is important to keep an eye not on the rather useless metric of Average Page Views per Visit or Average Time on Site but rather on the distribution of page depth. Here's how that picture might look like (from a post I wrote in July 2006!)…

From the deep detail reported by your web analytics tool you can choose to aggregate into buckets you most care about (like mine above). Categorizing the visits into Abandoners, Flirts, Browsers, One-off-Wonders, Loyalists will dramatically change your view of content consumption. Over time, as you move to deeper consumption, you'll see direct business rewards.
The above image emphasizes a sale/conversion at the end, but even if you are a content-only website improving Page depth helps you because more pages equal (at the very minimum) more ad impressions!
Where is it? The final table will be in Excel. If you use Google Analytics the data you need is here: Audience > Behavior > Engagement > Page Depth tab. If you use WebTrends, Yahoo! Analytics, Coremetrics please click around to find the data. They all have it.
+ Loyalty (Count of Visits)
If Page Depth helps you optimize for a single session experience, Loyalty helps you optimize pan session behavior. Put another way… how good are you at getting the same person to visit your website multiple times? For ecommerce or non-ecommerce websites, loyalty can mean the difference between life of survival and raking in profits like crazy.
First set a goal for the % of site Visits you would like for people who've visited more than x times. [Set a goal for x too. :)] For ecommerce websites use your Days to Conversion report (more on this metric below) to set your goal. For content sites perhaps mirror your content update schedule. If you are the New York Times and you update the website 24 times a day then should the average person be visiting the site at least 90 times per month?
Your BFF, as always, is analysis and not just reporting the metric. Create this simple segment in five seconds…

Apply to your keywords and campaigns and referring sources reports and identify which sources drive loyal traffic. Apply it to your content reports and figure out which content drives Loyalty (Sports? Op Ed? International? Cat Stories?).
Where is it? In every web analytics tool on the planet. If you use Google Analytics the data you need is here: Audience > Behavior > Frequency & Recency.
Outcomes:
Macro Conversion Rate.
+ Micro Conversion Rate
Pick your favorite benchmark and you'll notice that less than 2% of visitors convert. Focusing on just the Macro Conversion Rate means you don't care if you received any business value from the 98% that did not convert. I refuse to accept that uber-lameness.
Identify your Micro Conversions (/Goals) and obsess about the long and short term business value they deliver. You'll quickly realize the Economic Value they create for you is often far greater than the Revenue your Macro Conversion reports! And optimizing for that will ensure you win HUGE.

Where is it? In Google Analytics it is here: Conversions > Goals. Even if you are a content site the data is there. Details in the Goal URLs report. Setting up goals takes two minutes, setting goal values might take you a week (see measurement strategies here). If you use other tools, please check with your vendor.
+ Per Visit Goal Value
This Key Performance Indicator 1. helps you move beyond the obsession of focusing on the 2% (because it forces you to focus on Every Visit!) and 2. encourages you to create a business that uses the web to deliver multiple outcomes to your visitors.

Every visitor will not convert, but every visitor will, hopefully, deliver some Economic Value. Looking at this metric helps you identify Goals that contribute higher value, and and understanding of simple things like where you should focus on. If Twitter delivers 87 cents of Per Visit Goal Value and Google delivers 97 cents then perhaps I want to keep focusing on my SEO strategies rather than following the advice of the Social Media Guru who's just informed me Search is dead.
Where is it? In pretty much every single report in every single web analytics tool. Click on the Goals tab.
That's it!
For a medium sized business we ended up with nine metrics. Seems about right if you are making more than five million dollars of economic value. They key difference from websites that are in the small business category is that we are going to shoot for multiple conversions, deeper site engagement and better analysis of acquisition efficiency.
Time now to deal with the big boys and girls… large websites!
Best Metrics / KPIs for Large Sized Business Websites
Acquisition:
CPA
Click-through Rate
+ % New Visits
My choice of this metric perhaps betrays my refusal to rest on my laurels. There are clearly a finite number of people in the world relevant for any business. But staying hungry and staying foolish is a popular mantra for me. I use this metric to constantly calibrate my acquisition strategy to understand which inbound marketing efforts are bringing new "impression virgins" to the business.
If you look at your Earned, Owned and Paid media then this metric is especially important for your Paid media efforts. Except for your re-targeting / behavior targeting campaigns, you want your paid search, display, affiliate, and social efforts to bring new visitors to your franchise.
Where is it? It's like air, everywhere! Don't forget to segment for optimal analysis.
Behavior:
Bounce Rate
Checkout Abandonment Rate
Page Depth
Loyalty (Count of Visits)
+ Events / Visit
Every awesome large website delivers complex experiences (videos, demos, dynamic slideshows, configurators + + +) via sophisticated technologies (Flash, AJAX, Gadgets + + +). Almost all of the time we leave measuring their effectiveness on faith (or the HiPPO). I love event tracking because it helps us measure these often astonishingly, expensive initiatives.

Of 110,842 visits to the site, 9,054 interacted with your delightful experiences and each of those visits had 2.24 Events per Visit. Is that good? Bad? Could be better? Are these 2.24 interactions delivering higher economic value to your business?
In the above case the answer was a big NO. In your your case you'll decide based on your strategy and goals. At the end of the analysis you'll make significantly smarter decisions about your content (especially because the Analysis Ninja that you are, you'll triangulate performance of this metric with first, Page Depth and, second, Loyalty).
Where is it? Most web analytics tools do some type of event tracking. Please check with your vendor (it might not be called event tracking in their lingo, just describe my first paragraph above). In Google Analytics the data is here: Content > Events.
Outcomes:
Macro Conversion Rate.
Micro Conversion Rate
Per Visit Goal Value
+ Days to Conversion [or Time Lag for Content sites]
Another pan session metric I adore.
Life, no matter how hot you are, is not a series of one night stands. Yet because of how they analyze the data most companies end up optimizing their web marketing campaigns for one night stands. Come here and convert NOW! If yes: Oh, I love you. If no: Kill the campaign!
That approach is not just short-sighted; it is an insult to your visitors. Convert them at a pace they are most comfortable with. This metric helps you understand how quickly or slowly your visitors convert. You can, at the very minimum, change your campaign messaging and come hither calls to action and adjust your landing pages. If the Days to Conversion are much longer, then create a robust (slow dance) micro conversion strategy.

If you have a non-ecommerce website then there is something delightful for you in the Google Analytics Multi-Channel Funnel reports. Checkout the Time Lag report . It is showing you exactly the same data as the Days to Transaction for Ecommerce sites. The metric you see immediately above is called Conversions. It is essentially your Goals (/micro conversions).
Optimize your "hello, nice to meet you, what would you like, here is what I have to offer, why don't you check with your spouse, come back and check it out again, multiple times, I'm still here, you ready to convert / deliver economic value, here's how… " process.
Where is it? Days to Conversion is in the Ecommerce section of your web analytics reports. It is a standard report. (Don't forget to segment your sources. Deep insights await.) Time Lag may or may not be a standard report in your tool. Please check with your vendor. In Google Analytics it is a standard report here: Conversions > Multi-Channel Funnels > Time Lag.
+ % Assisted Conversions
This is the newest metric I've made standard for all my clients / partners / BFFs. And it is a sweetie.
Assisted Conversions builds on the above mental model. It takes a while for a majority of your visitors to convert (macro and micro conversions), so why does almost all of web analytics focus on single channel analysis and optimizing that single channel in a silo? Just because the Affiliate click was the last one before conversion should it be optimized for that conversion? Especially if the Visitor originally came via Facebook (or Google or whatever)?
How many of your conversions had more than one ad / media / marketing touch prior to converting? Really smart Analysts at really successful companies understand that…

…and then use that data to optimize the portfolio of channels rather than individual channels for the company.
Even if you don't do portfolio optimization (and desperately hope you do) you can easily see how the above data will cause you to execute a different marketing optimization and expectation strategy for Email (1.18 Assist / Last Interaction rate) vs. Organic Search (0.61).
I am being modest when I say that this metric and subsequent analysis will have a fantastic impact on your company.
Where is it? % Assist Conversions may or may not be in your web analytics tool. Please check with your vendor. In Google Analytics you'll find it here: Conversions > Multi-Channel Funnels > Assisted Conversions.
And we are done!
For large businesses we've identified 13 key metrics that would give a robust end-to-end view of business performance. The key difference vs. medium sized businesses is that we are really, really, really focused on pan-session (multiple visits) behavior. Put another way, we really care about people here and not just a single visit.
Here is a summary of the metrics I am recommending in this post…

I hope the picture above will quickly help diagnose where current gaps in your measurement strategy might be.
Additionally if you are a small business you'll know what else to measure when you start to become medium sized and if/when you cross that threshold you'll know the metrics that come with your large business status. :)
You'll notice that I'm not focusing on KPIs like AdSense Ads CTR or Page Load Time or Actions per Social Visit or Search Exits (I love this metric!) or Content Distribution vs. Content Consumption Rate or Conversation Rate (in case of a content site) etc. That's simply because these KPIs tend to be unique to the type of business you are running. My strategy above was to focus on just the KPIs that would be applicable across all types of businesses.
That brings me to a very important point.
While it is my hope that you'll find my recommendations above relevant and yummy… the most optimal way to identify that best key performance indicators for your company will come using the process and structure outlined in the Digital Marketing & Measurement Model.
I'll end with the thought I started this post with… we have more data than God wants anyone to have. But web analytics does not have to be scary or impenetrable. Use the roadmap above, focus on all three elements (acquisition, behavior, outcomes) and I promise you'll soon be on your way to being as happy as God wants everyone to be.
I wish you all the best!
Okay as always it's your turn now.
Does your business use the above recommended metrics / key performance indicators? Do you have an absolute favorite metric that's not mentioned above? Which metric above do you find most useful? Which one most useless? What is your strategy for identifying the most relevant metrics?
Please share your suggestions, critique, and helpful best practices via comments.
Thank you.
PS:
Couple other posts on metrics / KPIs you might find interesting:

• Web Metrics Demystified
• Your Web Metrics: Super Lame or Super Awesome?
• Kill Useless Web Metrics: Apply The "Three Layers Of So What" Test

Like this post? Share it:

Occam's Razor
by Avinash Kaushik

Produce Actionable Insights: Mate Custom Reports With Adv Segments!

74 Comments | Print | PDF

99.9996253% of Web Analytics reports produced are utterly useless.
Partly because of a lack of any tie to business strategy (ensure you have a Digital Marketing & Measurement Model!), partly because they are out of the box standard reports that web analytics vendors create for “average” people (and we both know that you are not average!), and partly because all they do is present data in the aggregate (a punishable criminal offence if there ever was one!).
As a cure to this malaise, I’ve encouraged y’all to switch to using only custom reports (bring just relevant metrics and dimensions into one place, and throw away 90% of web analytics data that does not apply to you). Here’s a blog post: 3 Awesome, Downloadable, Custom Web Analytics Reports
Read the post and go from “OMG there are so many reports and I don’t know what to do with them” to “OMG I can’t believe just five reports give me 90% of what I need!”
My second prescription was to (repeatedly!) pimp the value of advanced segmentation. I can’t even think of two other things that are quite as life altering as segmentation for an Analyst and/or a Digital Marketer. You go from looking at amorphous globs of goop to a crystal clear understanding of people, their sources, onsite behavior and business outcomes. Here’s my most recent blog post (with downloadable segments): 3 Advanced Web Analytics Visitor Segments: Non-Flirts, Social, Long Tail
Read the post and go from “Arrrhhhh this web analytics is so haaaard” to “Awww… being data driven is so much fun!”
Despite the obvious and incredible advantages, precious few in our dear community have their web analytics existence centered on custom reports and segments. Here’s the hashtag: #heartbreaking
If that is due to the lack of a relevant example to enchant you with its native sexiness then let’s fix that problem. Here it is…

Enchanted?
No?
Okay here is a higher resolution version: Page Efficiency Analysis with Long Tail Advanced Segments
Now?
Maybe not yet. Okay let me break down the components.

Enchanting Analysis: Rule 1: Business Context.

Surprised that I am not jumping into telling you about metrics and dimensions and segments and all the other things that bring Analysts goose bumps? Remember all data is secondary. Your primary quest is to understand the business context, which in turn will dictate procurement of data.
Why this Custom Report?
We produce a ton of content on our site. Which content is most engaging? Which subject matter experts should we hire more of? What type of content (videos, demos, pictures, reviews etc.) do visitors value more? Which content delivers business or non-profit value?
These are very important questions to answer, and with content reports fragmented in tools like Google Analytics and, worse, metrics spread out across reports (or outright hidden), it is hard to answer those questions easily. That’s where my Website Content Efficiency Analysis custom report comes in. It hopes to bring all the key metrics into one place (no more hunting and pecking!) and help you analyze site content optimally. Log into Google Analytics and download the report here: Website Content Efficiency Analysis Report. The report you'll download is a V2 version from what you see in this post. It is more improved with a new technical analysis tab!
[If the report opens in a profile other than the one you want the report in, just scroll to the bottom and in the Profiles section you'll see Additional Profiles, click on the drop down, make your choice, hit Save.]
Why this Advanced Segment?
One of the key ways in which we get relevant traffic to our websites is to have them properly indexed by Google / Bing / Baidu / Yandex. Search Engine Optimization is key. But not just SEO or keywords or our brand name. It drives me bananas how much we still talk about words. Few people search with a word or two. People type phrases into search engines – some of them write entire stories – and those phrases and stories account for an enormous amount of traffic to your site.
[rant] It is so horrible that SEOs still get asked: “Can I rank #1 for word x?” Good lord! Wake up! [/rant]
So two relevant questions come up: How are we doing for the head (few, brand) words? How are we doing for the super important long tail search phrases?
Log into Google Analytics and download these two segments: 1-2 Word Search Query Visitors, 3 or More Word Search Query Visitors.
[If the segments open in the old version of Google Analytics simply open the new version in a new browser tab and copy the segment into the new version.]
We are not doing all this analysis because it makes us happy. It is mandated by the business reality described above. Always, always, always be a slave to business strategy/needs. Let that drive analysis. Don’t puke data out of Omniture/WebTrends and go looking for business problems to solve. Please.
Why Apply these Segments to this Report?
With the above two in hand you are now ready to answer the killer question: How are different pieces of site content doing in terms of SEO in driving visitors via head words and tail phrases, and is it delivering business value?
OMG! OMG! OMG!
Yes! Think of all the possibilities. The ability to focus people in your company on the right content creation. The opportunity to balance efforts between head and tail phrases. The chance to understand what is actually driving business value (you know: the thing that pays your salary!).
This is why as little children we told our moms we wanted to grow up and become Web Analysts. :)
Let’s break down various components of this enchanting bit of analysis. . .

Enchanting Analysis: Rule 2: Establish Macro Importance.

It is possible that I was completely wrong about doing this analysis. Before you waste precious hours of your life (and even more precious hours of your management team / client) always look at a 10,000 meter level view to see if there is any there there.
So that is what we do first, and here is that simple, yet absolutely critical, piece of data. . .

First you will notice that I was right!
Back up a little bit. What we are looking for is whether this analysis is important to do. My proxy for that is how much website traffic are we talking about? Often we end up obsessing about a keyword or campaign or xyz without realizing we are talking about 0.05% of the traffic. Sub optimal.
To the being right part… and this is where the Google Analytics UI is simply brilliant…
In a flash and a bang I can see that my first head segment (one or two words typed) accounts for 13% of the site traffic (yowza!) and the long tail segment (three words or more typed) accounts for an awesome 24% of the site traffic (zoowee mama!).
We’ve established that the analysis we are doing is worth doing, that the long tail is worth focusing on, and any insights we can find will have a material impact.
Life Lesson: In life, before you get too deep into any analysis, use a barometer to establish the work is worth doing. How much desirable traffic or desirable outcomes does your report represent? Be explicit about it and your boss will pay attention to your reports / analysis.

Enchanting Analysis: Rule 3: End-to-End view, or Death.

Very early in your analysis you want to establish a view of the dataset that gives you the end-to-end view of performance. The lack of this view is why I am so critical of standard web analytics reports. You can tickle GA, you can twitch Site Catalyst, or you can rub WebTrends just the right way and find the data. But they all conspire to work against you by not giving you want you want.
Why wait for the vendor? Pull all the data you need (and honestly only you know what you need) into one place. Don’t be satisfied with a report that just shows Visits and Bounces or Time on Site and Load Time or Goal Completion or (worse) %Exits or …. one of the many data distractions so liberally available.
Here is how I define end-to-end… Metrics on your report should give you clear understanding of your performance in these three areas: Acquisition, Behavior, and Outcomes.
So often we tend to obsess about acquisition (impressions and clicks and visits and abc), and sometimes we care about outcomes (revenue and conversions and xyz). It is rare that we care about behavior. You need all three.
In this report my acquisition metrics are Entrances and Unique Visitors – how many people came, how many entered on this page, and with a glance at both I get an idea of how many might have come multiple times.
My behavior metrics are Bounces, Pageviews, Avg. Time on Page – how many people choose to leave right away (“I came, I puked, I left”), how popular the page was for all visits, and how long do people stay on the page (if they stay on the site).
Here’s how that part of the report looks. . . 
 
Already you can start to see wonderful patterns in the data, and since you are not looking at the data in aggregate you can start to see how the two different groups behave.
Once more marvel at the approximately 2x more traffic you are getting from your long search queries people type into Google. Because you have behavior here as well you can start to notice the differences between behavior of the two groups (a lot more page views for the smaller head visitors, but significantly less time on site!). You can start to draw conclusions about the value of the tail.
While you are already doing better analysis because you are looking at Acquisition and Behavior together, this picture is of course incomplete. We are missing outcomes.
Should we be even more crazy about the tail folks since they spend so much more time, and there are so many of them? Perhaps.
Why not look at the bottom-line.
Our outcome metrics here are Per Visit Goal value (how much value each unique visitor adds to us each time they visit) and Total Goal Completions (for when you have multiple goals –and everyone should have macro and micro conversions!). Here’s the rest of the picture. . .
For the sake of clarity I am showing just the behavior and outcomes here; the report has all three in one place.
Delightful, is it not?
While the tail traffic does very well, 14,400 Unique Visitors delivering 3.057 Goal completions, the Per Visit Goal Value is significantly lower than the head traffic, 88 cents compared to 197 cents for the head traffic. So you can double your traffic, but the site is monetizing this traffic for a lot less than the head traffic.
So is the traffic any less valuable? Should you still invest in SEO for the long tail? What is the difference in the types of Goal Conversions between these two groups? What content is driving each set of behaviors? This and all other questions you’ll answer in the next steps. For now the job was to simply get an initial burst of solid starting points from the end-to-end view of the key metrics.
Life Lesson: Friends don’t let friends have reports that are missing one of these three elements in the metrics being presented: Acquisition, Behavior, Outcomes. Choose the metric(s) that is most optimal for you in each bucket, but if your reports don’t have all three buckets. . . rethink why you even have them.

Enchanting Analysis: Rule 4: Look for Surprises, Love.

This is very important. When you analyze data your defacto mode should be to look beyond the top ten rows of data. You should look for things that surprise you. Far too often we look for things we are looking for and we move on. Totally sub optimal.
In this step we’ll analyze the dimension we care about – website content (pages) – and answer the questions above. Which content acquires more head and tail traffic? Which content drives more conversions and business value? Which content does this and which one does that and which one drives deeper engagement and which one drives more bounces and is it only for the head or the tail phrases and which one. . .  so many lovely things you can dive into easily.

If you are using Google Analytics for analysis of an ecommerce website you can easily add Keyword as an option for the drill down. I've created that version of the report for you, click here to download it into your Google Analytics account: Content Efficiency & Keyword Drilldown Ecommerce Report. You also have the ability to filter to phrases and keywords that contain a word(s) that you are most interested in analyzing first. Finally, you can also create your custom report such that it drills down into the keywords used to get to that page (head or tail).
Any of the above methods will allow you to dive deeper and look at the actual keywords and key phrases easily. Now you can start to understand things to love in your SEO efforts and drink to the sad situations you’ll surely find.
And there is more. Rubber meets the road in this step. It is raw analysis that you are going to be doing here. Have a notepad next to you and jot down both the obvious insights that will drive immediate response, and write a love letter to your SEO and blow their brains about how many things you find easily in your data.
Life Lesson: Analysis is hard. Smart people do a lot of it. 

Enchanting Analysis: Rule 5: Create a List of Prioritized Actions.

When you apply relevant advanced segments to your custom reports you are going to find a lot of actions to take for your business leaders, frontline marketers, SEO contractors, etc.
Rule #5 is going to ensure that something is actually going to be done as a result of your blood, sweat and tears. Never give a long laundry list of “thing to do.” Ever.
Rather, convert the jumble of actions into a numbered list with the first item being the highest priority, then the second highest priority, and so on and so forth.
You are the person with most access to data and, thanks to rule #4, the person who analyzed the heck out of it. So now use that to create the numbered list.
How do you identify item number one for your list? This is where real Analysis Ninjas distinguish themselves, and leave the Reporting Squirrels in the dust. Compute the impact of each of your recommendations. If you paid $a to SEO for long tail phrase zxy it will add value $q. Use performance of existing words / visitors. Look at past performance. Look at competitive data. Make guesstimates (in the worst case scenario). But compute impact.
[Bonus]
For one specific and incredible way to compute impact please see item #8 in this post: Barriers To An Effective Web Measurement Strategy (+ Solutions!) The methodology outlined, illustrated below,…

…is exceedingly effective at showing the value of the action you are recommending, and the cost of the delay in implementation of the changes! This will allow you to make a very very effective prioritized list of actions.
[/Bonus]
Business leaders simply have a much, much easier time internalizing a numbered list (less thinking for them) and approving actions (more fame for you).
Life Lesson: Ninjas never submit long globs of text as observations or random actions. If you can’t prioritize, you’ve missed the most important step that creates data driven businesses.
You can see how deeply passionate I am about custom reports and the ability to truly find magnificently impactful things by applying advanced segments to them. I hope that the above example will make you pass all your current reports through the filter of Acquisition, Behavior and Outcomes. I hope you’ll never again look at any report without segments applied to them.
Good luck!
As always it is your turn now.
What is your favorite combination of a custom report and advanced segment? Do all your reports cover acquisition, behavior and outcomes? Why not? Got best practices to share for metrics that fall into each of the three categories? Do you have a segment you want to report on but your analytics tool does not allow it? How is your business doing when it comes to head performance and long tail performance?
Please share your tips, feedback and ideas via comments.
Thanks.

Like this post? Share it:

Occam's Razor
by Avinash Kaushik

The Difference Between Web Reporting And Web Analysis

86 Comments | Print | PDF

Someone asked me this very simple question today. What's the difference between web reporting and web analysis?
My instinct was to use the wry observation uttered by US Supreme Court Justice Potter Stewart in trying to define
po rn: "I know it when I see it."
That applies to what is analysis. I know it when I see it. : )
That, of course, would have been an unhelpful answer.
So here I what I actually said:

If you see a data puke then you know you are looking at the result of web reporting, even if it is called a dashboard.
If you see words in English outlining actions that need to be taken, and below the fold you see relevant supporting data, then you are looking at the result of web data analysis.

Would you agree? Got an alternative, please submit via comments.
I always find pictures help me learn, so here are some helpful pictures for you. . .
This is web reporting:
And so is this, even if it looks cuter:

And while you might be tempted to believe that this is not web reporting, with all the data and the colors and even some segments, it is web reporting:

See the common themes in all the examples above?
The thankless job of web reporting, illustrated vividly above, is to punt the part of interpreting the data, understanding the context and identifying actions to the recipient of the data puke.
If that is your role, then the best you can do is make sure you have take the right screenshots out of Site Catalyst or Google Analytics, or charge an extra $15 an hour and dump the data into Excel and add a color to the table header.
So what about web analysis?
The job of web analysis mandates a good understanding of the business priorities, creation of the right custom reports, application of hyper-relevant advanced segments to that data and, finally and most importantly, presentation of your insights and recommended action using the locally spoken language.
See the difference? It's a different job, requires different work, and of course radically different skills.
Examples of web analysis? I thought you would never ask. . .
This is a good example of web analysis:

[And not only because it is my work! Learn more about it here: Action Dashboard.]
Notice the overwhelming existence of words. That's not always sufficient, but I humbly believe always necessary.
When you look to check if you are looking at analysis or reporting look for Insights, Actions, Impact on Company. All good signs of analysis.
Here's another example of really good web analysis:

[Click on the image above for a higher resolution version.]
Ignore how well or badly the business is doing. Focus on approach taken.
Here are some things that should jump out. . . . A deliberate focus on only the "movers and shakers" (not just the top ten!).  Short table: just the key data. Most of the page is taken up with words that give insights and specific actions to take.
Another example that I particularly like, both for the style of presentation and how rare it is in our world of web analytics. . .

[Click on the image above for a higher resolution version.]
No table, no rows, no pies. And yet data holds center stage with clearly highlighted actions.
Normally, we all do the column on the left (it might look different, but we have it). Unfortunately we don't appreciate is the power of the middle column ("segmentation reveled"). That is super important because it gives the recipients exposure to the hard work that you have done and in a very quiet ways increases their confidence in your work. Guess the outcome of that? They take the actions you are recommending!!
Analysts constantly complain that no one follows any of their data-based recommendations.  How do you expose your hard work? In a garish Las Vegas show girl fashion where all the "data plumes" are, unsexily in this case, hanging off the body? Or, in quite concise ways? Only one of those two work.
One more? Okay here you go. . .

[Click on the image above for a higher resolution version.]
Diana has loads of observations, supported by visuals (sometimes it really helps to show the search results or the emails or the Facebook ad) with highlights (actually lowlights) in red, and finally recommendations.
And note the tie to outcomes (another common theme in all examples above). In this case, the search improvements are tied to the increase in donations I can make because of sales of my book. 1.5 extra smiles per month! (All my proceeds from both my books go to charity.) A good way to get attention from the "executive" and get him or her to take action.
Do that. A lot. Be creative. Yes it is hard work. But then again glory is not cheap, is it?
Exceptions to the rule.
Not every output you get from your Analyst, or "Analyst" :), with loads of words on it, instead of numbers, will be analysis. Hence my assertion that "I know it when I see it." Words instead of data pukes is just a clue, read the words to discern if it actually is analysis or a repeation of what the table or graph already says!
In the same vein not every output that is chock full of numbers in five size font, with pies and tables stuffed in for good measure, is a representation of web reporting. It is hard to find the exceptions to this rule, but I have seen at least two in nine years.
Top 10 signs that you are looking at / doing web analysis.
Let's make sure this horse is really and truly dead by summarizing the lessons above and using a set of signs that might indicate that you are looking at web analysis. . .

#1. The thing that you see instantly is not data, but rather actions for the business to take.
#2. When I see Economic Value I feel a bit more confident that I am looking at the result of analysis. Primarily because it is so darn hard to do. You have to understand business goals / outcomes (so harrrrrd!) and then work with Finance to identify economic value, and then you have to configure it in the tool and then apply advanced segments, and then figure out how things are doing. That is love. I mean that is analysis! Or at least all the work that goes into being able to do effective analysis.
#3. In the same vein, if you see references to the Web Analytics Measurement Model (or better still, see it in its entirety on one slide up front), then you know that the Ninja did some analysis.
#4. Any application of algorithmic intelligence, weighted sort, expected range for metric values (control limits), or anything that even remotely smells of ever so slightly advanced statistics is a good sign. Unknown unknowns are what it's all about!
Also mere existence of statistics is not sufficient. All other rules above and below still apply. :)
#5. If you see a Target mentioned in the report / presentation, then the Analyst did some business analysis at least. See the top right of the picture immediately above.
#6. Loads and loads and loads of context! Context is queen! Enough said.
#7. I have never seen web analysis without effective data/user segmentation. I think this statement is in both my books. . .  "All data in aggregate is crap." Sorry.
#8. If there is even a hint of the impact of actions being recommended then I know that is analysis. It is hard to say: I am recommending that we shift this cluster of brand keywords to broad match. It is harder to say: I am recommending. . . and that should increase revenue by $180,000 and profit by $47,000. Look for that.
#9. If you see more than three metrics in a table you are presented with then you might not be looking at analysis.
#10. Multiplicity! If you see fabulous metrics like Share of Search (competitive intelligence) or Task Completion Rate (qualitative analysis) or Message Amplification (social media) then they are good signs that the Analyst is stepping outside Omniture / WebTrends. I would still recommend looking below the surface to ensure that they are not just data pukes, but the good thing is these are smarter metrics.
User Contributions:
#11. From Carson Smith: If someone looks at your analysis / report / presentation / dashboard and has to ask "and… as a result?", then it might be reporting. What happened should be obvious.
[I love applying the "Three Layers of the So What" test to any analysis I present or see. I ask "so what" three times. If at the end of it there is no clear action to be taken then I know it is just web reporting, not matter how great it looks or how much work went into it. Ask "as a result?" or "so what?" to your work!]
#12. From Chuck U: 1) If it can be automated, it's probably not analysis 2) If your data warehouse team says they can automate it for you, then it's definitely not analysis. [#awesome! -Avinash]

Can you think of other signs? Please share your suggestions via comments. I'll add the best ones to this list.
In the list above, and in the examples in this post, you see my clear, and perhaps egregious bias for business analysis and business outcomes and business actions and working with many parts of the business and business context. But I've always believed that if you and I can't have an impact then why are we doing what we do?
I hope you've had some fun learning how to distinguish between web reporting and web analysis. It is a fact of life that we need both. The bigger the company, the more they want data pukes, sorry, reporting.
But if you have "Analyst" in your job title then you perhaps now have a stronger idea of what is expected of you to earn that title. If you have hired a "web analysis consultant" and are paying them big Rupees then you know what to expect from them. Don't settle for data pukes, push them harder. Apply the rules above. Send their "analysis" back. Ask for more. Raise your expectations!!
I hope now "you'll know it when you see it," and have more datagasms!
Okay, it's your turn now.
How would you answer the question about the difference between web reporting and web analysis? What signs do you look for when evaluating the work of your Analyst or Consultants?
Please share your thoughts via comments below.
Thanks.
PS: In case you are curious here's the current official definition of po rn, as outlined in Miller v. California:

(a) whether the 'average person, applying contemporary community standards' would find that the work, taken as a whole, appeals to the prurient interest,
(b) whether the work depicts or describes, in a patently offensive way, sexual conduct specifically defined by the applicable state law, and
(c) whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value.

Like this post? Share it: