All posts by Theodor

My view of the Conversion Conference & Emetrics in San Jose, USA, 2010

I had the luck to attend the first conversion conference held in San Jose, USA, during the time 3-6 of May, 2010. The conference was in conjunction with Emetrics that has been around for a while so there were plenty of tracks to attend. I think in total there were somewhere over 100 tracks!

The theme of the conversion conference was to explore various aspects of website conversion such as videos, ab testing, copy writing, usability etc. The Emetrics part is more about web analytics and less on the conversion part but of course they overlapped a bit.

I and my colleagues Rickard and Jenny traveled all the way from small Stockholm to San Jose, the capital of Silicon Valley to see and hear the most prominent people within the web analytics field (a 16 hour trip…very tiresome). Unfortunately you have to travel that far if you want to listen to these people at the same conference. You rarely get this chance in Europe.

The attendance on the conference came all over the world. There were people from Germany, Israel, South Africa, Poland and another Swede.

The feeling I got was that most people there were some sort of consultants. I think less than 50% of the attendants were from end user companies. I was also surprised to meet so few big company representatives. Maybe they don’t do web analytics or they already know everything.

So, if I do a short recap of the conference here are my high and low points.

The high points

  • Many tracks. A lot to listen to. As usual when you have such great variety the quality varies a lot but it was good to be able to choose what you want to listen to.
  • The venue. It was held in The Fairmont Hotel, a four star hotel with large spaces and large rooms for such a conference. Unfortunately some of the rooms were small and got filled quickly. Also the air condition killed me some times.
  • It was well organized. Everything run smoothly from the various sessions to the lunch breaks run smoothly.
  • Famous speakers, top of the line. Some of the most famous people within the field was there as speakers. For example the much famous Mr. Avinash was there.
  • Probably around 500 people or more attending both conferences. It was nice to see such large turnout, which indicates that people are getting more and more conscious of what web analytics can do for them.
  • Networking. Like all conferences you get a chance to network, and I got to listen to various peoples “problems” which was interesting. I recognized many of the problems – like handling a vast amount of keywords or doing AB testing.
  • My top 3 sessions.
    • The presentation on how videos increase conversion if you do it right. From my own experience I know that videos can increase conversion but the two guys presenting was humorous and very down to earth. They also showed some interesting examples that can be used in various cases.
    • The session about sharing knowledge within the organization was interesting to listen to.
    • The Expedia presentation. Expedia must most likely have to most advanced web analytics and conversion program out there. They use software and manpower to maximize their efforts in various areas. It took them three hard years to get where they are but I bet they got their investment back with interest.

The low points

  • Over half of the tracks were too basic from my point of view. Of course various people experience this type of conference differently but I think many of the tracks were to shallow for my taste.
  • Too many consultants presenting and they tend to have a tone of voice that tells you that their findings is the rule. Especially the more famous ones were more or less preaching which I think if you are a bit experienced within the field you see through it.
  • Too few company presentations. I like company case presentations. I think they give you a good idea on how you can solve your own problems while watching the likes of others. Unfortunately there were too few company case presentations. I would like to see much more but I guess they are hard to find.
  • Many of the presented cases by the consultants did not hold to scrutiny. On questions from the audience many of the cases had not been implemented and were rather a suggestion  from the consultant to their client, so the outcome had not been documented.
  • I am not sure if this is a bad thing but the knowledge on conversion among the audience seem to be very low judging by the questions and the twitter chat.

Conclusions

I think this was the best conference I have attended so far. Would I go again? Yes, no doubt about it. That it was held in Silicon Valley was very interesting, I got the wibe that if you work with IT business development this is where you should be.

I keep asking myself how far along are we at Web Guide Partner on conversion and web analytics issues? Well, I truly think that we are among the most advanced companies out there but not on everything. We have explored most areas and have a broad spectrum of knowledge but we lack the marketing part such as ppc and offline advertising. Also our experience with social media is halting. The vast amount of websites we work with also limits us to the amount of deep diving we can do.

All in all though I am glad I attended the conference, it makes me a smarter web analytics and conversion guy, hopefully we can use some of the knowledge on our own websites.

Web Analytics maturity as I have experienced it

I wanted to write this post to reflect over the experience that I have had so far over web data maturity on an organizational level. To become a data driven organization is not the easiest thing in the world but reaching this level though will lead to success.

The figure below shows how I have experienced the data maturity process during my time as a Web Analyst.

Panic, panic we need to measure now!

We need to measure our online activity – The first step is when people start to realize that they are so big that they need to measure their online activity. This comes usually as a realization when upper management start asking some of the more basic questions such as, what do people do on our site. This when they install a web analytics tool.

 

Is this data true or are they trying to trick me?

Can we trust the data? – The second step is to ask about the data. Can we trust the data? Is this data accurate? What does unique browser mean, why are these not the same as unique IP from our log servers? This type of questions is typical in the second phase as I have experienced it.

 

I am so smart if I have many reports

We need reports lots of them – When people feel satisfied with the data validation they realize they need lots of reports. So the Web Analyst guy becomes a reporting guy. I can assure you this is a big waste of time of the web analyst competence. All reporting should be done automatically since there are so many great software out there that will do it more accurate than letting a person to do it.

 

This Web Analyst sucks! He gives me so much data I feel dumb

What should I do with all the numbers? -Now you have all numbers you could ever wish for. Actually you have more numbers than you can handle and you are asking yourself why you ever hired a Web Analyst in the first place. Relax, first you need to realize that less data is better. Depending on how experienced your web analyst is ask him to only give you the minimum amount of data that is valid only for taking decisions on. Ask him also to give you his input on what to do since you will move a lot faster to the next level.

 

Wait a minute, maybe I should keep my Web Analyst, maybe I should promote him

Now I understand the numbers, how do I act? – The aha-experience hits you and you know how to interpret the numbers. You start asking the right questions and now you want to do something to improve your web business. You start looking at optimization.

 

My Web Analyst is gold worth, now I can get that promotion and bonus I was promised

How do we optimize? – Optimization is the nirvana of web analytics; it gives you the tool to do act. You start doing AB testing. This a small field in itself and there are several steps involved to have a good program in place but if you get here, you should be happy.

 

My boss said he promote me only if I can make our whole organization more effective

We need to coordinate our actions – Now that you feel comftable with split testing you move your horizon on an organizational level. So far usually only a few people within the organization reaches this level. The next step, which is rather hard, is to make your whole organization harvest the insights of good web analytics and AB testing.

 

I got the promotion now I sit on the board of directors helping them make strategic decisions based on the data

Let us use our data for strategic decisions – When you accomplish the previous task, which might take a long time to do, the insight will reach high level persons, people that want to use the data for strategic purposes, such as acquisitions and business development. This level is where you want to be in, only now you can title your organization as data savvy.

 

Now everyone needs the data all the time

How do we make the data on demand? – The next natural development is to make business critical data on demand. Each position needs their own set of data and therefore the web analyst plays a central role to be able to define the needs into data. You probably need to make investment in analytics tools for this level.

 

I think I am a rock star

Are we using data to make decisions all over? – People make decisions on data, congratulations you reached the highest web maturity level.

 

Everything we touch turns into gold

We are gaining market share – When you have climbed all the steps you reach the final goal, where you see increased market share and higher revenues.

Why we created a Data Warehouse (for the Web Analysts and everyone else)

On Thursday we launched our Data Warehouse. It was a special moment since it took us about a year of blood, sweat and tears to get all data in the same place. Much credit goes to Jon, Andrew, Harb and all the other guys involved in creating this masterpiece.

Why we created a data warehouse?

Our business are 100% online so we use mainly Omnitures various products to collect and analyze the data but it was not enough. The main problem is the lack of flexibility in the Omniture product suite hindering us to truly customize according to our need. We have about 60 websites that we keep track of and on top of that we have no control of the revenue data since it is tracked by a third party making our revenue based decisions more or less a guessing game. With the magic of the Data Warehouse we are able to eliminate this blindspot and make better business decisions.

What is the value of a data warehouse for a web analyst?

I must admit that in the beginning I had a hard time understanding the value of the data warehouse, although my attitude towards data is the more the better. I thought that the Omniture products were enough, and in some sense that is correct. The problem was that we have so many websites, and so many third party cooperation’s that when something went wrong no one could really make a good analysis. Now we can. Now I can see all the way from a visit to a purchase. Priceless.

So, how valuable is a data warehouse for a web analyst and can one get by with just a web analytics package? Well, I think it depends from case to case. As a general rule the more complex your business model is the bigger the need for a data warehouse.  At my previous employer we turned our tool into a data warehouse by customizing it to the extreme (thank you Yahoo Analytics for letting us do it). I know for example Dustin, the wholesaler of electronics are harvesting the potential of a data warehouse. For example by breaking down the figures they could micro segment their users sending out special offers depending on their interests and location.

How are we getting smarter by the data warehouse?

So far we have only had the thing for a few days but already I used it once. In future I see the data warehouse doing a lot of cool stuff such as correlation analysis of our top keywords and brands, projection of future and past trends and many more other things that does not come into my mind right now.

Of course it is like many others have pointed out, the tool itself is a dumb box, and it’s the people using it that will make the difference. Hopefully we will be able to get to the next level but I think road will be bumpy since it involves a mindset change for many people within the organization. If it is done the right way I think this will save a lot of money and energy by letting us to take the right decisions.

I think the web analytics market is going towards a more data warehouse approach

The field of web analytics is a fairly new one but fast evolving. I see more and more signals that the tools are becoming more sophisticated as their users, which is pushing the development towards a more data warehouse driven approach. For example the other day we had a representative of Web Trends that showed us their products and it was clear that they are going towards a data warehouse approach (and also copying Omniture).

I will most likely get back to the data warehouse when we have used it a bit more. Until then feel free to comment below with your own experience on your thoughts or experiences with a data warehouse.

Are you a Web Analytics Ninja? Take my quiz

In November I celebrated my third year as a Web Analyst and I must say I still love my work just as I did the first day. Just as this blog tries to give something back to the community I have trained people in real life to become good web analysts by sharing my knowledge. I am far from a teacher but I try to give it all and not hold back.

To be a good Web Analyst I think you need to be very broad in your knowledge. You need to know a bit of everything for example, how tracking works, how seo works, how to understand your customers, how to convince your coworkers and many other skills.

Rickard a collegue of mine was saying that Jon, another collegue, is a black belt in Excel. I thought that the black belt phrase is a good one to describe how good of an analyst you are, therefore I have created a “game” where you can find out if you deserve the black belt or not.

Download the file, print it out, count your points, and post your results in the comment field.

Of course you can download the file and post it on your blog. Creative Commons linces as they say.

Download Here

Post your score below

How I communicate the factors that affects our conversion

When working as a web analyst I have discovered that you are more than just a guy measuring your visitors and pulling out the stats. To be really good at your work I think one needs several skills among them understanding what affects your conversion. I constantly try to get involved in all parts of the business, from SEO to PPC to CRM and any other aspect that apply to the business model. I must say this is not an easy task but I think it gives me a lot of pleasure sniffing on other people’s area of expertise. What surprises me every time is that something that looks very simple for the outside might be very complicated when you dig deeper. Anyhow, the other day I was thinking of an easy way to display what affects our conversion and I came up with the following picture (I have removed a few factors but you still get the picture).

I think that displaying your most important factors that affect your conversion in this way makes it easier for everyone in the company to understand how you can improve. The hard part is to identify these factors, hence the argument that you have to know your company´s business model very well.

Each of the factors identified here can have several sub factors. For example, looking at your entry page you have a design factor, how you present your text and what you really say (the value proposition).

Factors Affecting Conversion Extended

So after finishing the drawing I realized two things. The first is that it gives you a very easy tool for communicating with various stake holders. The second thing was that it really showed the complexity of business optimization or in my case the complexity in my own role as a web analyst and conversion specialist. It was actually a bit of an aha-moment since to truly optimize our business I had to be that annoying Sherlock Holmes-type and not just mind my own business.

So, I would truly suggest you to make such a map show it to your various stake holders and ask them what they think. Even if they don’t like it, I think it will bring you a step closer to becoming that black belt web analytics expert.

My A/B testing guide

A year ago when I changed jobs I was faced with the joyful task of being part of the startup of our A/B testing program. Since we have several sites under our wings the methodology was very important because we wanted to draw conclusions from our tests to be applied to new websites and in some sense create a kind of “theory” on website design and content.

With 150 A/B tests and counting our methodology has brought us a step closer to understanding what affects a webpage conversion rate. This knowledge has made our life a bit easier since we can apply the findings on new projects with the comfort of knowing what conversion rate we can expect. Without going any deeper into the core of the methodology I thought I would give you my approach on A/B testing.

The below points are a guide of what you should think of when doing A/B split testing and are based on my own experience. Please feel free to comment on them if you don’t agree.

1. Be prepared that your testing will bring up new question that will need testing. So you are basically in a testing process rather than just one test.

2. Think of a concept you want to test. Is this concept aligned with what users want or is it based on your own cravings? Very good inputs for testing are user surveys and focus groups.

3. Look how much traffic you have on the page(s) that you will be testing on. Large amount of traffic allows you to do smaller changes but still see an effect.

4. If you have large amount of traffic consider doing A/B/C/n testing. This will allow you to cut down your testing time considerably.

5. Analyze where the traffic to the testing page(s) comes from (I mean URLs). If a larger part of the traffic is from Google look at the keywords which show user intentions. Try also to understand how this page fits in the user path. Ask the following question: Is this page critical for users to convert?

6. Make a wireframe of the page you want to test. Make your various versions stand out from the original page. If they are too similar you will probably not reach statistical confidence, hence not be able to draw conclusions.

7. This is a no brainer but I need to say it, design the page so it fits the rest of the site.

8. Communicate clearly to your coding guy (fronted) what the success metric is. Usually it is conversion or/and money but it could also be reaching a certain page or downloading something. News site for example have more page views per user as a success metric.

9. It is very common in the beginning of the A/B testing experience that you don’t track everything on your testing page, which is a mistake. As the saying goes, it is better to be safe than sorry, so you better put more tracking elements on a testing page than you thought of. This will avoid you to redo the test later.

10.  If you are working with segments, which you should, consider to divide your traffic into those segments. The segments could for example be based on entry keywords or previous page behavior.

11.  Front end code your testing page(s) and setup your testing tool. Direct 50% of the traffic to one version. If you have very large amount of traffic you can do the test on only a fraction of your traffic. For example I heard that Google only tests on 10 of their traffic. As testing tools you can use Google Optimizer or any other tool that fits the requirements but remember to do QA so everything runs smoothly when you launch your test.

12.  Launch the test. Keep an eye on it the first couple of hours so everything seems all right.

13.  Let the test run until you reach at least 90% confidence. Usually the confidence level is shown in the tool.

14.  When you reach confidence stop the test and do the analysis. Example of questions you should ask is; What do the numbers show? Did we reach our targets? Did we answer our hypothesis? Is this the best we can do or can we improve the numbers even more? What other questions type of questions did the test arise?

15.  Document your results and conclusion in an easy way so you can go back whenever you need to. Documenting is important if you do many A/B tests since you probably want to use your knowledge further down the road.

16.  At this point I must put in a warning since many people think that one test will be enough to get the lift you are looking for. Most of the times your results will be rather surprising which will more or less force you to do more tests. This is a good thing since the more you learn about your users the better conversion rate you will have.

17.  As a final point, do not forgett to make the A/B testing process fun. If you can involve your designer and coder in the beginning of the process it will make your testing people and your whole organization much smarter in the long run.