I have worked as a web analyst for a while but I havent seen a post adressing the issue how to monitor a redesign release so I thought I would give you my view on how to monitor the immediate impacts of a redesign.
At the end of this post you will find a link to the excel document.
So, over the past six months we worked hard to make this really cool and useful redesign of our website. With a few hundred thousand of visits each day we had to get it right. The day of the launch came and within the hour I got the first question that would be repeated over the week “How does it perform?”. I looked at my web analytics package and could not see any immediate changes. Of course any of you out there that have had experience with redesign and in general hourly web analytics data know that the conversion rate, the visits, the bounce rate and such varies during the day and no immediate patterns are shown during such a short time.
Fast forward about 48 hours. The first numbers are showing up in my excel sheet and we could relax a bit. Everything looked good. Now let me get to the point, I composed an excel document to follow the effects of the redesign that I want to share with you. You can download the document at the end of the post.
On the top of the page I have collected the four most important metrics and I look at the averages per day. This way I can quickly see if the first day of the redesign looks fishy or not.
Monitor at least seven days, if you see major fluctuations do it for at least two weeks.
The second data set, the graphs, is the ones I like most since they are very simple to distribute and even the most novice can understand them with just a glance of an eye. The important part here is to calculate the standard deviation to have an upper and lower level, to understand if you should celebrate or investigate.
In the excel sheet you will also find the bounce rate. Of course if you want to have more metrics you can just create them, but these three are the most important according to my opinion.
Below you can see an example of the data I use to create the graphs.
I am not saying that this is the only way to do it, but I think it will be good for most cases. Of course I would never do a site wide redesign without testing it first, more on that another time.