When AB testing hits the optimization wall

When AB testing hits the optimization wall

Have you ever hit the conversion wall? You do your first tests and often get double-digit increase in conversion. Everyone is excited and you feel that optimisation work is fairly easy. Then something happens. Your tests do not give you satisfactory results any more. You do test after test and only get incremental increase on your conversion rate. You feel frustrated and you have no clue if you next effort will give you that sweet double-digit increase or not. On top of that your testing take a lot of resources and you start wondering if you could putt hat effort into something else. This is when you have hit the wall. Your next step will determine if you will be a cutting edge online company or not.

Something that the consultants do not tell you

When working with conversion optimisation consultants it is in their interest to keep the account going for as long as they can. That is why few will tell you that you have hit the conversion wall. You should either stop doing further tests or that you need to build a conversion department. Building a conversion department will eventually take them out of work so they keep trying with tests after tests.

The 80/20 rule applies here too

Like in many other cases the 80/20 rule applies on your testing too. 20% of your tests stand for 80% of your conversion rate increase. Low hanging fruits are the usually biggest ones. When you have picked those ones you need to invest in a ladder. This investment is usually fairly substantial and might take long time to yield results.

When do you truly know that you hit the wall?

Well it is hard to say. Some companies have a longer way to go than others. It is usually when you have tried many different things and more or less run out of ideas. Take my work as an example. We are selling one product that is standardised. No colours, no sizes, no features, everyone gets the same thing. When testing our landing page, I built a few versions based on experience how a good landing page should be. Testing the first one yielded double digit conversion increase. The second one also, but then it got much harder. I started to optimize further into the funnel with little success. This is when I knew I hit the conversion wall. From here on I needed to think differently.

What do you do when hitting the wall?

This is when you need to take your game to the next level. You need to get into business development, use business intelligence data, setup conversion optimisation processes, use UX methodology, segment your users and get into personalisation. As you can see this puts your company on a new playing field where it will demand more resources than ad hooch testing and a new way of working and thinking.

Is it worth it?

If you are a small company I would say that you should pick the low hanging fruits and then stop. It does not make a lot of sense in getting a full time conversion employee before you grow bigger. If you are a bigger organisation and have specialized departments like business intelligence, fronted development, marketing department etc then it makes a lot of sense to invest in further conversion optimisation. Conversion optimisation will increase your bottom line revenue and in the same time you get smarter as an organisation. You can read more about building a conversion department on my other post here: http://www.onlineplanet.se/2012/11/how-to-build-a-world-class-online-conversion-department/.

Now, let me ask you a question, does your company have a conversion department? Do you know any company that does?



No responses yet

Leave a Reply