Web analyst part of the revolution

I have been working with data in various ways for about ten years by now. I have worked both with collecting data, processing surveys, managing workshops, analyzing strategic data and now lately as a web analyst and conversion specialist. Being in the business for a while I have seen the transformation towards a more data driven society. For example the last two years the web analytics field has literary exploded. New tools and features are being introduced weekly. Companies are craving to handle their data and more and more companies are in the process of hiring web analytics people. The data revolution is hitting every sector of the economy.

In an outstanding article in the Economist about the data revolution one could read:

“Revolutions in science have often been preceded by revolutions in measurement,” says Sinan Aral, a business professor at New York University. Just as the microscope transformed biology by exposing germs, and the electron microscope changed physics, all these data are turning the social sciences upside down, he explains.

The Internet has contributed to this explosion of data as the article continues with Wal-Marts huge data sets:

Wal-Mart, a retail giant, handles more than 1m customer transactions every hour, feeding databases estimated at more than 2.5 pet bytes—the equivalent of 167 times the books in America’s Library of Congress (see article for an explanation of how data are quantified). Facebook, a social-networking website, is home to 40 billion photos. And decoding the human genome involves analyzing 3 billion base pairs—which took ten years the first time it was done, in 2003, but can now be achieved in one week.

Of course no human is able to handle these huge data sets therefore a lot of resources are being put on tools.  The amount of digital information increases tenfold every five years and the industry grows with over 10% yearly, roughly twice the software industry. The tools are getting more and more sophisticated and analytical methods are being built into the tools. For example Microsoft’s search engine Bing are examining 225 billion flight and price records to advice customers whether to buy an airline ticket now or wait for the price to come down (Farecast).

The article in Economist supports my general feeling I have had lately that the role of data interpreter, call it web analyst, BI, statistician or something else is heading towards a crossway. In the companies that are in the forefront of the data integration the need for a traditional web analyst is morphing towards a more converting type of analyst or even towards a strategic business developer.

From my perspective I see two major trends:

Trend A – More and more companies realize that data is a critical part of their business and the world spins faster and faster. This puts a lot strain on their data collection and their data use in making the correct business decisions. In this case the role of the web analyst is usually more of technical sort, handling the tools, understanding their requirements and supporting the organization on KPI´s and creating ad hoc reports.

Trend B – The data tools are getting smarter by the year. For example the field of web analytics is merging more and more with traditional BI. As the tools get smarter they are doing more and more the work of a traditional web analyst. KPI reports are handled automatically, ad hoc reports can be subtracted instantly without any special knowledge, correlation reports and basic insight is generated by the tool. In this case the role of the web analyst is either more like a statistician who handles all data input or a more evangelist like where one interpret or support business decisions.

For a web analyst both of these trends are favorable since the occupation becomes more popular but they demand two different types of skillets. In both cases the value of data lies in how we interpret our data. A combination of good tools and smart people will always be needed but the role of generalist is becoming more and scarcer, since more people will need to have special skills in various areas.

How many companies have web analytics or conversion departments today? I see this changing in the future. The BI and web analytics function will merge and the department will become a key player within the company.

As a web analyst in the future you have to consider what type of trait you bring to the table when the tools will do most of today’s traditional web analyst job. Focusing on the business value is a must but in what way? I keep pondering on how this development will go and I have no clear answer at the moment. Maybe you do?

“The data-centered economy is just nascent,” admits Mr. Mundie of Microsoft. “You can see the outlines of it, but the technical, infrastructural and even business-model implications are not well understood right now.” – Economist, Feb 25th 2010.