Web Analytics & E-commerce (EN)

Day 2 @ XChange Europe 2012 in Berlin

The second day at XChange started with a new round of huddles. I was part of the huddle led by Isabelle Mouli-Castillo from Dell on “How to measure your site efficiency when it is not just about conversion?”.

 
Participants were both interested by lead generation-type and e-commerce-type sites. The conversation started on customer experience measurement, which can be measured by specific tools like eTracker,OpinionLab, Clicktale, Sessioncam, Kissinsights. These tools may also be used to assess the Net Promotor Scores.  3 main things may be performed with qualitative tools: 1. Brand awareness measurement, 2. User experience and 3. Broad site survey (interests, needs, demographics, etc…). One participant shared that they are using a 5 point scale in order to understand the customer journey : Search -> Get Inspired -> Get information -> Order -> After Sales. Micro-conversion points are defined for each of these 5 points. The discussion moved next on E-support, where it is difficult to find meaningful KPIs. Time spent on site, dwell time, repeat visitation may be used, but it is difficult to relay these metrics with user satisfaction and site efficiency. It is also difficult to relay consumption of content to conversion for high-end products. Search usage is important to measure self-help site sections.  In general, conversion rate shouldn’t be analyzed without segmentation based both on visitor profile and visitor behavior. Off-line conversion for web sites may also be measure with click-to-call and click-to-chat features, as well as specific telephone numbers for the websites. One participant shared that one of his customer is using different 800 numbers on product pages in order to track their efficiency.

 
My second huddle was led by Alex Emberey, Thomas Cook, on the topic “Social Media Today: Tools and Metrics”. Social Media may be measured by using reach as the first step of the social conversion funnel.  Reach is an interesting metric but not really useful, since a tiny minority of your followers are seeing your tweets on Twitter (about 16 %). The number of likes and fans are also used as a proxy for benchmarking with the competition. Fans worth depend on how your acquire them, how you engage with them and what they do. The issue is more on how are we resonating with people on Facebook, Twitter, etc… than the actual number of fans. Metrics that should be shared across platforms: content amplification (RTs, shares, etc…) than can be compared with competitors. It remains difficult to measure the long term impact of Facebook. Social media needs also to be put into consideration in the attribution models and we need to understand the impact of social actions in the longer customer journey. The issue with installing monitoring tools is that there might not be any comment and interaction with content produced by marketers. You have to look at the audience you want to attract, because there might be bad traffic and some comments may not be worth at all. One of the biggest challenge is to get a significant sample. You spend a lot of time to weed out non-significant comments and contributors. More easy to mine the data when you own the community that is of interest to you. Direct clicks are not necessarily representative of all traffic and you need impressions to balance it. Finding correlations between quality of social traffic and economic value is the right way to approach social media performance. A few metrics: applause rate (% of likes), amplification rate (% of shares  / RTs) and conversation rate (% of conversations). As analysts, we have also a duty to sanity check on the atual importance and impact of social media compared to the rest of the business. If we stop doing social marketing, what will we loose? We need also to recognize that social Web is not just Facebook, it’s the way people are using the web today.  One of the key things is to understand what you are doing now and how it is impacting your business.

 
During the lunch break, an informal meeting took place with the members of the European Special Interest Group of the Digital Analytics Association. Geddy Van Elburg provided an update on the EU privacy regulation.http://www.digitalanalyticsassociation.org/?page=sig_european

 

My last huddle was on "Democratising Data – Replacing Static Reports with Bespoke Visualization Tools" and led by Tom Betts, Financial Times. The conversation started with a testimony of the implementation of a 30-people team dedicated to data analysis and reporting with different roles and sub-teams: business analysts (making strategic information obvious, liaison with business units, subject matter experts -> listen to business goals and translate into measurement objectives), reporting sub-team, analysis sub-team and implementation sub-team. The discussion continue on the best way to deliver information. One participant shared his experience with role-based view, in order to provide personalize views of data. Visualization is a mechanism of communication with the objective to drive end-users’ interest. Better visualization helps to drive conversation with the business. While vizualisation can help communication, but it can also be bad with cuttered graphs. Mantra : “reports are not meant to answer questions, but to raise questions for in-depth analysis". A participant mentioned the work Hans Rosling on data vizualisation and recommanded to watch the following video : http://www.ted.com/talks/hans_rosling_shows_the_best_stats_you_ve_ever_seen.html

Then, a debate started on the extent that data access should be democratized. Some participants raised the issue that end-users don’t have necessary the context to understand and get insights from the data. If people had access to more data, what would be the first place where they would look at? Democratizing data is making people discuss about the data and leveraging it as an asset. Democratizing insights vs data. Purpose is for people to ask questions. Democratization of data may be necessary if the analytics function is distributed inside the organization. Telling people about their data and what can be done about it. Sharing the power of data instead of the data itself. Data should be made available to internal clients who own them and can understand them.

 

The conference finished with a feedback session with Gary Angel, as well as a vote on the location for next year: Berlin, Paris, London, Dublin, Amsterdam or Majorcca. Berlin and Amsterdam seemed to gather the most votes!

Congratulations to the Semphonic, AEP Convert and Rising Media teams for organizing this great event ! Lots of opportunities to discuss, to learn and to meet with peers from different countries!

Day 1 @ XChange Europe 2012 in Berlin

The first XChange Europe conference started in Berlin on May 29^th, 2012 with a welcome reception gathering 100+ attendees from Germany, the United Kingdom, Danemark, Austria, Sweden, Finland, Belgium, Czech Republic, Spain, Netherlands, France, India, Australia, Canada and the United States of America. XChange Europe is an initiative by Semphonic, a leading US Web Analytics consulting company, with the local support of AEP Convert from London and Rising Media from Munich.

 
XChange is a conference tailored for Web Analytics geeks and specialists, with a focus on discussions instead of Powerpoint presentations. In fact, there are no Powerpoint presentations at all and the conference is organized around 90-minute group discussions called "huddles". The groups are made of Web Analytics specialists working for advertisers, consulting firms and software vendors. The rule is that it is not the place for selling products and services.

The conference is organized inside the Scandic Hotel with dedicated conference rooms, right in the center of Berlin and near the Potsdamer Platz.

 

On May 30th, the XChange conference officially began with a panel discussion led by Gary Angel, Semphonic's founder and CEO, and with an impressive cast of speakers: Peter Pletsch, Quantitative Analysis Manager at Allesklar.com, Ulla Kruhse-Lehtonen, Customer Analytics Director at Nokia, David McBride, Analytics Director at Comcast, Web Analytics Director at Financial Times.

The discussion was mainly around the topic "How to make the leap from web analytics to big data (datawarehousing and cloud) ?". Here is a summary of the key points discussed:

• David McBride, Comcast: the rise of video of demand as a service at Comcast fostered business needs to understand the user behavior around this new strategic service. Hence, it was easier to get budget for the creation a customer database with cross-platform integrationlike video on demand. New resources with database skills were required at the Web Analytics team.


• Tom Betts, Financial Times: the growth was organic around the need to be more customer-centric for better targeted advertising rather than product-centric. By analyzing data at customer level, it was possible to get actionable insights who generated revenue, as well as excitement and the need to invest more.


• Peter Pletsch, Allesklar: in order to get more specific information tailored to business needs, it was necessary to build a datawarehouse and to develop a front-end in Excel. Using Excel was of great help to build people acceptance.


•  Ulla Kruhse-Lehtonen, Nokia: the move from web analytics to big data was made easier thanks to a pre-existing central datawarehousing solution, where web data was only a data source. The team was put together from scratch and organically, with a focus on data scientists' skillsets.

In closing, Gary Angel asked the panelists on the role of web analytics tools moving forward vs datawarehousing. While there is a strong tendency to use Web Analytics tools for data collection and to build custom reporting with customer-focused KPIs directly from the datawarehouse, datawarehousing's interfaces are less friendly for end-users, especially for Web-focused analytics questions. Therefore, Web Analytics tools have still a role to play in order to democratize access to Web data, while data warehousing solutions will be more used by specialists for advanced analysis.

After this panel discussion, all participants were split in 5 huddles of 20 persons. During the registration process at the conference, it was required for each participant to express attendance preferences regarding the topics suggested and a personalized huddle schedule was delivered at the beginning of the conference according to priorities expressed.

 

The first huddle discussion I attended was led by Isabelle Mouli-Castillo from Dell on "What could go wrong? How to change the culture of the company to test before implementing any change". To respect privacy of the discussions, I will only summarize the main topics discussed in huddles with no mention to individuals and companies. The huddle started with a strong contribution from an optimisation lead, who explained that he managed to get traction on testing within his company by focusing on answering to business questions instead of red vs green button testing. The size of the panel, 2 million visitor a month, was also of great help to build excitement around testing. How to prioritize questions and testing requests? Academic research can help funnel actions with web psychology. Personas also help prioritization around segments. The Net Promotor Score can also become a personalization criteria. In one company, testing requests are split in two categories: strategic validation and revenue generation. Surveys with strategic areas on users' perspective can also become an input for priorization. Some companies use also a matrix for evaluating and prioritizing requests. One participant emphasized the need to make tesing fun and not just work with team rewards when testing is successful ("gamification"). When testing involves multiples countries, a participant advised to use the organization cultural dimensions from Pr. Geert Hofstede in order to group countries. I raised the issue of testing in a multichannel environment, where conversion can also happen on other channels (retail, telesales, etc...), and it was recognized as a real problem that will be solved with big data integration.

 

My next huddle was led by Gemma Munoz, Mind Your Analytics and Matthias Bettag, Semphonic, on the topic "Competitive Analytics: Does a Predictive Model Give Your Company a Competitive Edge". The discussion was around using Web Analytics to predict outcome and business results. After some discussions, a need was felt by one participant to define what we mean by Predictive Analytics. Opinions were quite diverse on the subject and we had to use Wikipedia to get a clear and comprehensive definition: "Predictive analytics encompasses a variety of statistical techniques from modeling, machine learning, data mining and game theory that analyze current and historical facts to make predictions about future events." http://en.wikipedia.org/wiki/Predictive_analytics. Then, there was a discussion on the need to use different models with different user segments, for instance by using the CRISP-DM methodology (http://en.wikipedia.org/wiki/Cross_Industry_Standard_Process_for_Data_Mining).Since there is a cost associated with Predictive Analytics, a participant raised the question on what is the threshold to consider a prediction as relevant and cost-effective. Also, there is a risk of playing it safe with predictive analytics and pre-defined models, and we need to stay open to big discoveries. In conclusion, everybody agreed that predictive analytics is the most interesting and challengic topic around Web Analytics, and that we all need to become more accurate with models and statistics, in order to be able to trust data.

 
My last huddle session of the day was led by Ross McDonnell, Walt Disney Company, on "Getting the measurement foundations right". The huddle was organized in a methodological manner with an in-depth review of the main Web Analytics stages: Business Requirement gathering -> Measurement framework -> Tagging Requirement -> Q&A -> Reporting. What was very stricking in this huddle is that we all face the same issues on both sides of the Atlantic and that there are no magic solutions. Most requirement gathering are failing because no direct connection with customers point of view. Tag management system may alleviate some implementation issues, but not all of them. Regression testing for Web Analytics is horrible on web sites that change very week. Spidering tools are of a great help to automate testing but manual testing is still needed on critical scenarios. Emphasis was also made on the need to engage teams early and to determine objectives for campaigns and websites before they are released. Report fatigue was also discussed, where some customers throw reports away and prefer to focus on analysis on case-by-case basis. There is no point of sending reports if no action should be taken. If there is no story in reports, it is just noise. Regarding training, it was mentioned that it is difficult to train used on tools and it is often easier to train them around reports.

 
The day ended with a fantastic cruise along the canals and river in the center of Berlin. This was a great way to continue conversations, to meet new peers as well as to discover this great city from an other point of view!