Edward Appleton: How Helpful is Methodological Debate to Market Research?


Dieser Beitrag ist von  Edward Appleton:

Last week I jointly conducted a one-day Seminar on New Market Research with a pre-eminent Supplier of innovative Insights techniques.
It was a great session, with an audience of almost uniquely Clientside Researchers, and intimate enough to allow fellow Insight Managers from a range of industries to open up about the operational challenges and opportunities they face. The objective was to share views on how New MR could help accelerate the transformation process of MR from being a Data Deliverer to an Insights Consultants. A couple of articles I read during the same time frame detailing MR supplier companies‘ views on the same topic – New MR – provided a sharp contrast, and made me wonder if suppliers are in danger of losing sight of what really counts on the Clientside.

The articles in question both were from senior people in what I would describe as mainstream MR supplier companies; both were very strongly negative about traditional Market Research. Jon Puleston of GMI is reported to have taken aim at traditional brand trackers, describing them as „wasting respondents‘ time“ (as reported in Research Live by Jeffrey Henninghttp://bit.ly/17mAHP9 writing on the recent Greenbook Insights Innovation Exchange in Philadelphia (http://bit.ly/165sTQg). Wow. Larry Friedman, Chief Research Officer of TNS North America, writes in the Green Book Blog (http://bit.ly/162VWkr) about how Purchase Intent is effectively useless as a KPI as it is without any predictive validity – more detail below on this. Hard punches.
Contrasing the two, the gap in focus between the Clientside and Supplier debate struck me as worryingly large: clientside the discussion was (without being able to go into details) on how to become more effective operationally, how to increase one’s internal value as an Insights Consultant. On the supplier side, we witness – and probably remember – harsh statements about what current practices don’t work methodologically.
Is the MR debate – at least on the blogosphere – in danger of losing focus on issues that really count for lots of MR clients: impact, increased effectiveness, ROI?

Is an open hostility to traditional methods potentially destructive rather than disruptive?

Are we confusing „new & different“ with „better – because demonstrably helps Client organisation be more successful“?
Here’s my take:

1. The MR Transformation Process has Way To Go.
We often forget it: New MR vs trad MR is a largely supplier-side methodological debate – no doubt one we need to have. However, it’s not the end-game – which is about increasing the impact and perceived value derived from MR.

The Boston Consulting Group published a paper in 2009 focussing on transforming MR, entitled The Consumers‘ Voice. It suggests there are 4 stages of transformation in the MR process: Stage 1 being traditional „Market Research Function“ (essentially data deliverers), Stage 4 – Nirvanah – described rather excitedly as „Strategic Foresight Practitioners„.
My sense from the Seminar was that many of us on the Clientside were likely in Stage 2 mode („Business Contribution Team„): ensuring the Consumer Voice is integrated into the business, with a mandate from Senior Executives. We have not really shifted to the next level, or are just beginning: mastering the synthesis of various data sources, working more broadly across the Corporation beyond Marketing and Sales, growing a knowledge base, with a high level of influence on Senior Executives.

What’s holding us back? No doubt there are many reasons, I’ll point to two key drivers – money and talent.

2. MR Needs to Compete Aggressively for the Right Talent.
It’s perhaps not often stated, but the salary levels offered in MR often compare less favourably with those on offer in Sales or Marketing.
MR is also often perceived as a pass-through place rather than a Career choice.

Money isn’t always a primary motivator. However, without getting into any Behavioural Economics debate, I’d say that at entry and junior level in particular, money certainly matters.

To attract and retain the brightest talent needed to accelerate the MR Transformation process, we need to address hard-nosed questions such as reward and recognition. How can we offer higher salaries?

I’d suggest that charging higher fees works best when you have access to Board Level Influencers and Decision Makers, and that moving the needle at that level comes through systematically documenting MR business impact – how MR investment has helped drive top-line sales, helped save the company money by preventing failures, helped fill the innovation pipeline.
If we do this consistently, aim to document and where viable publish MR effectiveness in a very broad range of Business Journals, raise our profile in all sorts of manners internally and externally, then we are on the right track to gaining a better reputation and justifying higher remuneration.

3. Methodological Debate needs to be both Positive and Impact-Focussed.
I was interested to read the CRO of TNS, Larry Friedman, writing on the Green Book Blog about how MR needs to embrace new ways, re-invent itself effectively. One sentence in particular stayed with me for a few days: challenging us to re-think surveys, he states parenthetically:„some “standard” metrics like purchase intent for established FMCG brands, have a zero correlation with behavior – why are we still using it?

Wow. This from one of the major players in Research, and from a regional Chief Research Officer.

It made me wonder: how is this notion sellable to, say the CMO or the Head of Sales? Aren’t we handing weapons to people who – potentially – may not be believers in MR in the first place? How would the conveyer of this message (the MR clientside researcher) avoid suffering the fate of the proverbial Messenger – shot down? Should us Clientside Researchers have our Damascus moment and ditch Purchase Intent, cast our comparative Top Box scores of the past decade into the dustbin, and optimistically start afresh?
I confess not to having read more than synopses of Joseph Schumpeter’s Capitalism, Socialism and Democracy which is credited with the popularization of the phrase „Creative Destruction“. Nevertheless, to give it my own spin: as we look to re-energise our industry, we need to place the emphasis on the „creativity“ part, and let „destruction“ be a by-product of renewal.

Self-destuction is an unwelcome alternative, and there are plenty of other disciplines – finance, internet marketing, IT, Data Consultants – that potentially could be asked to deliver large parts of what MR is currently tasked with.
Curious, as ever, as to others‘ views.


Über den Autor

4 Kommentare

  1. Dear Edward, thanks a lot for this very interesting article. Indeed, the Clientside and Supplier debate is crucial, but also reflects on of the biggest dilemmas in our sector, let’s call it the “innovation dilemma”. Marktforschung.de posted an article (www.marktforschung.de/marktforschungdossier/image/lost-in-tools-warum-sich-marktforscher-mit-echten-innovationen-schwer-tun/) a few month ago and in my opinion you can find two crucial problems in it (and also in your debate).

    First, MR has to innovate. It’s a must. Sometimes for methodological reasons to deliver better results, sometimes for marketing reasons to have a new attractive packaging.
    Second, a lot of innovation is purely methodological and often incremental, not in any way disruptive. But, saying this, another dilemma occurs. On the one hand, the incremental innovation could be boring and clients could ask: Well, what is the real benefit? On the other hand, the disruptive innovation could be so far away from everyday life, that no one wants to use it.

    I am really convinced that the process innovation is more important than any other methodological innovation, because process comes first, then results.
    One way to cope with this problem: new and maybe unexpected cooperation. Or even more: clientside and supplier are working together and inspire each other and having a real collaboration.
    This means, we need at the first stage a process innovation and a new understanding of working together.
    The boundaries between clientside and supplier should get weaker. But, and this is for me an important question, is this possible? In my experience a lot of the bigger companies have such a strict organizational structure that it is nearly impossible to have flexible project descriptions or “fluid” accounting (e.g. flexible budgets), which is very important to innovate together. Innovation is always a risk. Organizations tend to avoid risks and usually they have a low willingness to engage in experiments.
    But, the suppliers need a cooperation process, how else should they know what the clientside really thinks about and needs to narrow or close the need gap you mentioned between the clientside and the supplierside.
    Just one short comment regarding your point about the “talents”. I also agree, money is an argument, because it is one “currency” to show appreciation. I think there are a lot more, for example, is the work you do important for your company? Is your work relevant for decision making? Does MR open career opportunities? Or, even, is it cool to be a market researcher?
    Kerstin Klär wrote something about the talents from a supplier perspective (http://ilovemafo.wordpress.com/2012/08/03/mut-forscher-gesucht/) and the article shows that we all have to work on this topic!
    Sorry, for the long comment and thanks again for the inspiring thoughts. Best, Oliver

  2. Thank you Edward for this contribution. I think it is an important isue.
    However, one question you pose after mentioning Larry Friedman’s verdict on some standard and purchase intention metrics I do not really understand: „Aren’t we handing weapons to people who – potentially – may not be believers in MR in the first place?“
    Actually I don’t see a problem with these statements of Larry Friedman or that of John Puleston at all.

    Option 1 is that these statements are intended to discredit competitor’s tools. Well, we know what to think of this. Either they are right, than tell the CMO not to buy them. Or they are wrong than you can forget them.
    It only becomes a problem if Friedman is right and you have selected these tools. Then it was a mistake. Shit happens. And not nice. But certainly not a matter of MR in total, but only one of selection.

    Option 2: A method (not a tool) is proven to be wrong, like Friednman suggests by saying that “some “standard” metrics like purchase intent for established FMCG brands, have a zero correlation with behavior“ And he asks: „why are we still using it?”.
    First all this is a ridiculous question. He should give the answer, he is the CRO at TNS. My question would be: Why don’t you throw it out and why do you continue to sell crap research if you know it is wrong? I would tell my CMO to not deal with him anymore.
    On the other hand, the clientside should be the one who can tell us/CMO if Friedman is right. Didn’t anybody on the client side check the correlation over all these years? If no, how come that nobody took a critical look? Even the CMO himself is responsible for this check as well as the researchers who work with the institute/method. The problem here would then not be the fate of Cassandra or one of innovation, but how research, its qualities, limits and relevance are controlled and handled on the clientside.

    • Edward Appleton am

      Oliver, thanks for your comments. On the innovation front, I think the plethora of frankly incremental innovations that MR agencies have brought to market over the past few years haven’t really demontrated a significant impact. I honestly think the industry is actually experiencing innovation in the form of disruption – a business model innovation – from technology suppliers of MR software allowing MR clientsiders to do the work themsevles. It’s the elephant in the room in my view. To your point on process innovation – ie collaboration – totally agree it’s needed and not easy to „step outside the box“ and hope that clientside folk see an Agency differently. My sense is that MR agencies have to start becoming more concrete in terms of what the implications of the insights mean – innovation ideas, messaging optimisation – without losing the core of being researchers. If we don’t do that, then I think the rise of e-marketing will make behavioural tracking the norm, and we can simply and easily be bypassed by eg AB testing lead by internet marketing.

      @thomas – first, again thanks for your comments. I think the TNS comment is problematical in that it suggests an ignorance or naivety about large Company processes and culture – as well as the existence and value of Purchase Intent historical norms. Marketing folk in my view are beginning to turn to entrepreneurial and start-up mindsets (those are the companies that are challenging trad. business models…) about research: test and prove via small steps of real-life experiments, speed being the driving force, with a „fail and learn fast“ mindset…..market research needs to move along with this in order to thrive in future. My concern is that we urgently need to ensure our relevance in this world, build on what strengths we have, not blatantly discredit our activities of the last 30 years and thereby our credibility. btw I have only heard from one Agency – and I talk to plenty – that has said „don’t include purchase intent in your questionnaire, do this instead“. I don’t sense my missionary zeal here, more a concern about how to close the deal.

  3. Hello all,
    Sorry for catching up quite lately on this interesting and urgent issue.

    I couldn’t agree more with the suggestion bringing clientside and agency researchers closer together. But to do this with a promising output one should take a step back and investigate why this is such a big issue …
    I think most of the arguments delivered by Boston Consulting Group in the study form 2009 are still valid and less of the proposed improvements can really be observed out there.

    After the release of the study I wrote some posts in 2009 (here and here, in German) on the BCG work and today – 4 years later – I can see that we are still suffering from drawbacks regarding our process of working together.

    Let’s recap what users of research and producers (clientside and agencies) of research complain:

    Users: ”We know there is a gap in customer insight, but only because the insight team doesn’t understand the business and therefore does not tell us anything meaningful. The fact that ‘our average user is 42 years old’ just isn’t helpful at all”.

    Producers: “We know there is a gap in customer insight, but only because the business leaders don’t invite us to meetings, so we are not intimate enough with the business to craft the best research or to ensure that our findings are properly leveraged”.

    Are these perspectives still valid? And if so, what can we learn about improvement of processes?

    Another though that came to my mind (because I’m constantly thinking about quality and relevance for our industry) is the development of a “new quality” of MR to our clients (internal and external)).
    If buyers complain the impact of our work, why should they pay for it anymore? Who is paying 5 to 6 digit numbers of money for vague “as well as” information? Nobody would, especially in times where budgets are decreasing significantly. And I wouldn’t either, although I see a lot of demand for research and insights for strategic decisions regarding the company I work for 

    It must be possible to be more concrete, inspiring, challenging without losing our common ground, ethos and skills as a profession.

    And I think that we really have the chance to do this. However in order to be successful we will have to leave the path of only selling tools and selling access to participants. Obviously this is the way the business works (worked in the past?) especially for the global research networks.

    Potentially this will open the door for smaller MR-agencies or spin-offs from the bigger ones who stop focusing only on tools and methodology and place strategic thinking, partnership and empathy more in the foreground.

    Not sure this will lead to “business model innovation” for the MR industry, maybe.