|

Measuring up?

Measuring up?

It will be a long time before machines truly understand the complexity of the world and how we perceive it – which is why media research still needs humans, writes Craft’s Konrad Collao

It’s been an interesting time in the world of media research (and research more broadly). Mediatel’s Media Research Awards, Future of Media Research conference and Connected Consumer Awards have passed, with the Connected Consumer Conference imminent.

We’ve also seen the publication of the British Polling Council’s inquiry into the polls’ failure to predict the 2015 General Election.

Following these events closely it has become increasingly clear that we are living through a critical period in media and media research. We should all be asking ourselves: how can we know? It seems like the kind of esoteric question asked by earnest undergrads in halls around the land, but it’s fundamental to media.

Clients buy and media agencies sell according to what they commonly agree they ‘know’ about audiences and users, consumers and respondents, or whichever other label we choose to describe the people we are interested in reaching.

In this context research exists purely to further that knowledge, be that to prove sales cases, to truly explore new behaviours or to simply give the industry a picture of what the hell is going on ‘out there’.

I was particularly struck by a debate about the effects big data will have on media research. We all know that exponentially increasingly amounts of data are being collected. The received wisdom seems to be that this is an unequivocal boon.

The fashion has been to signal “the end of the polymath” as agencies specialise, with the most anticipated specialisation being in data analytics. A particularly striking comment from an industry grandee was that some of the more innovative observational methods – full disclosure, that we at Craft specialise in – while interesting, will become “peripheral”.

Looked at one way, we can see merit in this argument. Ultimately media is all about selling. To set a price you need to measure, to prove and convince: reach, frequency, ROI, ‘engagement’ (whatever that is – can anyone define it?).

With more accurate ways of measuring, with more measurement data than we know what to do with, with people better versed in mining it, the major existential question we at Craft ask ourselves is: what is the role for observation and intuition in an age of supposedly incontrovertible ‘proof’?

[advert position=”left”]

Anyone who has faced a quantitative dataset, especially derived from an ad hoc study, knows that one of the first questions to ask yourself is: where do I start? It’s not always clear what you are looking for, where you might find it or once you have found it, what it means.

So despite quantitative research’s reputation for being ‘objective’, ‘rational’ and other adjectives that convey the impression that numbers cannot be argued with because they take ‘subjectivity’ out of the equation, we believe that humanity can never be eliminated. Because the human can’t be eliminated, nor can intuition.

So, is that a bad thing? Undoubtedly, not recognising the effect of intuition and biases (we all have them) on research design and analysis is problematic. But if one recognises that the very act of researching – of setting hypotheses, designing an experiment and analysing its results – makes the researcher present, then we can mitigate those effects.

Even that phrasing suggests the presence of intuition and humanity is problematic, that what we ideally need is a machine. The problem is that machines aren’t very good at understanding humans for the foreseeable future at least.

Let’s embrace the role that supposedly subjective observation can play in furthering all our knowledge.”

Yes, they can beat us at Go. Yes, they can drive a car (sort of). But it will be a long time before machines truly understand the complexity of the world and how humans perceive it, which is what media ultimately plays on.

Media research will therefore need humans for a while longer. Those humans will need to understand the world and the people they are researching so that they can draw meaningful conclusions. So let’s embrace the role that supposedly subjective observation can play in furthering all our knowledge, and incorporate that into the research and media mix.

If that sounds ‘unscientific’ we need look no further than an example from physics to make us feel better. Sir Isaac Newton didn’t initially get the idea for his law of universal gravitation through measurement. He observed an apple falling, had an idea and set out to prove it.

Describing his initial observation as ‘unscientific’ does it an immense disservice. It was seminal, the subsequent experimentation and research supportive.

We worked out the world was round not by failing to fall off it but by noticing the masts of ships sailing back to shore before the rest of the ship. History is littered with examples of dud observations too, but the observations needed to happen for the hypotheses to be disproved.

Scientific discovery is the story of intelligent people observing usually quite simple phenomena for the first time or thinking about them differently, then using the scientific method to prove or disprove their hypotheses.

In a world of constant media and technological flux, where orthodoxies are obsolete and change is the norm, to us it seems the role for people observing and interpreting the world has never been more relevant. Such analyses provide the guidance on what to look for, where to find it and what it means.

That isn’t just a philosophical point. One example of where observation can reach the parts that measurement cannot is our winning entry at the Connected Consumer Awards, for work into 16 to 24 year olds’ relationships with emerging social networks for the BBC.

As Richard Marks stated when judging, actually working out whether we should do something on a certain media platform is a good place to start with building an audience strategy, commercial or otherwise.

While measurement could tell that a lot was happening on Instagram, Snapchat and WhatsApp, it couldn’t shed light on the complicated interplay of who, what, when, why – and crucially, what the BBC should do about it. We know that context is queen when it comes to media consumption. The best way to understand context is through observation.

Combining observation and measurement, qualitative, ethnographic and quantitative methods, provides synthesis. Before placing our faith solely in ‘rational’, ‘objective’, ‘scientific’ data and measurement to the exclusion of all other ways of knowing, let’s remember the wise words of one of the cleverest of them all, Albert Einstein:

“Not everything that can be counted counts, and not everything that counts can be counted.”

Media Jobs