Forget what you think you know about mobile surveys -

chances are it’s already out of date

Where were we?

Mobile research has been hyped for over a decade, but it’s been a torturous business from the start. SMS surveys (remember them?) proved costly and limited in what they could achieve. Mobile-friendly surveys took an age to become, well, friendly - and mobile survey apps aren’t taking the world by storm either.

So what’s been going on? The industry has been pretty good at identifying the opportunities (talking about it for 10 years is no mean feat) but each solution has come about with a misalignment of the twin “tipping points” of respondent behaviour and mobile technology. Two steps forward, one step back.

SMS surveys were only undertaken by a core of respondents who were willing to put up with a frankly horrendous interface and the cost of sending back each answer.

Research firms invested time & money into platforms that no one actually wanted to use, and couldn’t deliver what the marketing blurb promised.

Mobile-friendly was the next wave. Actually, just being able to render an online survey on a phone was the first challenge but, as phones got smarter, the functionality and survey interface improved. We started to care about survey length, correctly identifying that a 20 min survey on a Blackberry was akin to what Carl did “for science”...

The problem was two-fold:

Those 20 min (& longer) surveys were still being commissioned (so preventing respondents from using their phones was, bizarrely, important), and not enough of them were actually using their phones in the first place to take mobile surveys mainstream, which hindered efforts to keep survey lengths down - what was the incentive to do so if everyone was still using computers to take your survey?

Then came along the survey app. This promised to solve the “mobile issue” once & for all, and certainly the technology opened up all manner of research opportunities. Every panel company and its dog rushed to develop an app, encouraged panelists to download it and built sales targets around it.

Great!

Except for three things:

There weren't enough app surveys for respondents to take and make it worthwhile to them.
There weren't enough app users to cover the (few) app surveys being commissioned.
No one thought about data output & analysis when handling media such as photos.

On top of that, there is a great big elephant sitting in the corner that, in our eyes, makes any great surge in survey app usage now unlikely.

As with virtually every other technology that MR has adopted, survey apps are not designed with industry collaboration at top of mind.

The respondent coverage each app offers is limited to the panelists / community members who install and use it. That's usually a fairly modest (& skewed) subset of an online panel, so the first thing it affects is sample feasibility.

Not all panels have rolled out their survey app in all of their countries, so multi-market coverage is the next issue.

Plus, (and this is the killer) you can't just go and include another mobile panel in the mix to help you out - not unless you want to re-script the survey.

Remember, these apps aren't designed to talk to each other, and respondent suppliers aren’t keen on helping their members install competitor apps.

Even if you do re-script, you need to make sure that the functionality of both apps meets your needs, and that you can get the data out and merge it together. You wouldn’t expect to re-script an online survey just to increase the respondent base. That’d be crazy, wouldn’t it?

So we got the tech but completely forgot to deal with scalability.

Oops.

Silo Mentality by Bill Crooks

Where are we?

Back to mobile-friendly surveys, then. This time round, they started being called “device agnostic” and they began to incorporate features that the mobile apps had, such as photo & video taking.

Not only could they start competing with apps in terms of functionality, they blew them away on respondent coverage as more & more participants started defaulting to mobile to check their emails. We recently had 70% of respondents take a survey (unprompted) on their mobiles. It was 10 mins long, and the audience was women over 40 in the USA. That’s the behaviour tipping point in action right there.

There's one area, though, where apps continue to have the upper hand technologically, and that is for offline survey taking - participation without needing a wifi connection or reliance on mobile data networks.

The ability to postpone data uploads is important when you're talking about media capture such as videos, and vital when network coverage is patchy. Sometimes you need respondents to record multiple events, and apps can deliver recurring surveys to them without intervention.

This means that the survey app is still there to cover “offline” tasks such as respondent diaries, in-store audits and mCAPI F2F.

Even so, the previous limitations have not magically disappeared. You still need to have the app to do the research, and that’s still going to limit who does it, where it’s done and what you get out the other end.

The fact is that the behaviour tipping point has not aligned with the tech. Mobile surveys almost carried that over the finish line, were it not for the awkward need to run research where people are and do things, rather than where the internet exists.

Where are we headed?

To really reap the benefits of the mobile “opportunity” survey apps need to solve the coverage problem or online surveys need to solve the offline problem.

Guess what's happened?

Online surveys can now be offline too. You don’t need an app. Apps haven’t solved the participant issue, and now there’s really no need to because mobile surveys can now have all the functionality of their app cousins, including that tricky offline access.

Yes, offline is the new online.

What is this witchcraft you speak of?

If it all sounds a bit oxymoron, the key idea is pretty straightforward: allow the device to run through the survey and store the inputs when offline (just like an app), but use a simple (common garden variety) survey link instead so that anyone can access it. When an online connection is available the results get uploaded. Simples.

What does it mean for researchers?

It means you can deploy to panelists, customer lists, community members or even a face to face interviewing team. Or all of them together. Whoever you like.

It means you can invite via email, SMS, chat or app notifications (oh, the irony!). However you like.

It means you can have participants take photos & videos, and capture location on the fly. When they’re online and when they’re not - it makes no difference. Do you want to gather open-ended responses? Mobile Research 101 suggests this is a BAD IDEA for which you’ll be shamed & ridiculed, but how about by-passing the keyboard and the fat fingers? Audio capture and auto-transcription could be your new friends.

It means app functionality without app restraints.

Do you accept a weaker audience for your mobile research because of the platform you want to use, or a weaker platform because of the audience you want to interview? Yesterday you may have needed to make that call. Today, you really don’t - and you should push back on anyone suggesting it because the solution is out there.

What does it mean for the industry?

First off, scalability can be achieved on both behaviour and technology, simply by removing those pesky development silos and building industry collaboration into the solution. A share of a pie (& the opportunity to share more & better pies) is usually better than no pies. Makes you wonder what else could be achieved.

Mobile research providers have a decision to make. Do they continue to promote proprietary solutions that miss the mark for a huge amount of research that should be doable (but isn’t)? Or do they bite the bullet, co-operate with each other and reap the rewards from the pent-up demand for the mobile research that clients always wanted to commission?

It could mean something else. After so many years of talk & hype, how many more chances will research buyers give mobile to get it right? It’s not the last chance saloon - researchers are a naturally curious breed, and past disappointments don’t mean that new or improved solutions won’t get a fair hearing, but it becomes harder to generate interest in the next big thing (and drive adoption of it) when the next big thing has been the next big thing for so long.

So, there you have it. The bottom line is that your next mobile survey can be online AND offline AND anyone AND anywhere AND anytime. Now you know it can be done, go and challenge your mobile research supplier(s) to make it so. There’s been enough talking. It’s about time we saw some action.

Let's go bake some pies...!