Digital nourishment for hungry researchers
How
is your panel vendor?
How do you measure how “good” a panel supplier is?
This article isn't about who has the biggest panel, highest response rates or deepest profiling. All will play important roles in deciding who is best placed to support the study you're handling.
However, they're only as useful as the expertise of your panel vendors (we're talking about the people you deal with, not the companies they work for). Let's pretend you're doing home improvement rather than market research and you have just found the ideal heating system for your home. For maximum efficiency you (or your installer) need to know how the components work, both individually and in combination. Lack of expertise will mean your home doesn't heat as it should and your monthly bills will be higher than they should.
As with heating engineers, the key question you can ask of your supplier contacts is this:
Do they know their stuff?
If they do, this will be a simple confirmation of their awesomeness. It means they are more likely to be inquisitive and naturally curious about what you need and how they'll make it happen. Even if other elements of their offer are less compelling than their competitors, you can be confident that the person you're dealing with has the know-how to get the most out of their own panel resources.
If they don't, chances are you're missing out on wider know-how which will help you nail your next project (& the one after that, and so on). You're also paying more than you need to. This is one case where higher cost reflects lower expertise.
Here's an insider tip - the moment they say “This is how we've always handled it”, it won't work out. If there are wider issues with the organization, change supplier. If not, change your contact.
So, here it is...
The DataChefs "Panel Vendor Awesomeness Test"...
The test is super-simple to set up. It's a straightforward quote request - use your own version of this example and send to your supplier(s):
-
You have a main sample of N=1000 and incidence rate of 60% within a profiled demographic (eg nat rep adults)
-
You also need some minimum reads on specific main brand buyers (at least 150 of each). Within the main sample, the IRs of each boost are as follows:
-
Boost A: 5%
-
Boost B: 5%
-
Boost C: 10%
You can make up your own specifics. For example, this could be for coffee buyers in UK with a 15 min survey length. It could be women who dye their hair in Germany. Or whatever - that's not important. What we want to look at is the mechanics of the feasibility calculation.
The Results
Quite a few vendors will produce something very quickly along these lines:
On the surface, it seems to answer what you need, but the boost IRs are among main sample qualifiers, and not gen pop (or the targeted invitees) ie 5% is 5% of 60% = 3%:
That's not helping the cost side of things, but some of the boosts will fall out naturally from the main sample. That means you're filling some (in this case, 200) of the boost targets at high (main sample) incidence. You can decide whether or not you want to increase the main sample to maintain 1000 non-boost qualifiers or not. Let's assume we want to do that:
Good, but it's not quite Carling:
The Boosts are accumulative, not independent. If respondent X doesn't buy brand A, they might buy brand B or C. By treating the boosts as independent, you're effectively screening a respondent who doesn't buy brand A before checking if they might be a brand B or C buyer.
You want to be able to calculate not just the natural fallout of boosts from the main sample fielding, but also the natural fallout of boosts B & C from boost A fielding, and then the natural fallout of boost C from boost B fielding...and so on.
That makes the sample usage more efficient than the above scenarios (& more accurately reflects what will happen in field), and the combined boost incidence will be higher than any individual boost:
What do we end up with?
-
1000 main sample respondents at 60% incidence
-
200 boost respondents at 60% incidence (falling out naturally from main sample)
-
250 further boost respondents at 7% incidence
Alternatively, the total sample (main + boost) works out at 1450 respondents at 28% incidence - depending on your vendor, this might give you a better price.
And that's it! We used real-life CPCs to calculate what the different quote scenarios cost. As you can see, there's a significant difference between the highest & lowest and that's the discrepancy you could find within just 1 vendor, depending on who's looking at your request.
Better expertise can be had at lower cost. So where do your suppliers end up? Are they at the top of their game, or is a tactical substitution in order?
If you want to check the blended incidence for your own projects, we've created this handy calculator to help you do just that!
Do you want the calculator as an app on your phone or tablet?
If you're yearning for more intuitive ways to manage projects, to tap into a wider range of tools & participant sources, and want a splash of mobile magic while you're at it, we certainly hope this gets you thinking...and if you want to delve a little deeper, take the next step & get in touch!
