Balancing a respectable incidence rate with the right screening criteria presents an ongoing challenge for research project managers. Asking the correct type of screening question for a B2B survey adds a new level of complexity because not only are the surveys generally more technical, the incentives tend to be more prominent as well, leading to people wanting to get in on the survey for the “reward.” Great researchers focus on putting the respondent at the center of the survey, respecting both their time and their profession to ensure the most seamless and pleasant survey experience possible.
Interestingly, I recently had a conversation with a CEO at InterGeo in Germany. His company manufactures Global Navigation Satellite Systems, often used in construction and other industries in place of GPS receivers. He told me he uses a pretty unique screening question in his surveys to get feedback from engineers about his new product concepts. He called the method a “rock solid” way to ensure the input was “innovative, and bullet-proof.” He explained to me that, over the years, he – would sometimes sit in on interviews with people that are “wannabe” GNSS engineers. That is, they may know some of the terminologies, and possibly even have some of the background to potentially provide useful feedback. However, on occasion, they did not have legitimate experience with GNSS, and in fact may have ulterior motives for giving feedback (such as notoriety, the monetary incentive, or even, in some cases, resume padding). So, he asks a non-sensical technical question, one in which the only appropriate answer is “the question makes no sense.” While this takes the qualifier to an extreme, it is an effective way of eliminating imposter engineers from new product surveys.
Screening questions are a balance, make them too strict, and you risk your incidence rate. On the other hand, you want to make sure you get the highest level of participation from the smartest and innovative people possible. To balance out these competing demands requires a look at multiple factors, including the size of the available sample, the type of new product, and the technical knowledge of the target user. In some cases, certain brands should consider a lower usable base but with stricter screening criteria. But for most cases, too severe screening criteria may result in too many people excluded from your survey.
In all cases, it is essential, should a participant not be eligible for a survey, that you terminate quickly and politely. Your surveys should be designed to be smart and should respect the survey respondent’s time and profession, while still maintaining robust qualification criteria to keep out the “imposters.”
Posted by Mitch Henderson
@MitchClear