Tests on Data Quality in the COVID-19 Environment

We have published several Free Covid-19 Tracking Reports to help you understand how consumers are responding to the crisis, and what to expect in the future.  To provide you with the best research leadership possible, we have also analyzed historical results in prior crises, re-fielded tests, analyzed our studies in the field, talked to sample providers, and monitored competitive/industry learning.  The bottom line is that research continues to deliver insights reliably in this environment:

• We have not had significant difficulties recruiting respondents to participate.

• Historically, mean Purchase Probability, Uniqueness, and Value for our entire concept database did not vary more than 0.1 points in most cases over the 12-month periods before, during, and after the Credit Crisis – and competitors report the same learning.

• We’ve re-tested 13 ideas across a number of categories and price points, for both leading and challenger brands (thanks to Precision Sample for providing some of the sample for this).

Below are more details on the results and recommendations for future studies.  

We continue to provide custom online surveys, FastTrack™ evaluations, low-cost ACUPULSE™ omnibus studies, and Live Online™ qual+quant solutions to help you navigate and come out even stronger during the rebound in the months ahead. Please contact us if you have questions or if there’s anything else we can do to help.

********************************************

DATA QUALITY RE-TEST RESULTS AND TEST RECOMMENDATIONS

• On 10 of the 13 concepts, there were no significant differences in Purchase Probability, Uniqueness, or Value (or General Interest for 3 unpriced concepts) versus the original tests.

• Three of the 13 changed, for (mostly) clear reasons:

  • One health concept generates higher Purchase Probability and higher Value now, likely because it included a bacteria cleaning benefit – we believe this increase is a real/sustaining change.

  • One food concept generates the same Purchase Probability but significantly lower Value now … reflecting the heightened attention on a premium idea that didn’t really justify its premium.

  • One personal care concept related to shaving had significantly lower Purchase Probability and the same Value, perhaps because people are shaving less.

• This is consistent with 2 competitors who’ve re-tested in the U.S. and abroad and found nearly identical results.

• 12 personal care ideas tested in rapid succession on Impulse/Gut Reaction identified/ranked the top 5 ideas and bottom 2 ideas in the same order, and only one of the “middle” ideas tested significantly differently on this measure, perhaps because it had an on-the-go benefit after gym/sports which is a sore point right now.  

• Mean agreement levels were statistically equal for 13 of 14 insight statements, and the 14th was only off 0.3 points on a 0-to-10 scale.

All this notwithstanding, we have seen some movement in sub-segments as well as on diagnostic measures, indicating there may be some changes in isolated cases in how consumers react, understandably, but not to the level of changing overall recommendations.  

All in all, we believe consumers have a clear cognitive capacity to project their feelings and conceptualize the future that lets them rate an idea or attitude in general instead of limiting themselves to the moment.  

Going-forward, we’ll go the extra mile to help you get the learning you need:

1. CONTINUED CARE:  We will continue to monitor results closely and re-test examples as needed.  We predict results may elevate for anything involving promises of infection control, sanitization, or saving money, which would be real/sustainable advantages in the foreseeable future.

2. PREMIUMS:  We encourage you to make sure any premium pricing in concepts is fully justified by perceived efficacy advantage that sustains good value, as consumer scrutiny on value has increased.

3. NEW STUDIES DURING CRISIS:  We are happy to include/re-test a previously-tested idea for comparison – at a substantially reduced cost. While we expect similar results, to the extent they vary higher/lower, we can use this to help you understand any impact current conditions may be having on newly-tested concepts. Re-tests are also recommended when you are interested in closely comparing results to previously tested ideas among specific segments or on detailed diagnostics.

4. STUDIES AFTER THE CRISIS:  For extra confidence, we would be happy to re-test an idea that was tested during the crisis for the first-time once it has passed, at a substantially reduced cost, to confirm the learnings hold true.

Spark In-Market Validation

Last year, we unveiled Spark MCR's groundbreaking approach to more completely evaluate ideas than traditional surveys by capturing Impulse and Emotion, as well as conscious Reflection. Spark Multi-Cognition Research® is 2x more likely to capture consumers' "gut" reactions, and better discriminates across ideas to find the real winners. Now we're unveiling in-market results from a new study on Pinterest for a major retailer which demonstrates that Spark MCR results are significantly and more strongly correlated with higher online "engagement" and e-commerce sales than the traditional approach.

Moreover, Pinterest Promoted Pins with higher Spark Impulse and Affinity ratings are 4x more likely to have higher online engagement, and 2x more likely to generate higher e-commerce sales.

Finally, Spark MCR provides a practical tool that takes conscious and non-conscious measures into account and has sales data to back it up. Let us know if you would like to have a quick call to share the results and discuss Spark in more detail.

 

 

Research On Research: Methods Matter

Everyone knows the questions are important – but so are the answers!  In this head-to-head comparison, we assessed two research scales to see which performs better.

The Contestants

The Competition

• Sequential monadic concept test with 3 food concepts

• A separate cell using each scale, with a series of forced choice comparison questions at the end

• Nationally representative sample of 209 adults 18+

The Results

• TWICE as many respondents thought the ACUPOLL Scale was easier to answer, more intuitive, and better captured how they feel.

• This wasn’t just consumers’ opinions – they actually completed the survey with the ACUPOLL scale 15% faster!

The Implications

1. More Accurate Research: The ACUPOLL Scale provides clients with results that consumers indicate better represent their natural and true opinions, without forcing them to choose among labeled answers that often split hairs.

2. More In Less Time: With 15% faster data collection, your surveys can cover more ground in the same amount of time, with less consumer fatigue.

3. Better Statistics: Unlike traditional labeled scales where “extremely” is closer to “very” than “very” is to “somewhat,” the ACUPOLL Scale is an equal-interval scale that facilitates more accurate and reliable statistical analysis (correlations, regressions, etc.).

Contact us today to learn more, and find out how we can apply this to give you
Greater Clarity for Better Decisions™!

Additional perspective is below if you’re interested!

• The “System 1” Scale: As Nobel Prize-winning psychologist and author of Thinking, Fast and Slow describes, “Our thoughts and actions are routinely guided by [our brain’s] ‘System 1’ and generally are on the mark.”(1) System 1 is the automatic, impulsive, and more emotional side of the brain, as opposed to System 2, which is more effortful, reflective, and has a risk of “over-think.” An easier, more intuitive scale like ACUPOLL’s allows consumers to react more impulsively, consistent with their true feelings, which results in more accurate data for better learning.

• Ahead of Our Time: ACUPOLL launched with this scale in 1991. The 0-to-10 pain intensity scale recommended by the National Institute of Health was introduced in 1993.(2) And, Fred Reichheld used this scale when he introduced the Net Promoter Scale in 2003.(3)

• Consistent with Better Research Quality: Twice as many consumers also said the ACUPOLL Scale would be “more enjoyable in a long survey.” The Advertising Research Foundation’s Foundations of Quality study concluded, “As Survey Enjoyment increases, so does a respondent’s level of attention and engagement, ultimately affecting data quality.”(4)

1. Kahneman, D. (2011). Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux.

2. McCaffery, M., & Beebe, A. (1993). Pain: Clinical Manual for Nursing Practice. Baltimore, MD: V.V. Mosby Company.

3. Reichheld, F. (2003, December). The one number you need to grow. Harvard Business Review, 81(12), 46–54.

4. Walker, R.W., & Cook, W.A. (2013). You can’t put a price tag on a survey participant’s enjoyment: the latest findings from the ARF’s “foundations of quality” research. Journal of Advertising Research, 53(3), 254–257.