Tip #4: Keeping it Real

Congruence. Not a word you often hear in everyday conversation. But it’s a word that is critical to market research, in at least a couple ways. One way is the how aligned research results are to past studies. Another is how well the questions we ask match the respondent’s reality.

It’s that second one I want to talk a bit about today.

Tip #4: Making It Real

When I put together the design of a piece of research--whether it’s the method, the specific questions or the answer choices, I feel compelled to make sure that it all reflects reality as much as possible, given a research setting.

Moose, the office puppy, keeping things real.


At a high level, the more a method reflects reality or lets us peer into a person’s real life, the more accurate the learnings are going to be. That’s the power behind ethnographic methods and other methods that are all about observing behavior--digital or analog. These methods align with real life because they are basically just observing real life. The downside is that just observing, if you will, doesn’t allow much room for posing questions or directing behavior so that it answers our relevant research questions. And, while they can tell us about what people do, they aren’t quite as insightful around what people think.

As contrived as a focus group or survey is, we can design into that guide or questionnaire activities and questions that borrow from these more observational methods. For example, once we were using focus groups to help with packaging design for an insole. To begin, we had set up a mock shelf with different brands and models of insoles. We asked consumers to imagine they were at the store and to select the product they would go home with. We learned a lot just by watching how they made their choices and then having them talk through what was going on in their minds. The value came from how much the process mirrored reality.


I firmly believe that good research is simply a good conversation. The best interviews end up being conversations like you might have with a good friend. Surveys can also be good conversations. Granted you have to work it a bit harder to turn it into a conversation, but it can be done. I find that part of it is getting the flow of the questions to be as natural as possible. It’s not easy--there are considerations like qualifying respondents that are often difficult to make flow like a conversation, but with some thought it can be done.


A key element of a good conversation is the language you use. In the research industry, there is a real tendency to make questions stilted, professional sounding, and typically in passive voice. When was the last time you had a great conversation in the passive voice? I have found that finding your voice is an important step to writing more engaging and conversational questionnaires. And writing the questions in the same voice you would use if sitting down and interviewing someone works well. Use “I” instead of “we.” Be direct and straightforward. Read your questions out loud. Do they sound like you are in a conversation or taking a survey? It’s all about keeping the conversation real and flowing.

Answer Options that Flow

Of course, conversations, at least good ones, are not one sided. So how the person you are engaged in conversation with responds is important. In an interview or focus group, they have the leeway to respond in their own voice, in their own way. But in a survey, unless it is an open ended question, you are giving them their voice in how to answer. This is where some preparatory qualitative can help you find consumer language. You can also play the conversation through your mind as you write out the answer options. It’s much like writers create dialog in a book. Imagination, empathy, and listening all contribute to being able to create that dialog. Only this dialog is written as a question with answer options.

Reflect Real Choices

Another area that can help make things more real is the types of questions and the answer options for a question. Think about the standard purchase intent question. It is an accepted norm in the research world, but if you look at it closely, you might realize that it is so far from the reality of purchase decisions that it is no wonder the top two box answers are seldom predictive. How often do you look at an item on a store shelf the choices that run through your mind are whether you will definitely, probably or probably not buy the product in question. Really think about that. Reality is more along the lines of “I want a loaf of bread. Which one of these will I buy?” You may just pick one out of habit, or you may compare prices, brands and other important criteria to narrow down your choices. Do I want sourdough, wheat, multigrain. Boudin’s, Wonder or Sara Lee brand? (Why am I suddenly wanting a sandwich?)

Real Behavior

Lastly, let’s talk about things like pricing, advertising and packaging. In my opinion, it is as far from reality as it gets to ask a consumer at what price a product would be considered too expensive, too cheap, or just right. As much as Goldilocks and Van Westendoorp like that approach, when was the last time you, as a consumer, got to interact with a brand that way? Other than a survey. We recently did a survey among buyers of a brand, where this approach resulted in the optimal price being half of what these same consumers had actually paid to buy the product. Price, like advertising, brand, and packaging is a marketing element intended to elicit specific attitudes and behaviors. How we behave and how we think we behave are often two entirely different things. There are examples where I do not like the color palette of a brand’s packaging--but I buy it anyway. Or where I love an ad--find it super appealing--but don’t end up buying the product advertised. What we like about prices, packages, ads, and other marketing elements doesn’t necessarily correlate to what we would do after exposure to them. I find that stealing a page from the Behavioral Economics playbook helps to create designs that more accurately assess the behavioral and emotional results of marketing elements rather than just the articulated appeal.

Those are just a few ways I feel we can make research more congruent with real life. I hope you enjoyed my thoughts and learned something that help you with your future research efforts.