What do you use your cell phone for that 10 years ago you didn’t? I know that I, an ancient relic in technology terms, check my email on my phone, make Amazon purchases, check Instagram and sometimes post on there, text friends and family, use Facebook messenger to stay in touch with old friends, and lots of other things. and many other things that even a year or two ago I didn’t think I would do unless I was on my laptop. In fact, there is more computing power in my current smartphone than I had in my first computer. (Actually, my current phone has as much “hard drive” storage as my laptop. Kind of amazing really.)
According to the Pew Research Center, 96% of Americans have cell phones. The numbers are not dissimilar around the world. When it comes to phones, the world is going mobile. And one of the results is that people are taking online surveys on their phones. The last survey we conducted--85% of the respondents took it on their phone. And that was using a traditional consumer panel, not a mobile one.
I have even taken surveys on my phone. Some were well created to take on the phone. Others, too many to be sure, we horrendous to take on the phone. So, here are my lessons learned that will help you make sure your surveys work on phones as well as they do on laptops.
Lesson #1: Think Small Screen
Yes, phone screens are bigger than they were a few years ago, but realize they are still much smaller than the laptop you probably program them on. Make sure the format of the survey is “responsive.” Responsive format, if you are not familiar with the term, is literally making the survey, which is basically a website, respond intelligently to the device that it is viewing it and format it to best fit the screen size. Believe it or not, there are still survey platforms that don’t support responsive formatting.
Test your survey on a mobile phone. It’s actually fairly easy to do. Just send a test invite to your email, and assuming you can check your email on your phone, then take it on your phone. There. That wasn’t so hard was it?
Avoid grid questions. I believe grid questions are the research equivalent to the designated hitter; evil incarnate in so many ways. So I try to avoid them anyway. (We found years ago that 10% of respondents would drop out at the sight of a grid question.) That said, sometimes you just can’t avoid them. But in their native format, a grid question is truly impossible to answer well on a small screen. I have tried. Either the print is about 2 point font and impossible to read or the columns run off the screen. Either way, that leads to straight-lining or dropping out. Most survey platforms will turn your grid into a series of individual questions that once answered automatically go to the next. That’s the way to do a grid question in the first place anyway.
Use standard text fonts. Fancy stuff just messes with the respondent. And avoid large images. Also avoid long pages to keep scrolling to a minimum. It’s a pain to scroll on a phone.
Lesson #2: Use the Twitter Rule
When writing your question text, use the Twitter rule before they expanded--and keep it short--128 characters or less. I view this as a unique challenge because I love to write in complex sentences under normal circumstances. I love reading books by authors that write in almost poetic prose. That’s great for fiction. But when it comes to survey questions, think Hemingway and keep it short and clear. You want as much real estate for the answer options as possible.
This is not just because of the small screen. It is also a result of the changes in how people read when online or on devices. People don’t typically read on their phone for pleasure. (Unless they have pulled up their Kindle app because they couldn’t stay awake last night to finish that chapter that was pulling them into some awesomely great literature.) People read on devices to get them somewhere. It’s very purposeful reading and they don’t pay attention to extraneous stuff. So they aren’t reading your questions in great detail, they are reading them quickly to know how they should frame their answer.
This goes against all traditional research training, I know. Research on research for decades has shown how important the phraseology of a question can be in impacting the way people answer. And I still believe wording is important--nay, critical. But part of the criticality of it is how brief it can be. So keep it to 128 characters or less.
One way that I have found is helpful in accomplishing this is to move away from phrasing questions that use traditional research style-speak--a kind of impersonal, indirect way of writing. (“We would like to know…”) and replace them with a more direct, more personal style. One that requires you to find your voice. It’s what good writers do. Writers of books and surveys. (I will take on this topic in more detail in another email.)
Lesson #3: Recognize the limitations
The screen size is the obvious limitation, but so are other things. Think about the context people might be taking the survey in. They are likely to be killing time as they wait for a doctor’s appointment, or reading their email on the bus. They may be waiting in the airport for their flight to board. You get the picture. They may not know how much time they have to be on the phone or may be easily distracted. So you need to be sure your survey itself is brief and engaging. Also consider that they may be doing this on two bars of service, so downloading large graphics files or the delay when they press Next may be interminable.
There is also the fat finger effect. I suffer from it. And know others who do. The implications are that your open ends will have some very interesting spellings. (Also, sometimes you will need to translate from Autocorrect mistakes.) More importantly, make sure any need to select answers or press the Next button are done large enough to be easily done.
Don’t use pop-ups or new windows. Embed the elements that would be a pop-up or go to a new tab into the survey instead.
Lastly, in terms of limitations, avoid using large logos and creative assets in the survey. Think of how much time it takes for your logo to load with each question. And how much real estate it takes up on screen. And, in the end, the respondents are not normally your customers--so brand impressions are not normally that important in a survey. (Ok, if you are a brand that is doing DIY research with customers, and not blind, that doesn’t apply. I get it.)
But enough of the glass is half-empty view.
Lesson #4: Consider taking advantage of the new possibilities
The glass is half-full perspective--with mobile there are things you can do that you can’t with a laptop.
Most phones have a camera that can do still shots or video. Did anyone say pseudo-ethnography? Or “take a selfie” to prove they are a real person? The possibilities are endless and fascinating. Think about rather than asking them what is in their fridge, actually having them take a picture of it. Or asking them to take the survey in the grocery store. Or at their desk. Just make sure you take into consideration that they may start the survey in a situation where they can’t do the assignment and either give them a heads up in the invitation or allow them to come back to the survey when they are in the right place.
A phone also always has a microphone. What if instead of typing in an open end you had the respondent record their answer by voice for you? With the abundance of inexpensive transcribing services, this may not be as much of a nightmare to analyze as you might otherwise think. And there is a certain credibility and impact that hearing an answer has over the written word.
Here is an idea whose time may have finally come. If you want to follow-up with qualitative, you can totally connect on Zoom or on the phone immediately for a follow-up interview.
Lesson #5: Consider moving into the phone comfort zone
Lastly, consider formatting questions so they are like a messaging experience--what almost everyone with a phone does--rather than a survey experience. It is sure to be more engaging and will feel more native and comfortable to the respondent. There are some platforms that actually use a messaging format to ask and get answers. Or you can just mimic the format.
I hope this is helpful. If you are a user of research, this hopefully helps you know what to look for in the survey questionnaire. if you are doing the DIY thing, then I hope you are able to learn a thing or two from this. (And sales pitch here--one thing I do in the "if you can't beat 'em, join 'em mode is to provide training and help with design including questionnaire design for just these types of needs.)