How do you decide between open-ended and closed-ended questions in your survey?

I generally use close-ended questions in large scale quantitative studies for practical reasons, i.e., easier and faster recording and tabulation. However, I would use open-ended questions for non-standard questions especially at pre-test stage to capture the broadest spectrum of possible responses, and make the close-ended options truly exhaustive.

As a behavioural researcher, I posit that the current dependence on closed-ended surveys, which are inherently susceptible to biases due to their restrictive nature, is expected to diminish. This shift is anticipated as we increasingly integrate artificial intelligence (AI) capabilities to scrutinise and interpret open-ended responses and interviews. Such a transition is likely to mitigate biases inherent in the formulation of survey questions, thereby enhancing the reliability and depth of insights garnered from research participants.

We specialize in using open-ended questions in our research and we firmly believe that the depth that can be explored by using the right mix of OE and CE questions is unparalleled. However, OE data can be complicated to analyze, and the challenges must also be considered. As to when to use OE vs CE, we follow a simple process. When we already know all the possible outcomes of a question, it is advisable to use a close-ended (CE) question. However, when we are unsure of 'all' the possible responses to a question, we prefer to ask it in an OE manner. So anything related to demographics, purchase funnel, consumption occasions, etc. could be suited for CE, but anything related to perceptions, imagery, or subjectivity, could be an OE question.

Open-ended questions can be used for a few cases like: • When you wis to explore unknown or new phenomenon. • When you need a more comprehensive, insightful analysis using opinions or experiences. • When there is a need to gain valuable context & understanding using qualitative data. • When the emphasis is more upin exploring to gain in-depth insights.

Be mindful of the respondent’s privacy and avoid asking overly personal or sensitive questions unless absolutely necessary. Let respondents know that there are no right or wrong answers and that you value their honest and candid responses.

The sequence of questions can influence the responses. Avoid placing sensitive or difficult questions at the beginning of the survey. Start with general questions and gradually move to specific ones.

It is important to consider 'all' possible options while designing a close-ended question. Many researchers often ignore certain possibilities, thereby forcing respondents to choose between the available options, leading to an avoidable bias. Also, it should be noted that the CE questions should not have overlaps and should be mutually exclusive to get the best results.

Use close-ended questions when: • You need to measure something specific with questions that offer pre-defined answers. • When there is quantifiable data, and straightforward Q&A is desired. • When a easy & simpler analysis could be done statistically.

While most researchers understand the value of open-ended (OE) questions, its practice is still relatively less due to some obvious challenges. 1. OE data is much more complicated to analyze. They usually require an additional layer of semantic analysis, apart from the statistical analysis that all research requires. 2. OE questions need more time to answer and take more effort from the respondent's brain. So their use needs to be properly spaced and smartly paced to get the best output. 3. OE questions also lead to easier fatigue for the respondents, so excessive/too complicated OE questions could end up with lazy and inaccurate responses that defeat the purpose of the research.

Consider using / inplementing ai probing techniques for OEs in your survey if your platform supports it - this can (in some cases) help get much richer detail from a respondent.