Adapting Conjoint To The Mobile Phenomenon
With “smart” devices like smartphones and tablets integrating the “smartness” of personal computers, mobiles and other viewing media, a monumental shift has occurred in the usage of smart devices for information access. The estimated sales of smart devices crossed the one billion mark in 2013 (Gartner report, Nov 2012). The widespread usage of these devices has impacted the research world too.
A study found that 64% of the respondents preferred smartphone surveys, with 79% of them preferring them due to the “on-the-go” nature of them (Research Now, 2012). Prior research (“Mobile research risk: What happens to data quality when respondents use a mobile device for a survey designed for a PC”, Burke Inc, 2013) has suggested that when comparing the results of surveys adapted for mobile devices to those on personal computers:
- Respondent experience is poorer
- Data quality is comparable for surveys
This research dissuades conduction of complex research techniques, like conjoint, on the mobile platform.
Via this paper, we demonstrate some approaches for conducting choice-based conjoint or its equivalent on the mobile platform. Through detailed diagnostics, we provide direction as to how to adapt conjoint surveys to the mobile platform, and when to apply a specific approach. We highlight the challenges of conducting conjoint (specifically adapted to the mobile platform) and potential ways to overcome them.
The research highlights the differences in data and respondent experience while taking conjoint surveys on mobile platforms versus personal computers. Furthermore, we establish that we can simplify/adapt different conjoint techniques to the mobile platform in order to create a better survey experience for the respondents.
To achieve our objectives, we conducted quantitative surveys in the US and India. For each country, we conducted a quantitative survey among 1000 respondents. The topic of research was evaluation of various brands of budget tablets (nine attributes were evaluated). The sample distribution is as follows:
- Personal Computer (PC)
– Choice-Based Conjoint (200 respondents) – Conducted to establish a benchmark for results
- Mobile devices (Smartphones, tablets)
– Choice-Based Conjoint (200 respondents) – Conducted to evaluate the differences in results to the CBC conducted on PCs
– Conjoint Value Analysis (200 respondents) – Conducted to understand if the simple nature of the tasks (two concepts per task) produces good results on the mobile platform. We estimated the results using HB regression at an aggregate level.
– Partial Profile (200 respondents) – Conducted to understand if limited attributes per screen produce good results on mobile platform
– Shortened ACBC (200 respondents) – Conducted to understand if the pre-screening and adaptive nature of the exercise would help in engaging respondents better. We omitted the choice tournament in the interest of keeping the survey short.
We compared the results in order to evaluate the ability of conjoint methods conducted on mobile (CBC, CVA, Partial Profile, and Shortened ACBC) to mimic survey results and respondent experience offered by CBC on PCs.
Based on the results, the key takeaways are:
- In today’s mobile world, conjoint on mobile is definitely possible. Respondents enjoyed taking a conjoint-based survey on mobile more than they did on a PC. This was true for all the techniques we tested on mobile.
- Conjoint on mobile platforms also does better in predicting hold out tasks. Only CBC on mobile predicted worse than CBC on PC. While Shortened ACBC outperformed all techniques in predicting hold outs, simplified tasks such as CVA and partial profile also did significantly better than CBC on PC in predicting the hold-out tasks.
- The respondents felt that they could comfortably read and understand all the concepts on the mobile platform. This was further confirmed by the high hold-out prediction rates.
- Respondents felt that all the product combinations looked realistic to them. This gave us confidence that the respondents evaluated each task fairly and the results were accurate.
- Respondents, probably taking the survey on-the-go, took more time in completing the surveys on mobile platform.
The results strongly indicate that conducting conjoint on mobile platforms can be done with confidence. By simplifying the techniques and following some guidelines, we can obtain better responses from the respondents.