Research Project:
User Experience with Symptom Checkers

Understanding the socio-technical implications of symptom checkers and proposing design implications

Image alt tag

Background

An AI-based symptom checker is a type of healthcare app that solicits symptom information from users and provides medical suggestions and possible diagnoses. The rapid development of digital technology and the healthcare industry has contributed to the increased popularity of symptom checkers in application (app) markets.

However, the reliability of symptom checkers is questionable. Healthcare consumers may put their health at risk if they uncritically accept the diagnoses from symptom checkers. Recognizing that individuals’ perceptions and interpretations of symptom checkers’ responses may influence their subsequent actions, it is important to understand more fully the user experience with symptom checkers.

Problem

It still remains in doubt how healthcare consumers experience existing symptom checkers and how to make symptom checkers safer and more reliable. To fill this gap, this project aims to understand the socio-technical implications of AI-enabled symptom checkers and propose potential design solutions.

Solution

This project followed the user-centered design principles in the product development cycle by utilizing mixed research methods, such as semi-structured interviews and an online experiment.

First, by conducting qualitative interviews, we were able to collect attitudinal data and in-depth individual experiences with existing symptom checkers.

Second, based on the interview findings, we designed porotypes and then conducted a remote, within-subject experiment to evaluate the proposed porotypes, as we wanted to explore how new designs work for users compared with the existing one. By doing experiments, we could also observe how users interact with the checker.

Research methods

  • Semi-structured interviews

  • Prototyping

  • Lab-controlled experiments

My Role

  • Led and conducted the whole project in all research stages

  • Research plan creation

  • Data collection

  • Participants recruitment

  • Data analysis

  • Report writing

Timeline

  • Overall: 7 months

  • Discovery & Research: 2 months

  • Design & testing: 5 months

Team

  • 6 UX researchers

My Research Process
Research Project: Exploring User Experience with Symptom Checkers

Generative Study 1

A semi-structured interview study to explore user experiences and needs in the use of existing symptom checker apps

Research question: How do users perceive the effectiveness of chatbot-based symptom checker applications?

Timeline: 1 month

Methods:

  • App feature analysis: to shed light on the functions and features of the chatbot-based symptom checker apps

  • App review analysis and semi-structured interviews: to acquire in-depth insights as to how users perceive chatbot-based symptom checker apps

Key takeaways: The existing symptom checker apps lack the functions to support the whole diagnostic process of an offline medical visit. Users perceive the current apps lack support for a comprehensive medical history, flexible symptom input, user-friendly conversation design (e.g., comprehensible questions and human-like language style), and diverse diseases and user groups

Research Project: Exploring User Experience with Symptom Checkers

Generative Study 2

A semi-structured interview study to explore how users perceived the reliability of existing symptom checker apps

Research question: Based on the preliminary findings of study 1, we conducted more interviews focusing on "how do users perceive and understand the reliability (authority) of symptom checkers?"

Timeline: 1 month

Methods:

  • 30 Semi-structured interviews

Key takeaways: Users assess the authority of symptom checkers using various factors: including interaction design of symptom checkers (e.g., explanations might enhance users' trust in symptom checkers), associations with established medical authorities like hospitals, and comparisons with other health technologies

Research Project: Exploring User Experience with Symptom Checkers

Evaluative Study 1

Research question: Based on the qualitative findings, we noticed the importance of explanations, so we wondered "how could explanations be used to promote the diagnostic transparency of online symptom checkers?"

Time: 2 months

Method:

  • A lab-controlled experiment

  • Designed a prototype with 3 different explanation styles based on our interview findings to explore what kind of and whether explanations can improve user experience

Key takeaways: Providing explanations can improve diagnostic quality, medical decision-making, user trust in diagnosis, diagnostic transparency, and health awareness

Research Project: Exploring User Experience with Symptom Checkers

Evaluative Study 2

Research question: Still based on qualitative findings, we noticed people desired for user-friendly conversation design of symptom checkers, so we proposed the question "How should we design the conversation of symptom checkers, and how do different conversational styles influence user experience with symptom checkers?"

Time: 3 months

Method:

  • A lab-controlled experiment with follow-up interviews

  • Designed and implemented a chatbot-based symptom checker with different conversational styles to explore how these styles affect user experience

Impact of this project

  • Shedding light on the future design of symptom checkers

  1. Provide explanations to help users make informed decisions

  2. Provide emotional support to improve the user experience

  3. Offer control and customization to users

  • Policymaking

  1. Governments may confine the release of unqualified checkers and improve their transparency

  2. Health providers may guide users by introducing the limitations of symptom checkers

  • Influence on healthcare

  1. Less people have to go to the doctor

  2. Especially for people in rural area, they could use apps to check symptoms and get treatment on time

Thank you for reading my project!

Feel free to contact me!
...or just say hello on my social media.