Superhero Zip

Creating an Amazon Alexa conversational experience to help children learn social-emotional skills

User

4-8 Years Old Kids

Time

2 Years

Lab

User Empowerment Lab

My Focus

Research
Conversational Design

Overview

This academic research with the University of Washington iSchool delves into children's interactions with technology, such as Alexa or Siri, to gauge if they can derive social-emotional lessons from existing voice user experiences. The study started in 2020 with a team of five researchers. After reviewing 3,767 Skills available in the “Kids” category of the Alexa Skills Marketplace, we identified various interaction patterns but found experiences lacking depth. We then conducted 26 interviews with parents to inform the development of a web application aimed at supporting children's Social and Emotional Learning based on their ideas. We've tested a prototype of the website. My role was as a UX researcher and designer, responsible for conducting interviews, analyzing data, creating visuals and diagrams, and storyboarding.

Check out our recent 2023 ACM IDC publication!

2022 ACM IMWUT publication

Smart Speakers Can Teach Kids

Social-emotional skills are essential for children’s development of interpersonal relationships and prosocial behaviors. As effective tools, smart speakers like Amazon Echo could support children’s social-emotional learning (SEL). However, it is unknown whether the current systems meet parents' and kids’ needs for SEL and how future designs could bridge the gap.

Top 3 Challenges

To answer these research questions, I created an interview protocol to learn about parents' needs on using conversational agents (CA) for SEL and thoughts on specific design concepts. Meanwhile, a structured content review and testing of the current "Kids" skills available on the

Amazon Skills Marketplace

to examine the prevalence and design. However, I had many challenges:

Too many Alexa skills to analyze

Analyzing 3,767 Kids Skills on the Amazon Skills Marketplace was time-consuming

Alexa Skills Demonstration to 30 Kids + Parents

How to demonstrate skills in an interview to get kids and parents’ opinions on skills?

Alexa Skills Analysis

How to help team members analyze voice interaction patterns and use scenarios?

Solving Each of The Challenges

Too many Alexa skills to analyze

I first manually analyzed 100 skills but it was too time-consuming. Therefore, I communicated with a team member to create a web scraper that extracted metadata about all Skills, established a codebook with 100 Skills, and assessed interrater reliability with more sets of 100 Skills.

Alexa Skills Demonstration

To help the team analyze voice interaction for Skills that might provide an opportunity for SEL, I created user flows for representative skills using the Miro tool to better visualize interactions and identify similar design patterns between different Skill diagrams.

Alexa Skills Demonstration

Interviewing kids was not easy. To effectively communicate ideas and concepts of different Alexa Skills in a fun way, I used storyboards to model existing skills and illustrate the use scenario to kids and parents.

Impacts

Two parts of our study results include

Parents’ perspective on using CA for SEL

Prevalence and design of current Skills to support Kid’s SEL

Parents are optimistic but also have concerns such as inadequate conversationalists, value misalignment, and replacement attachment relationships. Through analyzing data and diagramming Skills, we found 5 interaction styles:

“The Lecturer”

Echo talks at the user, tells them things without interacting

“The Delegator”

Echo tries to prompt interactions between people but doesn’t participate itself

“The Bulldozer”

“Interact”s with the user but continues on its conversational path no matter what the user says

“The One-Track Mind”

Interacts with the user, but only within a very narrow script (such as forced-choice responses)

“The Doorman”

User can interact to navigate through the experience but can’t engage with the device about the content itself

Reflections

Design for kids is HARD

Understand design affordances for kids better and how designers could utilize research contributions

Don’t fight alone

Communicate with challenges and Collaborate with peers to solve problems together

1 picture = 1000 words

Simple illustrations and diagrams help with analysis and communication