Return to site

A/B Testing and Usability Study of a Bilingual Family Science Night

· UXresearch,UXdesign,ABTesting,UsabilityStudy,LearningExperiences


My Role: Lead UX Researcher, Principal Investigator, and Manager in a team of 5 researchers

Project Overview: The big overarching questions is to understand: How do we promote science engagement between children and adults in a bilingual science night?

Methods: ethnography, surveys, A/B testing, card sorting, and concept testing

Data Sources: field notes, surveys, and observations

Skills: designing, facilitating, observing, and synthesizing

Deliverables: 1) fishbone diagrams, 2) journey maps, 3) research report, 4) prototypes, 5) final products, and 6) surveys

Tools: PowerPoint and Excel

Background/Context: In order to promote family engagement in science learning, I collaborated with a local K-8 schools to design and execute a Family Science Night using drones. Upon selecting the drone used, my team and I realized we needed to improve the instruction set to promote opportunities for family engagement as well as accessibility in Spanish.

In collaboration with El Sol Science Academy and UC Irvine School of Education

This project was funded in part by the Newkirk Center for Science and Society at the University of California, Irvine (UCI)


1) Understand how bilingual families learn science together

2) Design bilingual family science events to support science learning experiences

Opportunity and Process

Opportunity. The TR-D5 drone was selected because of its' building component and affordability. We had over 400 family members attend so we wanted affordable products that also had an opportunity to learn. The original instructions (see Image 1) was descriptive and procedural. My team wanted to ensure that families would learn together as they built the drones.

Image 1. Original Instructions in Box of the TR-D5.

Process. The process of designing and executing a bilingual family science night has rarely been done. First, my team turned to the academic literature to look at family science nights to understand what is already known. Second, we created 3 prototypes of the instructions that we believed would promote family engagement and conducted A/B testing and usability studies. Third, we piloted the prototypes and improved on them before final evaluations.

1. To create initial conceptualizations, a literature review and fishbone diagram were created.

2. To develop prototypes, we conducted A/T testing, child play and test sessions (observations, and surveys.

3. To further improve our prototypes, we piloted the instructions.


1. Initial Conceptualization. To create the first initial conceptualization. A literature review was first. conducted to understand what behaviors would promote science engagement between adult caregivers and students (see Deliverable 1).

2. Prototype Development. A packet (here) including a survey with 3 versions of the instructions and a drone kit was given to over 14 families. Adult caregivers were asked to help with the child to build a task and to help improve the instructions. The original instructions were taken out. An example of a survey is indicated below in Image 1.

Image 1. A/B Testing survey

Play/Test Sessions. We also implemented various test sessions watching and observing children and families using the instruction sheets (see Image 2.)

Image 2. Play Test Sessions with the Instruction Sheets

3. Pilot Study. Two Rounds of Pilot Studies were conducted to test the usability of both the Spanish and English instruction sheets (see Image 3-4)

Image 3. RSVP Flyer for Family Science Night

Image 4. First Pilot of Family Science Night


Objective 1) Understand how bilingual families learn science together

Based on the Fishbone Diagram, Deliverable 1, understanding the learning of family science was articulated.

Objective 2) Design bilingual family science events to support science learning experiences

Based on the literature review and prototypes, a bilingual family science night was designed to support and promote engagement between adult caregivers and students (see Media Release.)


Fishbone Diagram. From the literature review, a fishbone diagram representing the literature and design features were created.

Journey Maps.

I created a descriptive journey map of the family night experience as well as a drawing of the flow of the experience. The user flow is labeled 1-7.

Research Report.

The research report is still underway at the moment.


Below is the second iteration of our prototypes of the instruction set. One is in English and one is in Spanish (Image 5 and 6).

Final Product. Below is the final product of the instruction set which includes prompts for scientific thinking and parent engagement. One is in English (Image 7) and one is in Spanish (Image 8).

Image 7. Final Iteration of Family Science Night Drone Instructions (English) with Science Thinking Prompts

Image 8. Final Iteration of Family Science Night Drone Instructions (Spanish) with Science Thinking Prompts


Summative surveys were created in both English and Spanish


Feedback and Reactions

The event was designed around a simple yet powerful idea: bring families together, provide them with an opportunity to engage in the engineering process, and witness how curiosity, imagination, and excitement come together to create a unique and meaningful experience for students and their families.


The families in attendance represent a variety of socioeconomic backgrounds. For some families, purchasing a drone is challenging. One parent commented that if it had not been for the event, her boys may have never had the opportunity to build and fly a drone. Most importantly, it also empowered some of our young girls who were joined by their fathers in the process.


Through the work that David has done with our families and our students, specifically our 5th graders, students have been inspired to think about college and careers in the field of science. We are both grateful and inspired by the work that David is doing with our young scholars.


- Ms. Jenny Zavala, Curriculum Specialist for El Sol Science and Arts Academy

This project was funded in part by the Newkirk Center for Science and Societyat the University of California, Irvine (UCI)

All Posts

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OKSubscriptions powered by Strikingly