Return to site

A/B Testing Museum Signage

Improving signage at a children's museum to improve museum user experiences that promote childrens' thinking and learning

· ABTesting,Surveys,LearningExperiences,UsabilityStudy,Interviews

Overview

My Role: Lead UX Researcher in a team of 3 researchers

Project Overview: The big overarching questions is to understand: What are the differences between museum signage that have instructions on how to play in museums (old signs) compared to museum signage that shows messages of learning through play (new signs).

Methods: shadowing, journey mapping, survey creation, and interviews,

Data Sources: interview transcripts, and surveys

Skills: interviewing, designing, and concept testing

Deliverables: 1) research report, 2) personas, and 3) new signage at the museum

Tools: PowerPoint, Google Survey, and Excel

Background/Context: To better support the museum user experience and advocate for learning through play, I conducted A/B testing and using the old museum signage and paper prototypes. The old museum's signage focused more on challenges and activities (see Image 1). The paper prototypes shared research findings of learning through play (see Image 2)

Client: Brown University's Causality and Mind Lab and Providence Children's Museum

National Science Foundation Award: 1223777

Image 1. Old signs at the Museum have challenges like "How would you like to set this up?" Source and Source

Image 2. Paper Prototypes, the new signs, that shared research findings on learning through play tested by covering the old signs. Source.

Objectives

1) Understand what the attitudes and beliefs about learning and the museum experience with signage focused on direct instructions (old signs)

2) Understand what the attitudes and beliefs about learning and the museum experience with signage focused on research on learning through play (new signs)

3) Compare and contrast whether or not attitudes and beliefs about learning through play shift with signage focused on research on learning through play

Opportunity and Process

Opportunity: Museum leadership wanted to test to see if informing users of research-based findings would enhance user experience and learning experience. A/B testing was conducted with the original museum signage that focused on instructions (see Image 1) and tested against paper prototypes focused on research-based findings on learning through play (see Image 2).

Process. Quantitative analysis and thematic coding were used. The reason behind this choice was primarily because of the ad-hoc and retrospective study design. See Figure 1 below for the project timeline.

1. After generating the interview protocol, I conducted interviews with museum visitors on the learning and user experience with the old museum signs.

2. After putting on paper prototypes covering up the old museum signs, I conducted interviews with museum visitors on the learning and user experience with the new museum signs.

3. Thematic Analysis and Quantitative Analysis were conducted to generate reports and personas

Figure 1. Project Timeline

Strategy

Data Sources. 44 interviews and surveys were conducted with adult visitors to the museum who came with children. Interview questions included: 1) What message did you take from the signs? 2) Did reading them make you think about your visit in a different way? and 3) What advice do you have for someone else with the same age as the children you came with?

Analysis: I conducted thematic coding of the interview questions and analyzed then quantitatively compared and contrasted the old and new signs.

1. Protocol Creation. A survey was created as well as an interview protocol. Surveys were created using Google Surveys and Interviews were audio recorded.

2. Interviews about Usability of Old and New Signs. Adult caregivers were approached and asked if they would like to be interviewed. They were asked to walk around and read the signs.

3. Synthesis of Data and Report Creation. Interviews were transcribed, thematically coded, and quantitatively compared. Personas and reports were generated of the tip sheets (See Deliverables 1 and 2).

Outcomes

Objective 1: Understand what the attitudes and beliefs about learning and the museum experience with signage focused on direct instructions (old signs)

Based on the prior interviews, 8 themes emerged and surveys were tabulated for the old signs. See Figure 2.

Figure 2. Old Sign Survey (N=20)

Objective 2: Understand what the attitudes and beliefs about learning and the museum experience with signage focused on research on learning through play (new signs)

Based on the prior interviews, 8 themes emerged and surveys were tabulated for the new signs. See Figure 3.

Figure 3. New Sign Survey (N=24)

Objective 3: Compare and contrast whether or not attitudes and beliefs about learning through play shift with signage focused on research on learning through play

After the A/B testing, new and old signs were compared (see Figure 4). Based on this graph, the new signs helped museum visitors understand the concept of learning through play in all domains.

Figure 4. Old and New Signs A/B Testing

Deliverables

1. Research Report of Visitor Responses to the Hanging Signs here

2. Personas of 4 Users

3. Final Signs Used Based on the A/B Testing about Learning and Play

This project is supported by the National Science Foundation under Grant Number 1223777 to Brown University in partnership with Providence Children’s Museum. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OKSubscriptions powered by Strikingly