Return to site

Usability Study of

High Efficiency Shower Heads

Understanding if checking for a leak is associated with finding a leak, and therefore fixing the leak

· UXresearch,UsabilityStudy,Statistics,Surveys

Overview

My Role: Lead UX Researcher with a team of 1 product manager and 3 research assistants

Project Overview: The big overarching question to understand: Does launching high-efficiency showerheads and conservation kits lead to improving water conservation efforts?

Methods: survey creation and concept testing

Data Sources: surveys

Skills: designing, facilitating, statistics (Structural Equation Modeling and cross tabs), and thematic analysis

Deliverables: 1) research report and 2) recommendations for survey change

Tools: PowerPoint, SurveyMonkey, and STATA 14

Background/Context: To support water conservation efforts in Orange County, high-efficiency showerheads and conservation kits were distributed to school children at various in school assemblies. Surveys were also included in the kits and returned to understand the user experience of conservating water using low flow shower heads and dye tablets to test water leaks. The kit and its contents are shown below. (see Image 1).

Client: Discovery Cube - Orange County and Golden State Water Company.

Image 1. Each Golden State Water Company conservation kit includes two low-flow aerators, a low-flow showerhead, one pack of dye tablets to test for water leaks, and easy-to-use instructions. When the kit is used correctly and in its entirety, it can potentially save an average household more than 2,100 gallons of water per month. Source.

Objectives

1) Understand the issues in installing low flow showerheads and faucets

2) Understand the impacts of using dye tablets to check leaks on fixing a leak

Opportunity and Process

Opportunity: An unknown amount of surveys and water conservation kits were distributed from 2010-2014. Over 4413 surveys were returned. Surveys were edited and changed based on hunches across four years without any usability testing. Moreover, there has been a backlog of 4 years of data that were entered but were not analyzed. This became an opportunity to not only understand the user experience of the conservation kits but also an opportunity to improve the surveys that is informed by the usability testing.

Process. Quantitative analysis and thematic coding were used. The reason behind this choice was primarily because of the ad-hoc and retrospective study design. See Figure 1 below for the project timeline.

1. For quantitative analysis, crosstabs and mediation models were analyzed using STATA 14. Using statistical software was used to analyze quickly the types of behaviors that users were engaging with, in particular when it comes to checking for leaks and fixing leaks.

2. For qualitative analysis, I led thematic coding of an open-ended survey item with three research assistants. Thematic coding was used given the open-ended nature of the question. The question focused on why users did not install showerheads, bathroom sink aerators and kitchen aerators. The open-ended question provided an opportunity to dig into the reasons that may have prevented water conservation efforts using the water kit.

Figure 1. Project Timeline

Strategy

Data Sources: It is unknown how many surveys and kits were distributed between 2010-2014. What is only known is the number of surveys that were returned (N=4413) via SurveyMonkey.

The 2010-2011 year has n=1599 surveys collected, 2011- 2012 year has n=1689 surveys collected, 2012-2013 year has n=589 surveys collected, and the 2013-2014 year had n=536 surveys.

Analysis. I used both qualitative thematic analysis to create themes of issues of installing low flow showerheads and faucets using MAXQDA 18 software. I used cross tabs and mediation models to understand the effect of checking leaks on fixing a leak for participants using STATA 14 software.

1. Direct Effect: A direct effect is a model that shows what the direct effects between an independent and dependent variable (see Diagram A). The arrow shows the direction of the effect size and the number on top of the arrow is the effect size.

Diagram A. Direct Effects of Checking a leak predicting Fixing a leak

2. Indirect Effect: A mediation model shows the indirect effect of an independent variable and dependent variable (see Diagram B). The reason a mediation model was used was to understand if finding a leak mediated checking and fixing a leak. In other words, does checking a leak lead to finding a leak and fixing a leak? Or is it mainly checking a leak leads to fixing a leak?

Diagram B. Indirect Effects of Finding a Leak of Checking a Leak and Fixing a Leak

Outcomes

Objective 1. Understand the issues in installing low flow showerheads and faucets

On average the percent explained by the indirect effect is 42.60% for 2011-2012, 32.39% for 2012-2013, and 23.73% for 2013-2014 (see Table 1). There is a decreasing trend on the percent explained by the indirect effect as time goes on. Please check the full report for the rest of the tables.

Moreover, a large percent of the effect of checking a leak to fixing a leak is not because of finding a leak. There is something else that is helping participants from checking the leak to fixing the leak but it is not just because they found a leak.

On average the actual number of people fixing leaks explained by the indirect is 57.61% for 2011-2012, 43.33% for 2012-2013, and 35.59% for 2013-2014 (see Table 2).There is a decreasing trend on the actual number of people fixing leaks explained by indirect. Less and less people are fixing leaks because checking for leaks helped them find a leak. Please check the full report for the rest of the tables.

Objective 2. Understand the impacts of using dye tablets to check leaks on fixing a leak

The 2010-2011 GSWC survey allowed a fill in the blank free response for survey takers. Survey takers were allowed to fill in why they were not able to install the shower head or the faucet heads. Using inductive analysis and descriptive coding of the free response comments, the codes and sub-codes were created as well as operationalized (see Table 8 and 9 in Appendix). From there the sub codes from Table 8 and Table 9 were grouped together to form 7 themes (see Table 7 in Appendix) that cut across why survey takes had trouble installing shower head or faucet heads. Participants had trouble installing shower head and faucet heads because they:

  1. Have had no time to install the aerators/shower head

  2. Plan on installing later

  3. Never received the aerator/shower head

  4. Trouble Installing the item

  5. It costs lots of money to install

  6. I do not know how to use it or what it is

  7. I am moving, remodeling, have no shower or no need for the item.

Deliverables

1. Research report attached here.

2. Recommendations

Archiving Current and Future Programs

Throughout the data preparation process, it was difficult to find the original surveys from 2010-2011 and 2011-2012. Due to the decreasing trend, the way the surveys were created from the earlier years may explained the higher percentages (survey layout, questions etc). Moreover, it is hard to be able to understand how the actual curriculum, presenters or materials from previous programs explain the higher percentages since there is no archives.

Study Design

It is recommended that there be a pre/post design administered with a comparable control group. Mediation models and cross tabs can be run to see if the program had any differences on fixing leaks or if merely just asking someone to check for leaks would help find a leak and fix a leak. The same survey would be administered to the comparable control group.

Changes to Survey Items

Below is the current item asking why participants have trouble installing showerheads, bathroom sink aerators and kitchen aerators (see Question 4 from current GSWC survey below):

Question 3. Did you have trouble installing showerheads, bathroom sink aerators and kitchen aerators?

Yes or No

Question 4. If you circled “No” above, choose one of the following reasons:

  1. Already have high efficacy shower head(s)

  2. Plan to install within the next two weeks

  3. Plan to save for the future

  4. Plan to give to a neighbor, family member of friend

  5. Need help to install

The seven themes are grounded in the data from 2010-2011. The current GSWC reasons item do not reflect the seven themes that came across from the 2010-2011 survey. Based on the analysis from the 2010-2011 GSWC free response on why participants are having trouble installing, it is recommended that the answers for reasons for not installing to be changed. The current GSWC survey items for Question 4, 6, and 8 should be changed. The answer choices are recommended to be changed to:

  1. Have had no time to install the aerators/shower head

  2. Plan on installing later

  3. Never received the aerator/shower head

  4. Trouble Installing the item

  5. It costs lots of money to install

  6. I do not know how to use it or what it is

  7. I am moving, remodeling, have no shower or no need for the item.

All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OKSubscriptions powered by Strikingly