Updated

Project-based research

We use user experience (UX) research methods and activities across a diverse set of projects:

  1. Call center as a source of recruitment to perform interviews
  2. Ethnio intercepts to gather data from visitors
  3. User testing to evaluate the UX and usability of a specific page

We reccomend a collaborative approach across different skill sets to successfully execute each activity.

These are the roles/skills we’ve used when implementing each of the tools mentioned above:

Rubric

📱 Call Center

The call center is a source for recruitment to gather information from groups that are harder-to-reach, like Spanish speakers (who represent a significant percentage of the total population of California and have been highly impacted by COVID-19). We use the referrals from the call center to perform interviews.

We used the call center to learn about Spanish speakers’ thoughts and feelings about vaccines.

Here’s how we use the call center to recruit interview participants:

StepsProcessToolsCOVID-19 roles and team
Meet with call center coordinatorOne-time meeting with coordinator to define the flowSpreadsheet, Slack, and emailResearcher and call center coordinator
Define screenerSend email to the call center coordinator and then forward to the call center operators.EmailResearcher and call center coordinator
Call center operators to complete a spreadsheetThe operator either 1) completes a spreadsheet with screener questions. (This can be a time consuming process for the call center operators, especially when dealing with a high number of incoming calls.) or 2) redirects the phone call to the researcher. This requires the researcher reserving time in case phone calls are redirected.SpreadsheetCall center operators
Share with researcher participants informationThe call center shares every week a list of names with phone numbers and some demographic information. The contact call center operators usually may not collect enough information to identify the best candidates.SpreadsheetCall center operator (Maximus)
Schedule an interview or receive the phone call direct from the call centerMost free scheduling tools ask for email addresses and not everyone has one. These free versions also have advertising that undermines the trustworthiness of the process.Text, Calendly, DoodleResearcher
Send reminderSome tools require a paid version to access this feature.TextResearcher
Perform interviewInterviews are usually 30 minutes.Phone
Send incentive$20 gift cards are sent through Ethnio. This requires the ODI Chief of Administration calling ethnio every time a payment is sent.Chief of Administration

🦉 Ethnio intercepts

We use Ethnio intercepts to gather feedback from existing visitors we are unable to reach out with other tools. Circumstances and context matter in user research, and someone who’s using the website in real time, for real purposes, will behave differently than someone who’s being told to accomplish some task via other platforms. We also use Ethnio intercepts as a source for recruitment./

We also use Ethnio to uncover visitor’s context of use and motivations when using the site. We use this information to better address those contexts and perform interviews to dig deeper into those motivations.

Projects where we have used ethnio intercepts include:

Here’s how to use Ethnio to gather data and recruit interview participants:

StepsProcessToolsCOVID-19 roles and team
Get or renew subscriptionPay a subscriptionEthnioChief of Administration
Assign seatsThe Chief of Administration adds or reassigns seatsEthnio
Design studyCreate a script and screener. You can duplicate an existing one and change the name.Screener
Define recruitment method: intercept, link, or native app intercept.If using an intercept, share the javascript code with the engineering team or page’s content captain to add to the desired page. There’s no need to share if we use a link.Javascript codeResearcher
Turn intercept onThere is an on-off switch in the dashboard next to each screener. It’s good practice to remove the code whenever n intercept is turned off. This is good accessibility practice and preserves our limited pageviews.Ethnio
Download responses and create a reportDownload all responses to either XLSX, CSV, or XML. There is a PDF option with graphs for close-ended questions.Ethnio/ export data

Intercept example

Ethnio intercept

📝 User testing

We perform usability testing sessions with specific audiences to test the performance of newly launched pages or prototypes. These sessions allow us to evaluate the user experience and uncover usability issues for the specific pages. We use usertesting.com to observe people as they interact with the page and perform specific tasks.

Projects where we have used user testing include:

Here’s how we use user testing sessions to assess usability:

StepsProcessToolsCOVID-19 roles and team
Get or renew subscriptionPay subscriptionUsertesting.comChief of Administration
Assign seatsNegotiate the numbers of tests and seatsUsertesting.comChief of Administration
Define goals and desired outcomes for a specific testCreate a script and screener (or duplicate an existing one and change the name) and define number and type of participantsUsertesting.comResearcher, content designer, and UX designer
Create report and share findingsUsertesting records videos and can create clips. There is a manual process to view videos and incorporate findings in a reportGoogle Slides and usertesting.comResearcher
Created and maintained by Office of Digital Innovation