Research: First Click Testing of Search Bar feature, using CKAN benchmarking

First Click Testing of Search Bar feature, using CKAN benchmarking

Research Date

July - September 2023

Summary of Research

• The Information Architecture (IA) of the 'Search Bar feature' has been built based on CKAN benchmarking & the team's collaborative prioritisation workshops

• The Search Bar features currently prioritised are documented in this Wiki

Question

• What are the most efficient task flows based on CKAN benchmarking?
• Do the task flows vary for a data expert versus data novice? What are the similarities and/or differences?

Purpose

This first click testing aims to identify if there are any gaps in the information architecture of the currently benchmarked CKAN website. The Search Bar feature has been benchmarked based on the prevalent best practices & standards used across other CKAN websites. This first click test aims to identify if the task flow & user flow identified leads to successful navigation and task completion . This will determine whether additional tweaks/iterations need to be made for Access the Data’s MVP - in order to deliver an efficient navigation & task completion experience.

Methods

Why is the first click important? Reference - User Interviews

"The first click dictates overall session success. Getting the first click right is a critical milestone in designing a user-friendly site. ‍ In 2006, Bob Bailey and Cari Wolfson conducted one of the most influential usability studies out there. Their findings are still relevant today, and will probably stay relevant for years to come. Their study revealed that when users had trouble executing the very first thing they wanted to do on a website, “they frequently had problems finding the overall correct answer for the entire task scenario.” ‍ Meaning when the first click fails, the rest of the session tends to tank as well. More specifically, when the first click is incorrect, the chance of eventually getting the overall scenario correct is about 50/50. Participants are about twice as likely to succeed in the overall mission when they select the correct response on their first click. ‍ This finding has been validated in a variety of other studies as well:

• When people get their first click right on a website, they’re 2–3 times more likely to find what they’re looking for than if their first click was incorrect.
• Participants who click down the right path on the first click will complete their task successfully 87% of the time, whereas participants who click down the wrong path tend to only complete their task 46% of the time.
• If the first click is correct, the chances of getting the whole scenario correct is 70%—but if the first click is incorrect, those chances drop to 24%".

Hence, based on the limitations & also available resources within the free plans of the usability testing tools, the first click testing was tweaked accordingly. Lesser research participants were recruited, but the first click- tests were overlapped with other open-ended & close-ended survey questions as part of the test - to triangulate the data.

Current Status

• Data collection and data analysis complete
• Research findings, insights, recommendations and next steps were presented to the team

All Issues

• Label: Research: Benchmarking
• First Click Testing of Search Bar feature, using CKAN benchmarking
• Impact of this UXR Project:
  • Provides support to the development team's current requirements as they build & deploy the landing and navigation pages of Access the Data
  • Promotion and advertisement of Access the Data's tools & services. Identifying and working with future partners/users within the communities of practice (example: data science). Better understanding needs of these users.
  • Value generation by increasing user sign-ups & attrition of Access the Data's tools & services
  • Provides a framework for future testing of other individual features as they are added & developed within Access the Data. This set of replicable processes & templates will enable research to be conducted at a faster pace as new features are continuously added & developed over time.
  • Guides & benchmarks for future products & features that need to be shipped

Outstanding task items

Assets

Google Drive Folders

First Click Testing IA of Search Bar feature, using CKAN benchmarking

Research Plan

Research Plan - First Click Testing of Search Bar feature

Recruitment Strategy

• Randomised Sampling 
• Recruited Participants via known networks & Slack channels of Hack for LA, especially all the Communities of Practice including UI/UX, Data Sciences, Ops, Engineering, Product Management, General. This also led to snowball sampling wherein Co-leads of respective Communities of Practice encouraged their community members to complete this study (as a way of understanding cross-functional workings of adjacent domains - example: UX Research & Ops).

Scripts

Interview recordings & Transcripts

Synthesis (Dovetail or Miro or Figjam)

Data Collection & Analysis - Optimal Workshop - First Click Testing of Search Bar feature

Presentation of Findings

Research Findings - First Click Testing of Search Bar feature

Action Items Spreadsheet

---

Related Wiki Pages

Research: Prioritization Workshop: Information Architecture (IA) of Search Bar feature, using CKAN benchmarking