Usability Testing: Bill Pay
The Ask
A client had recently implemented a new bill pay module on their website. Since the launch, the client had received a lot of negative feedback from their customers. I was asked to investigate the new design, test its efficacy, compare it to the client's competitors, and make recommendations for improvements.
My Role
This was a freelance project. I functioned as the lead for the project, conducting all research, moderating sessions, evaluating results, making recommendations, and reporting the findings. Although I created the screener to recruit the participants, I used an outside service to do the cold call recruiting.
1. Discovery
I met with the client to discuss the issues they were having with their new bill pay process. I was given a sample login to investigate the flow. I also requested and was given the user feedback comment cards and analytics. Finally, we discussed the most common use cases.
Later, I provided the client with my proposed statement of work outlining the scope of the project, my methodology and deliverables.
Tools Utilized: Statement of Work Proposal
2. Research & Competitive Analysis
Click on the image to see full description of the artifact
Tools Utilized: Workflow Modeling, GOMS Keystroke Analysis
Based on the clients' information, I created a screener and 8 participants were recruited. In order to cover all of the client requested scenarios, I created a test plan assigning one of three different personas to each user with a randomized sequence of tasks that would be performed either on a laptop or smart phone.
The tests were performed in person at the clients' office. The user was sat at a table with a laptop logged into a webex meeting to record their face and screen interactions (participants were aware and signed a release for the recording).
I provided an orientation speech and then the user was given a packet of tasks to complete and were asked to speak out loud while completing the tasks.
Tools Utilized: Satisfaction Scoring, System Usability Scale
4. Review & Recommendations
In order to derive my findings from the study I rewatched all of the test videos and took copious amounts of notes, including:
-
Time on Task
-
User Path to Objective
-
Successful Task Completion
-
Insightful User Comments/Mood
I also aggregated the user's reported task satisfaction and SUS answers.
3. Testing Moderation
I performed the presentation of the findings to the client remotely via webex. The presentation included:
-
A reiteration of the testing objectives and methodology
-
Detailed Findings (both quantitative and qualitative)
-
The Competitive Analysis
-
The System Usability Scale Score
-
Recommendations
5. Presentation of Results
Click on the image to see full description of the artifact.