Week 8 – Testing Prototype with Think Aloud Method

by rivartub

This week we were supposed to test our software prototype with real users. Before we actually did that, we further improved our prototype regarding a better background (more easy to discriminate bubbles and background), several hints for the user to make the game much easier to use (e.g. correct distance to the camera) and understand (e.g. countdown timer).

As learned in the lecture we used the Think Aloud method for our user tests. The idea is that a user gets some tasks he has to try to achieve and during that process he is supposed to comment loudly on what they are thinking regarding the state (What they think is happening?), the goals (What they try to achieve?) and their actions (Why they are doing something specific?). We introduced the method by explaining it to them and giving them the task to make a black coffee at a coffee machine, just to get a feel for it and to get them a coffee to light up their mood 😉

Test Setup
Our test setup was the following. We used a conference room in the TEL building that has a particular big television (see photos below). We asked for some personal information (Gender, Age, Nationality, Profession, Prior Experience) and introduced our method as mentioned above. We recorded their thoughts with a smartphone and took notes while they were playing our game.

1. Reach a selection menu
2. Play “Against The Time”

Test Protocols:
Person 1
Gender: Male
Nationality: German
Profession: Student
Major: IT
Age: 25
Experienced with application area. (Was playing with Kinect before – didn’t like it too much though.)
Major Issues:
– Bubble colors need higher contrast – too easily mismatched –> Priority: High
– Restrict Bubble Positions (when feet are red, then red bubbles should be in lower part of the game) –> Priority: Low
– Player Bug: Sometimes 2 player scores are displayed, which was quite irritating for the user (see pictures below) –> Priority: Medium

Person 2
Gender: Female
Nationality: Spanish
Profession: Teacher
Age: 24
No Experience with application area.
Major Issues:
– Tempting Bubble: Further hint needed, that hand should go to bubble

Possible Solutions:
– Better colors according to Human Factors concepts, maybe higher opacity, bubble position detection (no overlays) (not applying yet)
– Restrict bubble positions (not applying yet)
– Player Bug: Restrict player number in certain action screens (do not add everyone)
– Tempting Bubble: hint text or pointer (so that it’s clear)

At the end some photos of our test setup and how the game is looking right now. We think that we’re on a good way, although some details have to be improved as stated above. A proper highscore screen where the user can take a photo of himself is still missing, but we’re working on it 😉
IMG_20140117_151803 IMG_20140117_151808 IMG_20140117_151812 IMG_20140117_151817 IMG_20140117_151828 IMG_20140117_152008