What we learned from the user evaluation

The evaluation test at the Animal Protection Service in Breda was packed with lessons for us to learn. We installed SnoepDog at the gate of the training field, so every dog owner in the training course would pass by.  We selected participants ourselves. First we asked them to participate. Then we introduced SnoepDog by telling what it is and how it works. Then we asked them to use it and speak aloud anything that goes on in their minds while using the device. Afterwards, we interviewed them about their experience.  Here are the main results:

-  The interface proved to be more complicated to new users than we expected.  Although we explained in length the way SnoepDog works, it took most participants minutes to figure out what to do;

- Dog owners main focus is NOT SnoepDog, but their dogs! The constant shift of attention between device and dog resulted in a lack of concentration, which made the interface tasks harder to complete;

- The fact that the users of SnoepDog are dog owners caused an unforeseen problem: because within urban space dog owners are obliged to leech their dogs, they only have one free hand to  handle the device: the other hand is used to hold the leech. The problem is that most right handed dog owners hold the leech with their right hand. They tend to use SnoepDog with their left, which makes the handling harder. This especially the case when dogowners need to grab a bag. Tearing a bag of should be made easier with a same system used to tear of aluminum foil;

- Vision and sound can be improved to make the interface clearer;

- The interface uses too much text. Participants did not take the time to read;

- Most participants were eager to proceed the process, they want to get done with it as quickly as possible;

- Strong sun light caused the screen to fade, which made it harder to read, even when the screen was at its brightest;

- Some participants seemed to be more eager to please the researchers than to figure out what to do with SnoepDog. This even when we explained to them that we want to test the system, not how well they perform;

- Young people seem to prefer the game, older people the tax reduce;

- It proved to be hard for participants to think of ways to improve SnoepDog.