Showing results for 
Search instead for 
Do you mean 
User Testing, Guerilla Style

User Testing, Guerilla Style

Myself, Sam & Matt went out to Sheffield city centre yesterday afternoon for a spot of user testing. It's the first user test I've run in the wild, previous times have been with friends & family or work colleagues. It was an interesting afternoon and we certainly learned a lot. Firstly, for those not familiar with the term, user testing is a design technique which involves watching people use the product you are designing and noting down their reactions, comments and any problems which arise. The term is a little misleading since it isn't the users who are being tested but the product. It's a particularly powerful method because it is all too easy for the designers and programmers who make websites to forget how 'real people' use computers. In this case we were tesing wireframes (essentially pictures of a website) for the current My Account redesign. So, after Starbucks turned us down, the manager of Ha Ha agreed to let us use a table in exchange for some increased custom. While I set up the laptop & camera, Sam & Matt went out into the street with placards offering "Free Coffee for 15 minutes of your time". After a few minutes Sam returned with our first victim and over the course of 2 hours or so I went through the wireframes with 5 different people. It seems that free coffee is only a sufficient incentive to attract college students. Unfortunately this meant that all our participants were from a very similar demographic with a similar level of internet use (when I asked for favourite sites, all but one mentioned Facebook, Myspace or Bebo). Perhaps next time we should extend the offer to "Free coffee and a £5 book voucher" Smiley The first thing I noticed when watching users attempt the tasks I set them is that wireframes are awkward. Because they are images rather than real interactive web pages, I found myself constantly having to call on the users' imaginations. "Imagine that's a real text box." "That's a sample error. Imagine that you've typed it correctly this time." "Imagine that the address fields are real, what would you put in each one?" Most users were quite good at imagining but one imagined maybe a little too much: pressing backspace to delete some imaginary text in an imaginary text box, she was genuinely taken back to the previous screen (backspace is a shortcut for the back button in most web browsers). I found that asking people what they think of a page out of context isn't very helpful. After the introductions, my first question was to display the wireframe and ask the users "What do you think this site is and what does it do." Responses ranged from complete incomprehension to comments on the colour scheme and graphic design. Although such a question is appropriate when testing a site home page, in the case of a control panel section it is reasonable to expect that real users would have been provided with some context from the previously viewed pages. Every web designer knows that nobody reads instructions on the web but it's still surprising to see users completely ignore your carefully written hints & help. One guy even commented "It would be useful if the page said how long my password needs be" while staring at a big yellow box stating exactly that information. Of course, one of my mantras is "If it needs instructions, it's badly designed" and the fact that people won't read the instructions anyway is just one more reason to think carefully when designing web pages. If it's self-evident then instructions aren't needed. Today, instead of updating the existing wireframes with the changes suggested by the user testing, we decided to move up to the next level of fidelity and start building XHTML prototypes. This will allow us to test again in a more realistic environment in which users can interact with all form elements and input real data. Thanks to all our participants and special thanks to Sam & Matt for braving the rain and finding those participants! I'm looking forward to doing it again.

Tamlyn ‎08-07-2008 10:20 PM
‎08-07-2008 10:20 PM
Community Veteran
Oi, some students read books you know.
Just looking...
Hey man! its good to read this blog entry - coz i get to understand quite a few things, This is particularly useful to me coz i m developing a product and we are about to roll out user testing with the UI! I follow your blog from hereon! CHeers mate!
[...] Note: cross-posted from the PlusNet Community site. [...]
Just looking...
My first time wandering the PlusNet community pages... Terriffic to hear you're doing user testing, though I've noticed a lot of good things about the service that made me suspect it was happening somewhere! Asking users to imagine is no bad thing, the cost of a proper interactive prototype is one of the things that can stop it happening altogether. And if it is too polished, it can put people off being honest. The fact that some users will use a back button to delete in a text box is a clue that keyboard behaviour in a text box should be a little different from the rest of the page (like page up/down, or Enter). Choosing your time of day and venue will also influence the users you are able to recruit, not just your incentives. Good work, keep it up!