Myself, Sam & Matt went out to Sheffield city centre yesterday afternoon for a spot of user testing. It's the first user test I've run in the wild, previous times have been with friends & family or work colleagues. It was an interesting afternoon and we certainly learned a lot.
Firstly, for those not familiar with the term, user testing is a design technique which involves watching people use the product you are designing and noting down their reactions, comments and any problems which arise. The term is a little misleading since it isn't the users who are being tested but the product. It's a particularly powerful method because it is all too easy for the designers and programmers who make websites to forget how 'real people' use computers. In this case we were tesing wireframes (essentially pictures of a website) for the current My Account redesign.
So, after Starbucks turned us down, the manager of Ha Ha agreed to let us use a table in exchange for some increased custom. While I set up the laptop & camera, Sam & Matt went out into the street with placards offering "Free Coffee for 15 minutes of your time". After a few minutes Sam returned with our first victim and over the course of 2 hours or so I went through the wireframes with 5 different people. It seems that free coffee is only a sufficient incentive to attract college students. Unfortunately this meant that all our participants were from a very similar demographic with a similar level of internet use (when I asked for favourite sites, all but one mentioned Facebook, Myspace or Bebo). Perhaps next time we should extend the offer to "Free coffee and a £5 book voucher"
The first thing I noticed when watching users attempt the tasks I set them is that wireframes are awkward. Because they are images rather than real interactive web pages, I found myself constantly having to call on the users' imaginations. "Imagine that's a real text box." "That's a sample error. Imagine that you've typed it correctly this time." "Imagine that the address fields are real, what would you put in each one?" Most users were quite good at imagining but one imagined maybe a little too much: pressing backspace to delete some imaginary text in an imaginary text box, she was genuinely taken back to the previous screen (backspace is a shortcut for the back button in most web browsers).
I found that asking people what they think of a page out of context isn't very helpful. After the introductions, my first question was to display the wireframe and ask the users "What do you think this site is and what does it do." Responses ranged from complete incomprehension to comments on the colour scheme and graphic design. Although such a question is appropriate when testing a site home page, in the case of a control panel section it is reasonable to expect that real users would have been provided with some context from the previously viewed pages.
Every web designer knows that nobody reads instructions on the web but it's still surprising to see users completely ignore your carefully written hints & help. One guy even commented "It would be useful if the page said how long my password needs be" while staring at a big yellow box stating exactly that information. Of course, one of my mantras is "If it needs instructions, it's badly designed" and the fact that people won't read the instructions anyway is just one more reason to think carefully when designing web pages. If it's self-evident then instructions aren't needed.
Today, instead of updating the existing wireframes with the changes suggested by the user testing, we decided to move up to the next level of fidelity and start building XHTML prototypes. This will allow us to test again in a more realistic environment in which users can interact with all form elements and input real data.
Thanks to all our participants and special thanks to Sam & Matt for braving the rain and finding those participants! I'm looking forward to doing it again.