Here I discuss
briefly the kinds of testing I plan on doing for this blog and ones you could not pay me enough to do.
Exploring
A type of testing that is often ignored is what I think of as "exploring": watching what happens when the user explores an application for the first time and tries to figure out
what it does and if they can do things with it that
they will find useful. There are hundreds if not thousands of free apps available out there for people to wade through. Many users will download an application and try to use it and make a decision about whether to keep it in a very short period of time. In general, the more that a person knows about an application the worse they are at figuring out what will confuse or frustrate a new user. If you are creating an application you should solicit user feedback as early as possible (especially to make sure you are building the right thing in the first place) and look at a broad range of users. (And when you think your UI is pretty good, hand it over to a family member who didn't have a computer until they were an adult and tends to get annoyed rather than excited by new technology. I think most software engineers would find that a real eye-opener.)
Some companies do a reasonable job of
usability testing; they want to know how easy the software is to use. However, usability testing I have seen often involves giving the user a specific set of tasks to perform and seeing where they get into trouble. At a minimum you need to have at least some beginning users and some more experienced users. (If you are lucky the customers talk out loud while they are doing this so you can learn more about what they are
trying to do or
expecting to see.) I have done this type of testing once for a college course and seen the results from one such study at Microsoft that was really fascinating. I wish more companies did this, especially in the
early stages of the design process when the companies are capable of making significant changes based on the feedback they get from customers.
(Aside: If you have never worked in software you are probably thinking, "Why would anyone spend money to do usability testing if they weren't going to use the results to improve the UI (at least not any time soon)?" I will just say that that is a
very good question. A friend once said to me something along the lines of, "Only people who
don't work in software think that the things that happen in the
Dilbert comic are ridiculous, and could never happen," which I think is very apt.
Topic: The posting of Dilbert comics at software development companies as a form of passive-aggressive behavior: discuss!)
