I've come across a couple of alternatives to usability testing in a lab recently. The first is (extremely) rapid testing at a trade show booth, as described in the paper, Extremely Rapid Usability Testing.

Another is usability testing with a number of different audiences at museum conferences explained in, "Usability@90mph: Presenting and evaluation a new, high-speed method for demonstrating user testing in front of an audience".

The second method, which is done in 30 minutes, is basically this:

  1. Introduce the audience to the concepts of usability testing (which is observing users trying to conduct typical tasks on a website to identify usability problems) and site evaluation.
  2. Find a volunteer in the audience who is not already familiar with the site to test the site. Take the volunteer out of the room for a short period while showing the site in more depth to the audience.
  3. A representative of the site provides a more in-depth (but quick) overview of the site to the audience.
  4. Usability professionals, who will observe the tester, identify 4-5 tasks to be tested on the site with suggestions from the audience. These tasks should be confirmed as representative and typical by the representative of the site.
  5. While the tasks are being identified with the audience, a facilitator explains to the volunteer outside the room how to participate in a usability test (i.e., explains to the volunteer that the exercise is a test of the site, not of the volunteer, and reminds him or her to explain out loud the rationale for how he or she is trying to complete the tasks).
  6. The volunteer returns to the room to test the site by trying to complete the tasks identified. The usability professionals observe the volunteer with the audience and take notes about completion rates, problems encountered, and possible recommendations.
  7. The facilitators/evaluators discuss and present the usability issues they identified, sometimes with explanations or clarifications from the volunteer on why certain actions or choices were taken.
  8. The audience is encouraged to note any usability issues they observed.
  9. Solicit as many design recommendations to address the usability issues as possible from the facilitators / evaluators, volunteer, audience, and the person familiar with the site.

I see a lot of benefits to this method:

  • Educating large groups of people about an important part of the web design process
  • Having the input of many people (hopefully representative or potential users) contribute to identifying key tasks to be tested, usability issues, and solutions
  • Getting direct, in-person recommendations and feedback from a large number of potential users
  • Depending on the nature of the conference, the potential for a wide geographic representation

The main potential drawback I see is that by testing in front of a large audience (in some cases as many as 200 people), the volunteer participant may behave differently than he or she normally would (using a website while being observed by as many as 200 people isn't exactly replicating the user's natural environment) by being more persistent in completing tasks or just uncomfortable. The method also basically excludes testers not comfortable in front of an audience.

However, overall it seems like it could be a great complement to a round of usability testing in a lab or contextual inquiry, and I would be quite intrigued to try it out.