Novice evaluators' behavior when consolidating usability problems individually or collaboratively

Ebba Thora Hvannberg, Effie Lai-Chong Law

Research output: Contribution to conferencePosterpeer-review


An important, but resource demanding step in analyzing observations from usability evaluations is to consolidate usability problems (UPs) that were identified by several evaluators into one master list. An open question is whether consolidating UPs in pairs is cost-effective. A within-subject study examined if evaluators merge UPs differently when working in pairs than individually and what motivates their decisions. Eight novice evaluators took part. The number of discarded, retained and merged UPs, evaluators' confidence and severity of UPs in the two settings were measured. The results showed that UPs merged or discarded in the collaborative setting would rather be retained in the individual setting. Participants increased confidence and UP severity in the collaborative setting but decreased UP severity and confidence in the individual setting.
Original languageEnglish
Publication statusPublished - 2017

Other keywords

  • Usability problems
  • Collaboration
  • Consolidation
  • Severity
  • Confidence
  • Criteria
  • Usability Evaluation
  • Novice
  • Group
  • Hópvinna
  • Tölvunotkun
  • Upplýsingatækni


Dive into the research topics of 'Novice evaluators' behavior when consolidating usability problems individually or collaboratively'. Together they form a unique fingerprint.

Cite this