Portfolio: Design Process Overview
The design process outlined here is intended to rapidly explore multiple solutions to design problems. The procedure is based on a description of the design space according based on the dimensions across which potential interfaces differ. Initial probe interfaces are designed and submitted to mixed-methods user tests in order to evaluate users' reaction to the different dimensions and identify optimal areas within the design space.
This process was developed to address to design problems in my doctoral research:
These are two different types of problems. In the first, I needed to tradeoff the desired behavioral outcome (shorter selections) with user satisfaction. In the second, I simply wanted to determine what highlighting interface would be most user-friendly. The procedure proved valuable for both problems.
Design Space and Interface Dimensions
The design space of potential interfaces that accomplish the same goal can be categorized according to the interface dimensions across which the interfaces differ. For example, highlighting interfaces can differ with regards to visibility and timing. With regards to visibility, some interfaces provide a visible button for students to click in order to highlight a selection, while others require students to press a keyboard button or click on the selection. With regards to timing, to highlight text, some interfaces require students to take an action (e.g. click a button) before making a selection, while others require the action to occur after making a selection.
Dimensions can be discovered through a competitive analysis of existing interfaces. For my highlighting study, I compared over 20 highlighting interfaces that implemented highlighting in a variety of ways. However, for my other study, I could find no interfaces intended to reduce selection size. In this case I first brainstormed a set of possible solutions with colleagues, and then evaluated the differences between these hypothetical solutions. I found this process actually served to inspire further novel solutions, as was also the case with the highlighting project.
The user-test involves three methods of data collection. First, participants are asked to "think-aloud" while using the interfaces. They then fill out a survey containing Likert-scale measures of their response to the specific dimensions, as well as more general questions regarding their reactions to the interfaces. Finally, a semi-structured interview is conducted touching on more broad questions in the design space, such as what type of material a student focuses on while highlighting. Using multiple methods allows for stronger conclusions regarding the value of specific dimensions or interfaces, and identifies occasions when stated preference contrasts with observed frustrations.
Each participant in the user-test is given several interfaces to try out. Interfaces are assigned to users so that each participant gets experiences with as many levels of each interface dimension as possible. This ensures that their responses are grounded in knowledge rather than prediction.
The first user-tests are intended to explore responses to the interface dimensions, not evaluate candidate interfaces. As data is gathered, certain levels or combinations of the dimensions are thrown out. For example, in the initial highlighting tests, it became apparent that users strongly preferred that the highlighting action be taken after a selection was made, not before. Hypotheses raised by the user data are also tested. For example, in the initial tests, visibility was a divisive dimension, with some users expressing a strong preference for an invisible interaction (e.g. clicking on the selection) while others strongly preferred a visible interaction (a button). The following phase explored interfaces that combined the two, allowing users to either click on a button or click on the selection, and found that solution satisfied all users.
This design process produced guidelines for manipulating selections and implementing highlighting. The resultant interfaces proved effective experimentally. However, this process has several limitations. Two of the biggest are:
The first version of this design study, which looked at restricting selection, was published at the Joint Conference on Digital Libraries 2008:
Bauer, A. and Koedinger, K. R. (2008) Note-taking, selecting, and choice: designing interfaces that encourage smaller selections. In Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '08). pp. 397-406. ACM PressA more complete description of this design process, along with how it was used in my highlighting study, is available in my dissertation:
Describing the dimensions across which the interfaces differ ensures that probe interfaces cover the range of potential solutions. It is not necessary to create all possible combinations, only enough so that each level of each variable is covered.
Coordinating data from multiple methods allows us to compare behavior and stated preference, and evaluate both interface dimensions and specific interfaces.
Levels and combinations of interface dimensions are eliminated until all remaining interfaces can be included in a single user-test.