Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design

1. Description

This paper focuses on identification of missing criteria that allow to evaluate usability of virtual environments and drive the effective VE design. As result a Multi-criteria assessment of usability for virtual environments (MAUVE) is proposed, that can assist designers and evaluators of VE systems in enhancing the usability of these systems. The paper expands traditional usability principles by criteria specific for virtual worlds, such as wayfinding, navigational techniques, object selection and manipulation, as well as integration of visual, auditory and haptic systems. Authors distinguish two main categories associated with VE system usability: VE System Interface (i.e. software, hardware, hci) and VE User Interface (i.e. physiological).

VE system interface has a deep impact on user ability to complete the system designs task. It is split into two main parts: interaction and multimodal system output. Interaction influences user ability to navigate and move around the VE as well as manipulate with VE objects. As a result interaction is sublassed to: navigation, manipulation and wayfinding. Multimodal system output brings possibility to present users with multiple inputs and outputs such as speech, video and sound. Therefore, we subclass this category to visual, auditory and haptic input.

Considering VE user interface, MAUVE classifies two major usability criteria: engagement and side effects. The reason for this is that use of VE system is influenced and limited by maladies that may distrupt the comfortable user experience. Engagement is responsible for enhancement of user experience. It is subclassed to immersion (psychological state characterized by perceiving one’s self to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences) and presence (psychological perception of "being" in the VE).

The last part of evaluation criteria is formed by side effects that is subclassed to comfort, sickness and after-effects that influence the user experience during and after encounter with the VE.

The MAUVE methodology uses two-stage evaluation. In the first stage, traditional usability heuristics acess the VE usability. In the second MAUVE evaluation stage, a multiple criteria decision-making technique is used to first prioritize the VE usability criteria according to the needs of a particular application. The two stages of evaluation in MAUVE can be conducted separately or in combination. Prior to responding to the second set of questions, the evaluator should have assessed VE usability criteria via either user testing (e.g., interaction, multimodal system output)


Document: PDF

Key Words: Evaluation, Virtual Worlds

2. Quotes

Designing usable and effective interactive virtual environment (VE) systems is a new challenge for system developers and human factors specialists. In particular, traditional usability principles do not consider characteristics unique to VE systems, such as the design of wayfinding and navigational techniques, object selection and manipulation, as well as integration of visual, auditory and haptic system outputs.

Virtual environment (VE) system usability, on the other hand, is just beginning to receive the focused attention needed for identifying a taxonomy of VE- specific usability attributes (Gabbard and Hix, 1997; Kalawsky, 1999). Traditional evaluation techniques (Nielsen, 1993) do apply to virtual environments.

Most VE user interfaces are fundamentally different from traditional GUIs, with unique input/output (I/O) devices, perspectives, and physiological interactions. Thus, when developers and usability practitioners attempt to apply traditional usability engineering methods to the evaluation of VE systems they find few if any that are particularly well suited to these environments (Hix and Gabbard, 2002). Subsequently, very few principles for design of VE user interfaces exist, of which none are empirically derived or validated (Kaur et al., 1999).

Concerning the identification of VE usability and design guidelines led to the classification of two main categories associated with VE system usability. These include VE system interface (i.e., software, hardware, and overall man-machine interaction design) and VE user interface (i.e., physiological, psychological, and psychosocial) considera- tions. Similarly, Gabbard et al. (1999) suggest that two distinct domains make up interactive system development concerns: behavioral (i.e., view of users and their interaction with the application); and constructional (i.e., view of software developers and the system design).

VE System Interface

The usability of any VE system is thus influenced by VE system components as they impact users’ ability to complete tasks for which the system is designed to support (Bowman, 1999; Gabbard et al., 1999). Bowman et al. (2000)

Interaction should be natural, efficient, and appropriate for target users, domains, and task goals. Bowman (1999) proposed that any interaction within a VE may fall into 3 general categories: (1) travel (i.e., movement of user’s viewpoint from place to place); (2) selection (i.e., targeting virtual objects within an environment); and (3) manipulation (i.e., setting the position and/or orientation of virtual objects).

... usability criteria associated with interaction have been herein sub-classified as: wayfinding (i.e., locating and orienting oneself in an environment); navigation (i.e., moving from one location to another in an environment); and object selection and manipulation (i.e., targeting objects within an environment to reposition, reorient and/or query)

Wayfinding

Virtual environment navigation involves how users manipulate their viewpoint to move from place to place within the environment (Bowman, 1999; Hix and Gabbard, 2002; Gabbard et al., 1999; Kaur, 1999).

If there is insufficient or inappropriate information provided about the spatial structure of a VE or identity and location of target objects, then users are likely to have difficulties locating their current and/or desired destinations (Kaur, 1999).

Charitos and Rutherford (1996) define general requirements for designing spatial structures (e.g., paths should have a clear structure and start/end points; landmarks should be easily identifiable and recognizable with a prominent spatial location).

Navigation

Aside from simple head movements, navigation is the most basic and common type of interaction within a virtual environment (Bowman, 1999). For most VE users, navigation (i.e., travel) is what is necessary to allow users to move into position to perform required tasks.

For example, it is evident that users should be able to interact with and control their movement throughout a VE in a natural, streamlined fashion, while the method(s) of movement allowed should be flexible enough to support all aspects of a task (Gabbard and Hix, 1997; Kalawsky, 1999; Kaur, 1999).

Object Manipulation

Object selection and manipulation may be defined as the process of indicating virtual objects within an environment to reposition, reorient, or query them (Bowman, 1999; Gabbard and Hix, 1997).

Basic interaction issues (e.g., not being aware of which objects are active) appear to be more prevalent in VEs than in traditional direct manipulation interfaces because, unlike the latter, VEs have yet to establish standards, such as how to define active objects (Kaur, 1999). Furthermore, the added spatial dimensions intrinsic to VEs may place more demand on object manipulation precision.

Gabbard and Hix (1997) and Bowman et al. (2000) note that most of the gestures needed for an intuitive interface exist in current western culture (e.g., pointing to select, curling the index finger toward oneself to summon another, folding one’s arms to portray discontent). Furthermore, since manipulation of objects or tools in the real world is typically performed directly with the hands, hand gesture recognition and interpretation can serve as major components of direct manipulation interaction (Krapichler et al., 1999; Poupyrev, 2000).

Multi modal system output

A main feature of VE technology is being able to present users with multiple inputs and outputs, such as speech, video, and sound (Mills and Noyes, 1999). Multimodal output involves visual and auditory displays, as well as haptic feedback. These modalities are used to engage human perceptual, cognitive, and communication skills in understanding what is being presented in a virtual world (Turk and Robertson, 2000).

The human visual channel is considered the most powerful sensory system. Visual cues can be extremely compelling to computer users (Kalawsky, 1993). Conse- quently, the visual interface may provide users with the most salient and detailed information regarding a virtual environment (Durlach and Mavor, 1995). Trying to optimize the use of human visual sensory capabilities to create desired effects, however, often constitutes an overwhelming challenge for system designers. Users are particularly good at noticing even slight irregularities in a display, such as distortions or lags in presentation of visual images (Kalawsky, 1999).

Aural feedback may include sounds resulting from users’ own actions, others’ actions, and natural or ambient sounds (Gabbard and Hix, 1997). Sound may be used to enhance perception and improve user performance within a VE by increasing users’ physical and spatial awareness. Sound may also be an effective sensory substitution, such as when no haptic feedback is available (Bowman et al., 2000).

Bowman et al. (2000), Durlach and Mavor (1995), and Gabbard and Hix (1997) utilize the general term ‘haptic’ output when referring to either force or tactile feedback, as both support user-specified input and provide VE users with a sense of touch and feel. Gabbard and Hix (1997) and Durlach and Mavor (1995) suggest that two types of haptic feedback are generally provided: (1) kinesthetic—information sensed through movement and/or force to muscles and joints; (2) tactile— information received through nerve receptors in the skin (e.g., at the finger pad), which convey shapes and textures.

VE User Interface

Use of current VE systems is limited by maladies (e.g., discomfort, sickness, physiological aftereffects) that may jeopardize the experience (i.e., presence, immersion), well-being, and learning capabilities of users both during and after VE interaction (Stanney et al., 1998b).

User engagement in a VE should be fostered and sustained, thereby enhancing sense of presence. J. Maida (personal communication, November 25, 1997, as cited in Stanney et al., 1998b) suggests engagement is related to the following factors: level of ‘‘user motivated’’ interaction with a VE system; level of reaction by a VE system to users’ actions, which should be robust and have no perceivable lag or distortion of motion (Adelstein et al., 1996; Hansen and Haas, 1998); and motivation by users to continue to maintain involvement in VE interaction (e.g., ‘‘play the game’’), which may be independent of realism.

Kalawsky (1999) discusses the confusion often surrounding these two terms because they are frequently used interchangeably. Immersion may be referred to as the extent of sensory information, which is a function of VE system components (i.e., stimulating physiological factors), while presence refers to a user’s sense of ‘‘being there’’ in an environment (i.e., psychological factors) (Sadowski and Stanney, 2002).

Stanney et al., 1998b). When exposed to a VE, users are simultaneously in both the virtual and physical world. Considerable sensory discordance may result due to mismatches between perceived and expected stimuli. Such sensory conflict has been viewed as a primary cause of motion sickness (DiZio and Lackner, 1992).

Due to sensory discordance inherent to interacting with VE systems and need to adapt and readapt to the virtual and real worlds, disturbing effects that may result and linger post-exposure include: head spinning; postural ataxia; reduced eye-hand coordination; and other visual and/or vestibular disturbances.

MAUVE

The MAUVE methodology uses a two-stage evaluation process. In the first stage, traditional usability heuristics are used to assess VE usability. The heuristics are based on those proposed by Nielsen (1993) and have been modified to take into consideration the multimodal interaction inherent to VE systems. In this stage, an expert evaluator (i.e., usability practitioner) responds to a series of questions related to each of ten usability heuristics discussed in Nielsen (1993). The responses are tabulated to determine a score for each heuristic, indicating how well the system performs in each of the 10 categories. The evaluator is then presented with a score for each heuristic, as well as indications of problematic areas where scores are low. The results may be used to correct critical usability problems, or to enhance the usability design of future systems.
In the second MAUVE evaluation stage, a multiple criteria decision-making technique is used to first prioritize the VE usability criteria presented in the hierarchy in Fig. 1 according to the needs of a particular application. The evaluator then responds to a series of questions (see Table 2) related to each of the VE usability criteria listed at the lowest level of the hierarchy, to determine how well the VE system under evaluation performs with respect to those criteria. The final score for a VE system is synthesized by multiplying the criteria weights by how well the VE system performs in terms of the lowest level criteria.
The two stages of evaluation in MAUVE can be conducted separately or in combination. Prior to responding to the second set of questions, the evaluator should have assessed VE usability criteria via either user testing (e.g., interaction, multimodal system output), questionnaire responses (e.g., engagement, side effects), or a combination of both. Sadowski and Stanney (2002) have reported several different techniques to assess presence and immersion. The Simulator Sickness Questionnaire (Kennedy et al., 1993) can be used to assess sickness.

3. BibTeX

@article{Stanney:2003:UEV:777550.777555,
 author = {Stanney, Kay M. and Mollaghasemi, Mansooreh and Reeves, Leah and Breaux, Robert and Graeber, David A.},
 title = {Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design},
 journal = {Int. J. Hum.-Comput. Stud.},
 volume = {58},
 issue = {4},
 month = {April},
 year = {2003},
 issn = {1071-5819},
 pages = {447--481},
 numpages = {35},
 url = {http://portal.acm.org/citation.cfm?id=777550.777555},
 doi = {10.1016/S1071-5819(03)00015-6},
 acmid = {777555},
 publisher = {Academic Press, Inc.},
 address = {Duluth, MN, USA},
 keywords = {cybersickness, multimodel interaction, navigation, presence, usability, virtual environments, wayfinding},
}