eLmuse

Members

Gerken, Reiterer, Dierdorf, Leifert, König, Schmid, Gebhardt

Description

In the field of Human-Computer Interaction and Information Visualization, the majority of the evaluation methods applied collect data at a single point in time, e.g. in controlled experiments. Although such experiments discover valuable findings, certain drawbacks of the research method have been discovered. For example it is questionable whether an appropriate amount of training can be given to control learning effects or whether usability flaws might disappear over time, since users adjust their strategies. Longitudinal research methods, collecting data at several points in time, try to accommodate for these drawbacks, but have rarely been applied in human-computer studies although they are commonly used in other partially related fields such as social sciences and psychology. This project aims at building up a research framework for such longitudinal research methods by defining research questions, measurements and data analysis methods. Therefore certain studies will be carried out in different domains such as input devices, interaction techniques and information visualizations. Such a comprehensive longitudinal research framework should serve as a decision help for researchers and practitioners to choose the appropriate and tailor-made methods and procedures when dealing with a specific research question. In the long-term we hope to establish longitudinal research methods as a common practice in the usability evaluation toolbox.

Within the social sciences, which have served quite often as the methodological basis for human-computer evaluation methods, collecting and analyzing longitudinal data has emerged during the last 30 years as an important and indispensable research approach. So for example, longitudinal data allows researchers to study how the perceived quality of human relationships with and without children changes over the years or in market research to study how people change their level of consumption of certain products.In research literature this is expressed by an increasing number of publications in this field [1, p. vii]. Longitudinal data could be defined as follows: "basically, longitudinal data present information about what happened to a set of research units [in our case the participants of a study] during a series of time points. In contrast, cross-sectional data refer to the situation at one particular point in time" [1, p.1]. This definition also shows that longitudinal data could be obtained with one single retrospective study, where participants are asked about e.g. their attitudes or behaviour at several distinct time points in the past. Furthermore, although it is quite common it is not necessary to obtain longitudinal data in the field. A set of laboratory based experiments, inviting the same participants again and again can have certain benefits compared to a cross-sectional study as well as compared to a longitudinal field study.

In Figure 1 we present a classification of evaluation methods by means of four different dimensions: type of measurement, type of task, place, and duration of the study. We think that, although this classification can’t cover all dimensions relevant for evaluation methods, it allows us to create a huge design space for longitudinal research methods and specifically points out that such methods aren’t limited to field studies with real tasks and qualitative measures.

Literature:

[1] Toon W. Taris, „A Primer in longitudinal data analysis“, London: SAGE Publications, 2000.