a multimodal, ubiquitous tool for conducting diary studies and Experience Sampling Methods (ESM)

This project is supported by the Customer Research Center of Daimler AG.


Dierdorf, Gann, Ahmmed, Akram, Ali


 As a result of a rapidly evolving information landscape, which is more than ever being considered as ubiquitous and mobile, traditional data gathering methods from human-computer interaction and other disciplines have at times reached the limits of their usefulness. Researchers of various domains, however, are interested in investigating situations, processes, behavior, emotions, or interactions from within their natural environments. Controlled experiments with their artificial emulation of reality cannot meet this demand. Likewise, observations turn out inadequate when one wants to investigate subjective experiences of people. Moreover, they can only be seen as a snap-shot since they do not cover time. However, diary studies or Experience Sampling Methods (ESM) can be implemented when other methods cannot provide the required level of insight: for the investigation of phenomena from within their natural, spontaneous context over a larger period of time. Therefore, they contribute to other research methods as a valuable addition.


Since most traditional diary studies pose a great burden on the participant as well as a challenge on the researcher, we pursue two main goals:

  1. Leveraging multimodality, high connectivity, and sensors of contemporary smartphones to significantly alleviate the data gathering process for the participant while improving on data richness at the same time
  2. Providing an intuitive, yet powerful tool for researchers from all disciplines (i.e., non-tech savvy people) that allows creating and managing complex condition chains.


To help making our vision come true we will create three tools that are designed to act jointly. These are:

  1. Mobile client: The smartphone-based application allows to capture events in a convenient manner by making use of multimodality. Modalities can flexibly be combined to describe events precisely and suitably. The composed notes are transferred to a centralized server via 3G/Edge/WiFi. Once transmitted, they are immediately available to the researcher. Since PocketBee implements a two-way communication the researcher is able to modify the study on an individual basis, e.g. ask for more details about a reported event from participant C.
  2. Study designer: The PocketBee event architecture allows creating powerful condition chains. Events derived from sensors and/or via human recognition can trigger explicit and/or implicit data gathering. However, creating and maintaining overview about condition chains can be difficult, especially when complexity increases. Therefore, we develop a tool that allows a) visually creating intricate condition chains while b) reducing complexity at the same time. This is achieved by employing a zoomable user interface (ZUI) approach with semantic zooming.
  3. Study interpreter: Analyzing a large amount of mostly qualitative data (that can be related to time) poses a great challenge on any researcher. We will investigate how we can improve visualization and interaction in a collaborative setting on a tabletop.

Publication list

  • Article
  • Book
  • Dissertation
  • Thesis
  • Proceedings
  • Other
"There was an error during trying to get the publication list. Please try again or inform the admin, if it fails again."