Blended Interaction Design

Duration

01.07.2009 - 30.06.2012

Members

Florian Geyer, Ulrike Pfeil, Johannes Zagermann, Maximilian Ortwein, Tobias Zimmermann, Harald Reiterer

Description

The project Blended Interaction Design investigates novel methods and techniques
along with computational support that seek to augment the physical, cognitive,
and social aspects of creative interaction design activities.

Motivation

Interaction design is broadly perceived as a recent trend in Human-Computer Interaction (HCI) that emphasizes on designing user experiences with technology [1,2]. Its interpretations differ between the two converging perspectives: 'Interaction design as a design discipline' and 'Interaction design as an extension of HCI' [2]. In recent years, HCI research developed methods, techniques, and tools that aim on supporting interaction designers in practice. However, due to insufficient integration, application in practice remains sparse [3]. One of the reasons for this is that interaction design research 'has not been grounded in or guided by a sufficient understanding and acceptance of the nature of design practice' [4, p.56]. Consequently, there is a growing interest in the role and nature of design in HCI research [5,6]. These contributions have already increased awareness in interaction design practice [1,7] and will influence research over time.

Research Approach

In order to support design practice in HCI it is necessary to understand and describe design rationality [4], a notion that has been labeled with concepts such as the reflective practitioner [9] or the thoughtful designer [6]. More detailed descriptions and examples of disciplined behavior of designers can be found in [1,10] and [11]. These authors show that following activities, among others, are characteristic for design rationality:

• Sketching is at the core of design and is used to explore relationsships between design ideas, form and function as well as big picture and details.
• Exploring many different alternatives in a conversational way while constantly shifting focus between ill-defined problems and solutions is the prevalent work style.
• Spatial structures, like design studios and display walls make artifacts visible and provide the fundament for effective team collaboration.
• Social structures like design critique and frequent review cycles make judgements visible and are the foundation of design argumentation.
• Eventually, constant process awareness is essential to deal with the complexity of open-ended design tasks.

However, HCI struggles with integrating these 'designerly ways' [10] into practice based on several reasons:

• Because of the dynamic nature of design artifacts within interaction design - like prototype simulations or animations - computational tools are employed as powerful means of expression. However, the effort to built such artifacts often exceeds the value they have for the evaluation of design ideas.
• Due to predominant single-state document model of desktop-based software, most computational tools actually employed in practice impose a rather linear progression through design tasks [12].
• Many employed tools merely focus on the task of constructing a single design instead of supporting the messy nature of designer's explorative and spatial thinking modes [4].
• Artifacts and decisions are often buried in file systems and hide possible relations that might be valuable as part of the creative process of ideation.
• Desktop-based design tools also impose physical constraints on accessibility that makes informal creative meetings and creative collaboration awkward.
• Many of these limitations lead to a coexistence of informal artifacts like paper sketches and whiteboard drawings that are hard to archive and share but are valuable resources for design reflection and reuse.

Interaction Concepts

Based on the described principles we are designing and implementing visualization and interaction techniques based on a spatial, cross-device workspace that integrates with the physical structure of a computation-augmented design studio setting. Within our lab Media Room we are exploring various settings with interactive whiteboards, pen tablets, digital pen and paper, large high-resolution displays, and tabletops. Therefore, we combine physical interaction concepts that built upon the principles of natural interaction (e.g. tangible design artifacts) with an object-oriented zoomable user interface paradigm ZOIL [14]. By integrating informal sketching techniques and powerful design space visualizations based on a network of interconnected artifacts we will explore adequate support for collaborative creativity. Groupware features as well as techniques for communicating designs based on informal annotations will be used to facilitate co-located collaboration.

Case studies

Affinity Table - A Hybrid Surface for Supporting Affinity Diagramming

The most effective cooperation between designers frequently takes place in traditional co-located sessions in combination with structured methods and techniques that moderate the influences of social factors. The use of technology in such group sessions is often considered harmful since using desktop-based digital tools would isolate participants, leading to a breakdown of communication that is vital for a shared understanding in the group. Rigid interaction modalities may further impose limitations on crucial characteristics of design practice, like the workflow of design methods, coordination and communication as well as embodiment of thought. Hence, using affinity diagramming as an example, we investigate reality-based interfaces for supporting creative group work. Based on an observational study grounded in the realitybased interaction framework, we identified power vs. reality tradeoffs that can be addressed to find a close fit to embodied practice. Using this knowledge, we designed and implemented a digital workspace for supporting affinity diagramming. Its hybrid interaction techniques combine digital pen and paper with an interactive table and tangible tokens. An additional vertical display is used to support reflection-in-action and for enhancing discussion and coordination. A user study confirmed the applicability of our tradeoffs and the general acceptance of the tool design.

IdeaVis - A Hybrid Workspace and Interactive Visualization for Paper-based Collaborative Sketching Sessions

Despite recent developments in ubiquitous computing technology which blends in with the physical environment, many digital design tools replace existing physical practice by digital means, thereby changing not only the methods that can be applied in the group, but also traditional workflows that are used to cope with social inhibitors. Often, these tools do also not care for a session facilitator, thereby limiting the control such a trained professional can have over the group activity. In this case study, we designed IdeaVis, a novel approach for supporting co-located sketching sessions. Our system is based on digital pen and paper for augmenting the traditional paper-based workflows of sketching sessions. An additional focus and context visualization is used to support creative facilitators in exploring and examining the design activity, thereby increasing awareness over inhibitors that may impede the success of such sessions. The general applicability of our approach was confirmed in a user study with creative professionals. We demonstrate that live design activity visualizations can provide benefits for controlling typical inhibitors of creative group work without the need to change traditional workflows.

Bin and Bubble - Spatial Grouping Techniques on Interactive Surfaces

Arranging and organizing digital objects is a common task in various application domains. From sorting photos to managing files, many activities require users to move single or multiple items around for creating spatial aggregations or groups. The manual way of moving and dragging objects thereby serves as an implicit tool for filtering and synthesizing, thereby taking advantage of the human capabilities of recall, visual search and reasoning. In this case study we investigate the use of bimanual and multi-finger input for grouping items spatially on a tabletop interface. In a single user setup, we compared two typical interaction techniques supporting this task. We studied the grouping and regrouping performance in general and the use of bimanual and multi-finger input in particular. Our results show that the traditional container concept may not be an adequate fit for interactive tabletops. Rather, we demonstrate that informal and organic spatial metaphors are able to harness more benefits of multi-finger and bimanual interaction. We conclude with recommendations for the design of grouping techniques on interactive surfaces.

This research project is funded by the German Research Foundation DFG (Deutsche Forschungsgemeinschaft), grant number RE 1843/3-1

References

[1] Buxton, B.: Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann (2007)

[2] Löwgren, J.: Interaction Design. from Interaction-Design.org: http://www.interaction-design.org/encyclopedia/interaction_design.html (2008)

[3] Rogers, Y.: New Theoretical Approaches for HCI. Annual Review of Information, Science and Technology, Vol. 38, 1--43 (2004)

[4] Stolterman, E.: The Nature of Design Practice and Implications for Interaction Design Research. International Journal of Design Vol. 2, 1, 55--65 (2008)

[5] Winograd, T., Bennett, J., De Young, L., Hartfield, B.: Bringing Design to Software Addison Wesley Professional (1996)

[6] Löwgren, J., Stolterman E.: Thoughtful Interaction Design, MIT Press (2004)

[7] Moggridge, B: Designing Interactions, MIT Press (2006)

[8] Imaz, M., Benyon D.: Designing with Blends: Conceptual Foundations of Human Computer Interaction and Software Engineering, MIT Press (2007)

[9] Schön, D. A.: The Reflective Practitioner: How Professionals Think in Action. Basic Books (1983

[10] Cross, N.: Designerly Ways of Knowing. Board of International Research in Design. Birkhäuser (2007)

[11] Lawson, B.: How Designers Think, The Design Process Demystified. Architectural Press (2005

[12] Terry, M., Mynatt, E.D.: Recognizing Creative Needs in User Interface Design. Proceedings of the 4th Conference on Creativity and Cognition, 38--44 (2002)

[13] Memmel, T., Geyer, F., Rinn, J., Reiterer, H.: Tool-Support for Interdisciplinary and Collaborative User Interface Specification. Proceedings of the IADIS IHCI (2008)

[14] Jetter, H., König, W. A., Gerken, J., Reiterer, H.: ZOIL - A Cross-Platform User Interface Paradigm for Personal Information Management. In CHI 2008 Workshop - The Disappearing Desktop: Personal Information Management (2008)