permaedia

Members

Hans-Christian Jetter, Harald Reiterer, Michael Zöllner, Jens Gerken, Werner König, Mathias Heilig, Mischa Demarmels, Stephan Huber

Description

PERMAEDIA in a Nutshell

• Post-WIMP multi-user & multi-surface interactive spaces for personal and group information management.
   
• Novel post-WIMP user interfaces for a device-independent and consistent access based on the ZOIL user interface design paradigm and software framework.
   
• Novel applications of digital media in the fields of visual information management, information seeking, and e-Science.

An essential part of permedia is the development of the underlying cross-platform user interface concept spanning a multitude of domains (e.g. visual information seeking, digital libraries, e-science) and devices (e.g. multitouch tabletops, tablet PCs or tangible user interfaces). This ZOIL (zoomable object-oriented information landscape) user interface paradigm unifies zoomable user interfaces (ZUIs) with information visualization and direct-manipulation techniques to replace traditional menu-, window and hypertext-driven user interfaces with novel usable and scalable post-WIMP approaches.

Mission Statement

Current designs of digital media are too demanding for many users due to a lack of interoperability, unification and consistency ("information fragmentation" and "application fragmentation"). They often exclude individuals or entire social groups from benefiting from digital media instead of having an inclusive effect and empowering everyone. While the necessary technological infrastructure for the digital media of the next decade is often already widely available, users find the multitude of different user interfaces and applications with obsolete interaction techniques and with the lack of interoperability too hard to learn and to use. Many of the products for the end user are failing because today's user interfaces are still based on yesterday's application scenarios, technological restrictions and design conventions unconsciously inherited from the World Wide Web and the mobile phone of the 1990s, the personal computer of the 1980s and in parts even from the age of "mainframe computers" of the 1960s [4].

The reason for this is that many current products only partially reflect the Weiserian vision of ubiquitous computing and the broad scientific knowledge about the pragmatic, emotional, social or cognitive aspects of human-computer interaction. They have not been designed with the tools and methodologies of user-centered design and usability engineering. Often they are based on false assumptions about the users' work practice and are hard to learn and to use. Essential user tasks are impaired by the secondary task of handling the technology itself. Unsurprisingly they therefore fail to contribute to a "universal usability" [5] of digital media, appear unattractive and evoke negative feelings. These products are not considered as "pleasurable products" [6­] orchestrating a positive user experience and eventually fail on the market.

To unleash the true power of the digital media of the coming decade and its underlying infrastructure, novel applications with novel approaches in interaction design, user interface design and information visualization are necessary. Rethinking digital media from the user's point of view means more than optimizing or adapting existing user interfaces. It means rethinking the ways that information can be stored, accessed, visualized, managed and personalized by the user. It means rethinking whole information and communication services and products from a user's point of view. It especially means to let the user participate in all phases of the product's lifecycle to ensure that the user's needs are fully satisfied and the resulting digital media are usable and sufficiently adaptable.

Project Scope

The permaedia project's premise is that user-centered design (UCD) is the key to designing the successful nomadic digital media for the coming decade of post-WIMP ubiquitous computing (also see [7]). User-centered research of future applications hereby serves as a source of innovation [8] and ensures a positive user experience [9] [10] by integrating the user into the development and the design process. The project will especially address following key concepts:

The ZOIL user interface paradigm

The "ZOIL" UI paradigm (Zoomable Object-oriented Information Landscape) is the underlying concept of all user interfaces to be developed in course of the project and is aimed at providing a consistent and "intuitive" interaction and navigation paradigm [4]. ZOIL is based on the outcome of previous research projects in our research group (see section "Related Projects") and is aimed at achieving what the desktop metaphor was originally meant for (before the arrival of the Web as competing interaction paradigm): "intuitive" consistent interaction with digital media by directly interacting with visual objects or representations without regard to different applications, devices or data types. A ZOIL interface integrates all kinds of local or web-based information spaces and applications into one coherent conceptual model called the information landscape, which can be visually explored by direct-manipulation using multitouch or tangible interaction. To learn more about the ZOIL user interface paradigm have a look at the corresponding section below.
 

Natural interaction with multiple devices

Providing a consistent user interface concept for digital media that can span various devices is a demanding task for the user interface and interaction designer. By making use of platform and device independent interaction libraries (see related project SQUIDY) and new modeling techniques for content and functionality [4] permedia will lay the foundation for cross-platform designs and design processes.

Regarding information presentation on different devices the ZOIL user interface paradigm has a key advantage: Due to the nature of a zoomable user interface, information presentation in ZUIs scales to different display sizes and therefore is applicable on very different hardware platforms varying from smart mobile devices to living room electronics (e.g. iTV, video game consoles), tabletop computers (e.g. Microsoft Surface) or large high resolution displays.

Project Outcome

As result of the permaedia project we will present interactive prototypes based on the ZOIL paradigm for stationary and mobile use of future digital media. These prototypes will demonstrate the results of the user-centered design process and will be evaluated and tested in usability studies in the usability lab and in situ. 

Some ZOIL-based designs (e.g. Facet-Streams, DeskPiles) are currently designed and evaluated in cooperation projects with the Integrated Systems Group of Microsoft Research Cambridge, Microsoft Surface, and the NanoPhotonics Centre of the University of Cambridge. ZOIL is furthermore the conceptual and technological basis for other projects in the HCI group, e.g. Blended Library, Blended Museum, Blended Interaction Design, or MedioVis 2.0.

The user interface prototypes are based on a reusable ZOIL software framework developed in C# using Microsoft's .NET API and WPF.

Related Projects at the University of Konstanz

PERMAEDIA explores innovative solutions for multimedia document storage and retrieval in cooperation with the AG DBIS (Prof. Scholl, University of Konstanz) and the BaseX project.

ZOIL (Zoomable Object-Oriented Information Landscape):
A User Interface Design Paradigm and Software Framework for Collaborative Interactive Spaces

Key qualities of ZOIL are unification, visualization, direct-manipulation and platform-independence, meaning that a ZOIL interface integrates all kinds of local or web-based information spaces and applications into one coherent conceptual model called the “information landscape”, which can be visually explored by direct-manipulation of the information display on various devices.

Thereby the information landscape can be understood as a kind of virtual whiteboard of infinite size and resolution in which all data and functionality (no matter if local or web-based) is presented using visual formalisms and metaphors (e.g. visual networks, geographic maps, satellite images, time lines…). The user can access individual functions or content by visually zooming into the regions of interest and directly acting "on-the-spot" (e.g. adding annotations or multimedia content directly to a location on a geographic map; accessing documents and associated functionality by zooming into visual representations of document repositories, projects or business processes; semantically integrating new information objects into visualizations of networks with "drag and drop" ...).

The more the user zooms into the content, the more details and functionality is revealed by a “semantic zooming” approach (e.g. an iconic representation of a spreadsheet is zoomed continuously to a screen-filling and editable spreadsheet application). The whole navigation within the information and functionality is thereby reduced to a zooming and panning operation, which exploits the human abilities of visual-spatial orientation and remembering visual “landmarks” [15]. This kind of navigation is based on the work about Zoomable User Interfaces (ZUIs) of Perlin & Fox and Ben Bederson in the 1990s [16] [17] and Jef Raskin’s vision of the “humane interface” from 2001 [18]. Other recent R&D projects dealing with ZUIs are for example the Piccolo project [18] or Microsoft's DeepZoom technology [20].

ZOIL furthermore replaces the page-wise and sequential hyperlink interaction of Web interfaces and the abstract symbolic hierarchies and interdependencies of file systems and applications with an object-oriented [21] and direct-manipulative [22] visual representation of information and functionality:

To this day personal information management often means managing personal or public information in inadequate and inconsistent data structures that are spread over a multitude of websites, applications or devices (e.g. inconsistent file system hierarchies of application-specific data files spread over desktops, laptops and smart phones instead of a shared semantic network of interoperable information objects). All information in ZOIL will be unified in a centralized information repository that stores all information objects and their specific logical or semantic relations according to the user's needs. As proposed by semantic web or semantic desktop approaches [23] [24] this personal information repository, which can be enriched by external web services or "semantic" data sources, can be freely accessed and customized according to the individual user goals and tasks. New kinds of information objects and the functionality to work with them can be integrated "on demand" by downloading new class definitions for information objects as "plug-ins". Instead of installing monolithic applications or integrating web applications with different interaction models (which are then only loosely coupled through the file system or the operating system's clipboard) the functionality of a ZOIL user interface is easily extensible with modular commands or services that are integrated consistently into the user interface. In the original sense of an object-oriented user interface (OOUI) the interaction is simplified by preserving consistency in interaction by using logical domain object models and object hierarchies [21] [25]. These allow the user to build a consistent mental model of the system, to reason about the system's behavior and to deduce necessary actions from previous experiences (e.g. user interface consistency through "polymorphism" of information objects).

Interactive information visualization will play an important role within ZOIL. Instead of leaving the user with the either list-, table- or icon-based representations known from current desktop or mobile systems, ZOIL will also offer visualization techniques ranging from scatter plots to HyperGrids [26] which will be powerful tools for understanding, searching, filtering and querying arbitrary regions of the information landscape - especially in combination with direct-manipulative filters, magic lenses [27] or dynamic query approaches [28]. The nested information visualization in a zoomable user interface empowers the user to combine different visualizations for formulating complex queries in a direct-manipulative and visual manner [4] [29].

References

[1] UNFPA: The State of World Population 2007, United Nations Population Fund, 2007. http://www.unfpa.org/swp/2007/presskit/pdf/sowp2007_eng.pdf (last accessed on 25 Jul 2007).

[2] UITP: Better Mobility in Urban Areas, International Association of Public Transport, 2001. 

[3] Castells, M.; Fernández-Ardèvolm, M.; Qiu, J. L.; Sey, A.: Mobile Communication and Society - A Global Perspective, MIT Press, London, 2007.

[4] Jetter, Hans-Christian: Informationsarchitektur und Informationsvisualisierung für die Post-WIMP Ära, Master Thesis, University of Konstanz, 2007. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3209/ (last accessed on 25 Jul 2007).

[5] Shneiderman, Ben: Leonardo's Laptop: Human Needs and the New Computing Technologies, MIT Press, Cambridge, 2002.

[6] Jordan, Patrick A. W.: Designing Pleasurable Products: An Introduction to the New Human Factors, Taylor & Francis, London, 2000.

[7] Nomadic Media (ITEA if02019): User-Centred Design - Guidelines for Methods and Tools, 2005. http://www.vtt.fi/inf/julkaisut/muut/2005/UCD_Guidelines.pdf (last accessed on 15 Aug 2007)

[8] D'Hooge, H.: User-Centered Research Fuels Technology Innovation at Intel. Technology@Intel Magazine, April 2005. http://www.intel.com/technology/magazine/research/user-centered-innovation-0405.pdf (last accessed on 15 Aug 2007).

[9] Jetter, H.-C.; Gerken, J.: A Simplified Model of User Experience for Practical Application. Proceedings of the 2nd COST294-MAUSE International Open Workshop: User eXperience - Towards a unified view, Oslo 2006. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3151 (last accessed on 15 Aug 2007).

[10] Jetter, H.-C.: Die MCI im Wandel: User Experience als zentrale Herausforderung? Mensch und Computer 2006: Mensch und Computer im StrukturWandel, Oldenbourg, 2006. https://kops.uni-konstanz.de/handle/123456789/6042 (last accessed on 17 Jan 2022).

[11] Büring, T: Zoomable User Interfaces on Small Screens. Dissertation, University of Konstanz, 2007. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3208 (last accessed on 15 Aug 2007).

[12] Han, J.: Multi-Touch Interaction Research, 2006. http://cs.nyu.edu/~jhan/ftirtouch/ (last accessed on 15 Aug 2007).

[13] König, W. A.; Bieg, H.-J.; Reiterer, H.: Laserpointer-Interaktion für große, hochauflösende Displays. Mensch und Computer 2007: Interaktion im Plural, Oldenbourg, 2007. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3138 (last accessed on 15 Aug 2007).

[14] König, W. A.; Bieg, H.-J.; Schmidt, T.: Position-Independent Interaction for Large High-Resolution Displays. IHCI'07: Proceedings of IADIS International Conference on Interfaces and Human Computer Interaction 2007, IADIS Press, Lisbon, Portugal, 2007. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3145 (last accessed on 15 Aug 2007).

[15] Gerken, J.: Orientierung und Navigation in zoombaren Benutzerschnittstellen unter besonderer Berücksichtigung kognitions-psychologischer Erkenntnisse. Master-Thesis, Universität Konstanz, 2006. http://www.ub.uni-konstanz.de/kops/volltexte/2007/3210 (last accessed on 15 Aug 2007).

[16] Perlin, K.; Fox, D.: Pad: an alternative approach to the computer interface. SIGGRAPH '93: Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM Press, New York, 1993.

[17] Bederson, B.; Hollan, J. D.: Pad++: a zooming graphical interface for exploring alternate interface physics. UIST '94: Proceedings of the 7th annual ACM symposium on User interface software and technology. ACM Press, New York, 1994.

[18] Piccolo Home Page. http://www.piccolo2d.org/ (last accessed on 04 Feb 2011).

[19] Raskin, Jef.: The Humane Interface: New Directions for Designing Interactive Systems, Addison-Wesley, Reading, 2000.

[20] Microsoft Deep Zoom. http://www.microsoft.com/silverlight/deep-zoom/ (last accessed on 04 Feb 2011).

[21] Mandel, T.: The Gui-Ooui War: Windows Vs. Os/2 : The Designer's Guide to Human-Computer Interfaces. Van Nostrand Reinhold, New York, 1993.

[22] Shneiderman, B.: Direct manipulation for comprehensible, predictable and controllable user interfaces. IUI '97: Proceedings of the 2nd international conference on Intelligent user interfaces. ACM Press, New Yoirk, 1997.

[23] Karger, D. R.: Haystack : Per-User Information Environments Based on Semistructured Data. In: Kaptelini, V.; Czerwinski, M.: Beyond the Desktop Metaphor : Designing Integrated Digital Work Environments. MIT Press, 2007.

[24] DeepaMehta Homepage. http://www.deepamehta.de/ (last accessed on 15 Aug 2007).

[25] Collins, D.: Designing Object-Oriented User Interfaces. Benjamin-Cummings Publishing Company, 1995.

[26] Jetter, H.-C.; Gerken, J.; König, W.; Grün, C.; Reiterer, H.: HyperGrid - Accessing Complex Information Spaces. People and Computers XIX - The Bigger Picture, Proceedings of HCI 2005, Springer Verlag, 2005.

[27] Bier, E. A.; Stone, M. C.; Pier, K.; Buxton,W.; DeRose,Tony D.: Toolglass and magic lenses: the see-through interface. SIGGRAPH '93: Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM Press, New York, 1993.

[28] Ahlberg, C.; Williamson, C.; Shneiderman, B.: Dynamic queries for information exploration: an implementation and evaluation. CHI '92: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press, New York, 1992.

[29] König, W. A.: Referenzmodell und Machbarkeitsstudie für ein neues Zoomable Interface Paradigma. Master-Thesis, Universität Konstanz, 2006. Master Thesis, Univ. of Konstanz, 2006.

Surface Computing with ZOIL and Squidy