Cross Device Interaction


Johannes Zagermann, Roman Rädle, Mario Schreiner, Hans-Christian Jetter, Harald Reiterer

What is Cross-Device Interaction?

In recent years, the number of mobile devices has increased significantly. Along with this explosive growth, advancements in computing technology and the miniaturization makes them small, mobile, personal (super-) computers that we can carry with us. Despite their power, mobility, and density, these devices often idle away in our pockets wasting valuable computing resources.

We envision a future where mobile devices can contribute their (interaction) resources to a community of devices in their proximity. At any time, users can dynamically compose a seamless multi-device user interface (UI) and reconfigure this UI according to their current needs and the task at hand. In this vision, users also can seamlessly add or remove devices from the community without explicit setup or pairing. Instead, this happens implicitly as a byproduct of natural use in space, for example, by bringing multiple devices to the same room, placing them side-by-side on a table or desk, and moving them around as needed. Ideally, users will experience these co-located cooperating devices and reconfigurable displays as one seamless and natural user interface for ad-hoc co-located collaboration.


Cross-Surface Workshop at ISS 2016

Harald Reiterer co-organizes the third edition of the workshop "Interacting with Multi-Device Ecologies in the Wild". The focus of the third edition is Challenges and Opportunities of Spatial and Proxemic Interaction. The workshop will be held in Niagara Falls, Canada on November 6th 2016. More Information

Cross-Surface Workshop at CHI 2016

Harald Reiterer and Mario Schreiner co-organize the second edition of the workshop "Interacting with Multi-Device Ecologies in the Wild". The focus of the second edition is Challenges and Opportunities for ‘Bring Your Own Device’ in the Wild. The workshop will be held in San Jose, California from May 7 to May 12, and deals with challenges and opportunities of interaction across multiple surfaces in the wild. The second edition will further have a special focus on interactions that support Bring-Your-Own-Device Scenarios.

Read the associated blog entry

Cross-Surface Workshop at ITS 2015

Harald Reiterer co-organize the workshop "Interacting with Multi-Device Ecologies in the Wild", which is held at ITS 2015 on the island of Madeira in November 2015. The workshop deals with the challenges and opportunities of interaction across multiple surfaces in the wild.

Read the associated blog entry 




As a second incarnation of this vision, we present Connichiwa, a framework for creating cross device applications without the need for external sensing hardware or server infrastructure. Connichiwa abstracts away complex multi-device setup and pairing, works across different hardware platforms and architectures, and integrates full-stack web-technologies for cross device applications and communication. Beyond that, it runs a local web server on one of the participating devices, which makes external network infrastructure or an internet connection obsolete. As a framework designed to build truly self-sustaining cross-device applications, Connichiwa is compelling for many improvised activities (e.g., ad hoc meetings, outdoor scenarios, crisis management).

More details on: Connichiwa Website

Source Code: Connichiwa GitHub


As a first incarnation of this vision, we present HuddleLamp, a desk lamp with an integrated low-cost RGB-D camera that detects and identifies mobile displays (e.g., smartphones or tablets) on tables and tracks their positions and orientations with sub-centimeter precision. Users can add or remove off-the-shelf, web-enabled devices in an ad-hoc fashion without prior installation of custom hardware (e.g., radio modules, IR markers) or software. Because of HuddleLamp's web-based pairing, adding a new device is simply done by opening a URL on the device and putting it on the table so that it is visible to the camera.

HuddleLamp also tracks users' hands to enable interactions between and above devices, such as 'pick, drag, and drop.' This enables a new kind of computer-supported around-the-table collaboration. Users can sit around ordinary tables that can remain cluttered with non-digital objects (e.g., printouts, maps, notebooks) while their digital collaborations happen using spatially-aware mobile screens. They even blend into existing spatial and social practices and truly weaving themselves into the fabric of everyday life.

More details on: HuddleLamp Website

Source Code: HuddleLamp GitHub

In joint collaboration with:

Publication list

  • Article
  • Book
  • Dissertation
  • Thesis
  • Proceedings
  • Other
"There was an error during trying to get the publication list. Please try again or inform the admin, if it fails again."