Embodied Cross-Reality Interaction

through Transitional Interfaces

Our goal is to conceive, develop and evaluate novel concepts for human-computer interaction that dissolve the user interface as a barrier between real and virtual, and instead allow the user to seamlessly transition between different realities. 
Traditional user-centered approaches aim for “natural interaction” by mimicking our way of interacting and communicating with the non-digital world. Natural User Interfaces or Reality-based User Interfaces are typical examples of this approach. We take these forms of interaction as a starting point (see our Blended Interaction framework for a more detailed explanation), as consideration of the user’s real-world experience and behavior lowers the barrier for interaction, enabling users to walk up and use even advanced technologies like mixed-reality or extend-reality devices. 

However, we then go beyond traditional approaches by emphasizing the User Representation, which serves as the user’s proxy in the virtual world, potentially reflecting or redefining their bodily characteristics and thereby specifying their (new) abilities. 

This transformation of the user’s physical body into an (embodied) virtual User Representation enables more advanced forms of interaction and communication, such as virtual super powers. Hence, using mixed-reality or extend-reality devices, the user can engage in cross-reality interaction gaining benefits from virtually extended capabilities. Such extensions of our bodily capabilities depend on cognitive and sensorimotor aspects, which must be explicitly considered in user interface design. 

We call this extended form of interaction “Embodied Cross-Reality Interaction” and the supporting user interfaces “Transitional Interfaces.” Such interfaces support crossing between different realities: from being in the physical world and using our existing cognitive and bodily skills, to gradually moving into a virtual reality with more advanced or “magical” skills, and back again.