Project 2

Mobile, Autostereoscopic 3D Display and Tele-Presence Platform

The vision of Project 2 is to design and develop technology which can be used to build a roving display platform showing a 3D representation of a single person, which is located remotely and is user controllable. The display is semitransparent and will bring the illusion as if the other person is present in a room or a laboratory. The display hardware and associating software components are the two core aspects of this project. In order to achieve this vision, the project is decomposed into two major subprojects, where one of them focuses on the hardware and display aspects, and the other one focuses on software and algorithmic aspects. Both of these subprojects are outlined in the following.

1. 3D Displays

Here, we design a transparent life size 3D display with an attached acquisition system. The figure illustrates an evolution of 3D displays which have been designed and developed in the context of the BeingThere Center.

Protoype 1 is a parallax barrier display which is realized by stacking up LCD displays. While the 3D effect on the display can be perceived without special glasses, it is fully opaque. Prototype 2 is our first transparent 3D display. It is based on a projective foil and requires active shutter glasses. While a vivid 3D effect is achieved with the active shutter glasses, they are impractical due to their weight and size. With Prototype 3, we improve this transparent active stereoscopic display by replacing the active shutter glasses with lightweight passive glasses. In this case, the display, a glass panel, embeds a special anisotropic film. Two projectors behind the display project light with different polarization onto the transparent display. Due to the anisotropic properties of the setup, the displayed image appears vivid even in relatively bright light conditions. In addition, immersion is enhanced by adding motion parallax to the binocular cues: Eye-tracking is used to adapt rendered content to the viewer position. Prototype 4 is a lifesize version of Prototype 3. A bidirectional setup consisting of two life-size displays is installed at ETH and NTU. As a next step, we are developing glasses-free 3D display systems.

2. Gaze Awareness

Effective communication using current video conferencing systems is severely hindered by the lack of eye contact caused by the disparity between the locations of the subject and the camera. While this problem has been partially solved for high-end expensive video conferencing systems, it has not been convincingly solved for consumer-level setups. A gaze-corrected 3D model of the scene is rendered and, with the aid of a face tracker, the gaze-corrected facial portion is transferred in a seamless manner onto the original image. Two systems have been developed and published: 1) Gaze correction based on a Kinect and webcam, 2) Gaze correction solely based on a webcam. The proposed methods preserve both the integrity and expressiveness of the face as well as the fidelity of the scene as a whole, producing nearly artifact-free imagery. The method is suitable for a variety of scenarios such as mainstream home video conferencing, as it uses inexpensive consumer hardware, achieves real-time performance and requires only a simple setup.

Awards and Recognition in Media

The initial work on gaze correction has been selected as one of the Top 5 most promising ETH inventions of the year in March 2013. This technology has also received major attention in media. It has been covered by Gizmodo, Discovery Channel, Stuff, IBT, to name a few. The display technology in association to the BeingThere Centre has also received attention from printed media such as NZZ or TagesAnzeiger, belonging to the largest Swiss print publications.