Fish Tank Virtual Reality

This work explores creating an interactive system combining 3D interactive surfaces with physics-based hand interaction methods. The system works in Fish Tank Virtual Reality(FTVR) environment, the user can manipulate the projected virtual objects when looking at their real hand. To achieve dexterous manipulation, we designed methods to visualize contact points between fingertips and rigid bodies without adding force feedback. Vibration is used to enhance collision perception. This system can help to break the border between the virtual and real world.

This work has been presented on Asia Haptics 2020. The paper is here.

The principle of this project is to track the user’s head using a tracker, accurately locate the position of the gaze in real-time, and generate 3D vision using a projector with 3D glasses. The advantage is that there is no need to wear a VR headset, and users can still see the real world. The disadvantage is that only the user can see the non-distorted visual effects.