VROOM: Virtual Robot Overlay for Online Meetings

In collaboration with Yaying (Sunny) Zhang, Priscilla N.Y. Wong, Sean Rintel, James Scott, Xu Cao, He Huang, Minnie Liu, Zhao Jun, Matthew Gan, and Leon Lu


This is the local collaboration space view showing a local user (left) and a remote user (right) using VROOM to collaborate on a whiteboard. Both users are standing next to a whiteboard. The remote user's avatar is gesturing at the whiteboard.


An overview of the VROOM system. Left: Local (AR) user who's wearing a Hololens sees a telepresence robot with overlaid full body life size avatar. The telepresence robot has a 360° camera mounted on top of it and fiducial markers at the front of it. Right: Remote (VR) user who's wearing a headset sees the remote space in a 360° view. The user is also holding controllers in both hands to drive the robot and control the avatar.


Telepresence robots allow users to freely explore a remote space and provide a physical embodiment in that space. However, they lack a compelling representation of the remote user in the local space. VROOM (Virtual Robot Overlay for Online Meetings) is a two-way system for exploring how to improve the social experience of robotic telepresence. For the local user, an augmented-reality (AR) interface shows a life-size avatar of the remote user overlaid on a telepresence robot. For the remote user, a head-mounted virtual-reality (VR) interface presents an immersive 360° view of the local space with mobile autonomy. The VR system tracks the remote user’s head pose and hand movements, which are applied to an avatar. This provides the remote user with an identifiable self-embodiment and allows the local user to see the remote user’s head direction and arm gestures.



Publications

Press Coverage