Internship CEA/LISN: AR & virtual display walls

The aim of the project is to build virtual display walls with multiple resolutions and interactions using Augmented Reality (AR).

Supervisors

Martial Mancip, Huyen Nguyen, Jeanne Vézien

Contact : martial.mancip@cnrs.fr

Subject

This project will be based on the high-end multiple resolution data visualisation system at Maison de la Simulation (https://mdls.fr/). Specifically, we have been developing tools for the data analysis of numerical simulations either in-situ while running on a super-computer or with data written on disks.

We work on the visualisation of ensemble of simulations through the TiledViz infrastructure to build efficient analysis and visualisations of the results of massively-parallel simulations. We also use artificial intelligence and machine learning approaches to analyse complex data as produced in medicine/biology applications.

The focus of this project is to allow remote users to access this high-end infrastructure TiledViz using optical-see-through AR head-mounted display (HMD) to visualise and explore complex datasets. A typical scenario would be for this system to be used in collaborative meetings of people of various expertise to analyse scientific data.

There are several aspects to be considered in this project:

1. Transform 3D interactions with hand gestures, voice commands, etc. captured from the HMD device to 2D interactions that would be processed by TiledViz;
2. Capture the data flow from TiledViz infrastructure (located at Maison de la Simulation) to readapt it as a virtual wall to the current context of interaction of the remote users;
3. Adapt the rendering resolution on the HMD based on the distance of the user to the virtual wall;
4. Evaluate the performance and user experience of using of virtual wall via AR headsets and the real TiledViz system.

We will use LibVNC for Unity to visualise remotely desktop, laptop or virtual machine (from all OS) inside the AR interface. The interface for a single user with multiple displays will be developed before considering its extension to multiple users’ usage.

This internship is funded by CEA and will be conducted mostly at Maison de la Simulation and partly at LISN (Université Paris-Saclay).

We aim to develop this topic to a Ph.D. project on the interaction of multiple users during distance collaborative sessions.