Postdoctoral position on side effect automatic detection in VR (Heudiasyc)

Objectives

Virtual Reality (VR) offers capabilities to support and facilitate forensic activities. The European project, INFINITY (https://h2020-infinity.eu/), coordinated by Airbus Defence and Space, will innovate and deliver a pioneering an immersive virtual environment. It will provide law enforcement agencies (LEAs) with an analytical platform for cutting-edge intelligence extraction, analysis, visualization and collaboration across teams and countries to address the challenge brought about by the exponential increase in investigative data in before, during and after major criminal and terrorist attacks. Synthesizing the latest innovations in virtual and augmented reality, artificial intelligence and machine learning with big data and visual analytics, INFINITY will deliver an integrated solution that aims to enhance data-driven investigations.

In this context, we are looking for a talented and motivated post-doctoral researcher to process physiological data, create a model for side effects detection and propose adaptive feedback to support and facilitate forensic activities and to optimize virtual environments to maximize users’ performance, including removing potential side effects.

VR equipment may induce side effects (visual fatigue and cybersickness) because of locomotion and stereoscopy (Rebenitsch & Owen, 2021; Stanney et al., 2020). Working in VR, depending on the tasks, may reduce working memory resources available (Van Acker et al., 2018). Using a new technology such as VR, talking in public and being exposed to distressing material could provoke stress (Fink, 2016). In order to warrantying user’s well-being and task performance, we aim at detecting cybersickness, mental workload and stress in real time. Based on physiological (eye tracking, ECG, EDA) (Arza et al., 2019; Dennison et al., 2016; Vanneste et al., 2020), biological, behavioral, subjective and profile data, we aim at providing adaptive feedback to minimize the side effects of prolonged immersion for working in VR. This could be possible with machine learning techniques (Artime Ríos et al., 2019; Garcia-Agundez et al., 2019; Islam et al., 2020; Lohr et al., 2016; Porcino et al., 2020; Smets et al., 2018).

Contact & Supervision

Domitile Lourdeaux, Associate Professor, HDR, mailto:domitile.lourdeaux@hds.utc.fr

http://www.hds.utc.fr/~dlourdea

Research center

The Heudiasyc Laboratory was created in 1981. From the start Heudiasyc has been closely allied to CNRS and is attached to CNRS's INS2I (Information Sciences) section. Heudiasyc's research is in the field of information and digital technology (computer science, automatic control, robotics, and artificial intelligence).

The aim is to develop ways of representing, analyzing and controlling systems that are subject to criteria and constraints, whether these be expressed in scientific, technological, economic, or social terms. Research is organized around three teams:

- CID: Knowledge, Uncertainty, Data

- SCOP: Dependability, Communication, Optimization

- SyRI: Robotic systems in interaction

Two teams are involved in the INFINITY project: CID and SyRI. These teams aim to design systems able to adapt automatically and dynamically both to humans that use them and to the environment they operate in. The methods that we develop are tested in collaborative environments for capitalizing knowledge, recommending tourist and cultural circuits, and virtual environments for human learning or decision making.

CID develops HUMANS (HUman Models based Artificial eNvironments Software platform). HUMANS aims to create adaptive virtual environments which consider diverse human behaviours or rely on them, looking especially in collaborative work for crisis situations. HUMANS allows the orchestration of virtual environments to generate dynamically adaptive and relevant situations according to user’s profile.

SyRI develops the TRANSLIFE platform, an immersive virtual reality room used for research on informed interaction. It is a four-faced CAVE™ (Cave Automated Virtual Environment), which can also be transformed into a more open three-faced installation.

Research focus

In the INFINITY project, our program focuses on a monitoring toolkit and adaptive feedbacks. We aim at detecting side effects and then modelling adaptive feedback. The monitoring toolkit will allow to collect physiological data live to detect side effects. The adaptive feedbacks will decrease cybersickness, visual fatigue, stress and mental workload but also increase presence, flow, localization, agency, trust, concentration, and usability based on initial state detection. HUMANS and TRANSLIFE will be used to adapt feedbacks in Virtual Reality according to a fuller user profile (e.g. mental workload, visual fatigue, stress, stress or motion traits) and team configuration. The feedbacks will be related to the interaction (metaphors) and to the adaptive content (annotations, visualization of the data according to the user cognitive workload profile, to the expertise and to the needs). Our role would be to propose models and AI engine to improve the use of the VR in the INFINITY context (monitoring and feedback generation).

Your tasks

- To conduct pluridisciplinary research with European high-level research Labs (CENTRIC in UK, Deutsches Forschungszentrum fur Kunstliche Intelligenz GMBH in Germany, CERTH in Greece, etc.)

- To lead research on health and well-being of prolonged use of AR/VR and deliver advices, tips and tricks on how to maximize the opportunities the platform affords.

- To propose a dynamic user profile based on eye tracking, ECG, EDA, behavioral and subjective data.

- To perform signal processing (filtering, normalization…) of eye tracking, ECG, EDA.

- To propose a model for detecting cybersickness, mental workload and stress in VR with machine learning technics.

- To propose a model of AI engine to generate adaptive feedback in VR.

- To publish in top conferences and journals dedicated to VR.

- To participate to the administrative and reporting tasks inherent in a European project.

Skills

- PhD in Neuro-physiology, Artificial Intelligence or similar relevant fields;

- Motivation working in multidisciplinary research at the frontiers between Computer Science, Neuroscience and Psychology;

- High level in English (written and spoken);

- Very good communication skills, commitment, independent working style as well as initiative and team spirit;

- Experience with physiological data processing.

- Experience with Machine Learning

Experience in various areas will be appreciated:

- Cognitive science

- Biofeedback

- Virtual reality

Postdoctoral duration

12 months

Start date

The 1st of October 2021

Location

Alliance Sorbonne Université

Université de technologie de Compiègne

CNRS, Heudiasyc UMR 7253

Compiègne, France

References

• Artime Ríos, E. M., Sánchez Lasheras, F., Suárez Sánchez, A., Iglesias-Rodríguez, F. J., & Seguí Crespo, M. del M. (2019). Prediction of Computer Vision Syndrome in Health Personnel by Means of Genetic Algorithms and Binary Regression Trees. Sensors, 19(12), 2800. https://doi.org/10.3390/s19122800

• Arza, A., Garzón-Rey, J. M., Lázaro, J., Gil, E., Lopez-Anton, R., de la Camara, C., Laguna, P., Bailon, R., & Aguiló, J. (2019). Measuring acute stress response through physiological signals : Towards a quantitative assessment of stress. Medical & Biological Engineering & Computing, 57(1), 271‑287. https://doi.org/10.1007/s11517-018-1879-z

• Dennison, M. S., Wisti, A. Z., & D’Zmura, M. (2016). Use of physiological signals to predict cybersickness. Displays, 44, 42‑52. https://doi.org/10.1016/j.displa.2016.07.002

Fink, G. (2016). Chapter 1 - Stress, Definitions, Mechanisms, and Effects Outlined : Lessons from Anxiety. In G. Fink (Éd.), Stress : Concepts, Cognition, Emotion, and Behavior (p. 3‑11). Academic Press. https://doi.org/10.1016/B978-0-12-800951-2.00001-7

• Garcia-Agundez, A., Reuter, C., Becker, H., Konrad, R., Caserman, P., Miede, A., & Göbel, S. (2019). Development of a Classifier to Determine Factors Causing Cybersickness in Virtual Reality Environments. Games for Health Journal, 8(6), 439‑444. https://doi.org/10.1089/g4h.2019.0045

• Islam, R., Lee, Y., Jaloli, M., Muhammad, I., Zhu, D., & Quarles, J. (2020). Automatic Detection of Cybersickness from Physiological Signal in a Virtual Roller Coaster Simulation. 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 648‑649. https://doi.org/10.1109/VRW50115.2020.00175

• Lohr, D. J., Abdulin, E., & Komogortsev, O. V. (2016). Detecting the onset of eye fatigue in a live framework. Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, 315‑316. https://doi.org/10.1145/2857491.2884058

• Porcino, T., Rodrigues, E. O., Silva, A., Clua, E., & Trevisan, D. (2020). Using the gameplay and user data to predict and identify causes of cybersickness manifestation in virtual reality games. 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), 1‑8. https://doi.org/10.1109/SeGAH49190.2020.9201649

• Rebenitsch, L., & Owen, C. (2021). Estimating cybersickness from virtual reality applications. Virtual Reality, 25(1), 165‑174. https://doi.org/10.1007/s10055-020-00446-6

• Smets, E., Rios Velazquez, E., Schiavone, G., Chakroun, I., D’Hondt, E., De Raedt, W., Cornelis, J., Janssens, O., Van Hoecke, S., Claes, S., Van Diest, I., & Van Hoof, C. (2018). Large-scale wearable data reveal digital phenotypes for daily-life stress detection. Npj Digital Medicine, 1(1), 1‑10. https://doi.org/10.1038/s41746-018-0074-9

• Stanney, K., Lawson, B. D., Rokers, B., Dennison, M., Fidopiastis, C., Stoffregen, T., Weech, S., & Fulvio, J. M. (2020). Identifying Causes of and Solutions for Cybersickness in Immersive Technology : Reformulation of a Research and Development Agenda. International Journal of Human–Computer Interaction, 36(19), 1783‑1803. https://doi.org/10.1080/10447318.2020.1828535

• Van Acker, B. B., Parmentier, D. D., Vlerick, P., & Saldien, J. (2018). Understanding mental workload : From a clarifying concept analysis toward an implementable framework. Cognition, Technology & Work, 20(3), 351‑365. https://doi.org/10.1007/s10111-018-0481-3

• Vanneste, P., Raes, A., Morton, J., Bombeke, K., Van Acker, B. B., Larmuseau, C., Depaepe, F., & Van den Noortgate, W. (2020). Towards measuring cognitive load through multimodal physiological data. Cognition, Technology & Work. https://doi.org/10.1007/s10111-020- 00641-0