Internship on Intelligent Tutoring System in Virtual Reality

Keywords: Virtual Reality Training System (VRTS), Data collection and analysis, Machine Learning, Deep Learning.

Context:

In the context of Industry 4.0 and digital factory, digital twin and virtual reality represent technologies that can deal with several industrial issues like design, simulation and optimisation of industrial systems. They are helpful to perform operator’s training of industrial situation and evaluate them (Fratczak, Goh, Kinnell, Soltoggio, & Justham, 2019) (Havard, Jeanne, Lacomblez, & Baudry, 2019) (Matsas & Vosniakos, 2017).

VR is thus useful to make people learn. However, the training scenario cannot be dynamically adapted to the learner profile or status. For example, an Intelligent Tutoring System (ITS) should be able to detect when a learner needs help, during the scenario, if he does not fully understand what he/she needs to do; or the ITS can detect that the learner feels easy enough to add a new unplanned event in the scenario in order to test the learner reaction to a more stressing situation. That is why, in this internship at LINEACT CESI Rouen, we need to analyse VR simulation data and physiological data in order to be able to adapt the VR scenario to the learner profile and status.

LINEACT CESI has developed a flexible manufacturing system (FMS) involving several robots with different capabilities in a shop floor layout context. This use case is composed of an automatic production system and a set of manual workstations where the operator can be assisted by cobotic arms for assembly tasks. A digital twin associated to Human-machine interface (HMI) based on virtual or augmented reality is also present and allows to perform simulation in Virtual Reality Training Systems (VRTS). Particularly, Virtual Reality simulates a case study where an operator is working on a manual workstation to assemble a subassembly of a children's bike. Moreover, LINEACT owns physiological sensors (EEG, ECG, GSR, Head position, Gaze tracking) associated to the iMotions platform to record data, in a synchronous way. This use case and this hardware will be used during the internship in order to evaluate the learner profile and status thanks to physiological data.

Missions:

• Participation to the literature review on classification and prediction model for Intelligent Tutoring System in Virtual Reality Training System (VRTS) to adapt to the learner profile and status.
• Setup the data collection and storage components and participation to the design of the experimentation for acquiring data about the learner profile and status. With the platform available at LINEACT CESI Rouen, the following data are available: ECG, GSR, EEG, Head position, Gaze tracking, interactions and assembly simulation in VR, …
• Spatio-temporal data analysis from the experimentation.
• Proposition of a classification model based on Machine Learning or Deep Learning algorithm for detecting the learner profile and status.
• Writing a paper about the results obtained.

Skills:

• A master level in computer sciences,

• Skills in data sciences, database system, machine learning or deep learning,

• Not mandatory, knowledge in Unity - C# and Virtual Reality,

• Human skills
  • Good interpersonnal skills
  • Writing ability

Contacts :

• Vincent HAVARD (vhavard@cesi.fr), lecturer, LINEACT CESI Rouen.

• David BAUDRY (dbaudry@cesi.fr), Head of Engineering and digital tools research team, LINEACT CESI Rouen.

How to apply :

Submit you application to Vincent Havard vhavard@cesi.fr, Yee Mey Goh (Y.Goh@lboro.ac.uk).

Please, fill the email object as: “[Internship] HRI evaluation in VR”

The application must contain:

• CV ;
• A cover letter for the subject ;
• Results of the current master.
• Recommendation letters if available.

Thank you to send NOM prénom.zip.

Contract: internship of 5 to 6 months, starting in September 2020.

Location :

CESI Rouen

80 Avenue Edmund Halley

Rouen Madrillet Innovation

CS 10123

76808 Saint-Etienne-du-Rouvray.

Aknowledgment :

This internship is done within the framework of the project NUMERILAB, financed by the FEDER and the Normandy region.

Ce stage se déroule dans le cadre du projet NUMERILAB, financé par la région Normandie et les fonds FEDER.

References

• Akash, K., Hu, W.-L., Jain, N., & Reid, T. (2018, 11). A Classification Model for Sensing Human Trust in Machines Using EEG and GSR. ACM Trans. Interact. Intell. Syst., 8. doi:10.1145/3132743

• Fratczak, P., Goh, Y. M., Kinnell, P., Soltoggio, A., & Justham, L. (2019). Understanding Human Behaviour in Industrial Human-Robot Interaction by Means of Virtual Reality. Proceedings of the Halfway to the Future Symposium 2019. New York, NY, USA: Association for Computing Machinery. doi:10.1145/3363384.3363403

• Havard, V., Jeanne, B., Lacomblez, M., & Baudry, D. (2019). Digital twin and virtual reality: a co-simulation environment for design and assessment of industrial workstations. Production & Manufacturing Research, 7, 472-489. doi:10.1080/21693277.2019.1660283

• Matsas, E., & Vosniakos, G.-C. (2017). Design of a virtual reality training system for human--robot collaboration in manufacturing tasks. International Journal on Interactive Design and Manufacturing (IJIDeM), 11, 139-153. doi:http://dx.doi.org/10.1007/s12008-015-0259-2