D6.11 Evaluation Process and Results for Healthcare Engineering case

The purpose of this D6.11 document is to describe and interpret the final trial of the WEKIT prototype for the Healthcare Engineering Industrial Case. The selected use case for the trial is a core procedure to teach and train a student to perform a carotid ultrasound examination with quality intima media thickness measurements. The trainees performed the steps of the procedure using the WEKIT Player application, while the creation and validation of the training procedure has been done by the EBIT trainers using the WEKIT Recorder application. A control group used a paper procedure with the same instructions.

The challenge for each trial was: a) to explore the use of AR/WT to achieve the necessary level of trainee performance, in less time and making fewer errors than conventional training; b) help trainees to use the Wearable Experience methodology to record critical events and share them with their trainer and other trainees; c) extensively experiment with the use of the WEKIT recorder by experts to create authoritative sets of training procedures. All aspects of that challenge were met in the trial as will be presented in the following deliverable. The space and aviation trials had in common the use of the player to show the performance standards that trainees had to meet and the expected measurements associated with each workflow (as in the ‘after-action reviews’ that are required in many safety-critical contexts and in field-service maintenance). Of course, while standardisation of work flows and training is key in health engineering too, there can be much wider variation in the presenting conditions of hospital patients, adding complexity to the patient-side of the health trial.

The trials took place in different facilities with different Ultrasound equipment and procedures created and authored via the WEKIT recorder. Such trial organization allowed us to try and test the flexibility of the WEKIT tool platform with different environments and trainers and made us able to collect feedback about usability of the WEKIT application in the healthcare scenarios.

The trial sessions have been executed from July 2018 to February 2019. In the healthcare trial a total of 114 candidates have been involved.

The data and feedback from the participants were collected by means of on-line surveys designed in order to get comprehensive feedback on their experience.

The data collected from the participants have been analysed, and the results are described in this report. The longer trial timeframe was needed in order to comply with the objective to reach a large number of candidates for each industrial partner to sustain the test.