A video processing and machine learning based method for evaluating safety-critical operator engagement in a motorway control room.
In safety-critical automatic systems, safety can be compromised if operators lack engagement. Effective detection of undesirable engagement states can inform the design of interventions for enhancing engagement. However, the existing engagement measurement methods suffer from several limitations whi...
| Publicado en: | Ergonomics Vol. 67; no. 3; pp. 356 - 377 |
|---|---|
| Autores principales: | , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Taylor & Francis Ltd
Mar2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=176106902&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 176106902 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 00140139 ERO jtl: Ergonomics issn: 00140139 maglogo: Y pubinfo: dt: Mar2024 vid: 67 iid: 3 pid: 377 pub: Taylor & Francis Ltd place: Philadelphia, Pennsylvania artinfo: ui: 176106902 164218797 176106902 176106902 10.1080/00140139.2023.2223784 176106902 ppf: 356 ppct: 21 formats: tig: atl: A video processing and machine learning based method for evaluating safety-critical operator engagement in a motorway control room. aug: au: Jin, Linyi Ren, Qingyu Mitchell, Val May, Andrew affil: School of Design and Creative Arts, Loughborough University, Loughborough, UK sug: subj: Videorecording Machine Learning Safety Work Engagement Evaluation Posture Evaluation Ergonomics Automation Accidents, Traffic Prevention and Control Human China Support Vector Machine Stress, Occupational Deep Learning Task Performance and Analysis Descriptive Statistics Female Male Female Male ab: In safety-critical automatic systems, safety can be compromised if operators lack engagement. Effective detection of undesirable engagement states can inform the design of interventions for enhancing engagement. However, the existing engagement measurement methods suffer from several limitations which damage their effectiveness in the work environment. A novel engagement evaluation methodology, which adopts Artificial Intelligence (AI) technologies, has been proposed. It was developed using motorway control room operators as subjects. Openpose and Open Source Computer Vision Library (OpenCV) were used to estimate the body postures of operators, then a Support Vector Machine (SVM) was utilised to build the engagement evaluation model based on discrete states of operator engagement. The average accuracy of the evaluation results reached 0.89 and the weighted average precision, recall, and F1-score were all above 0.84. This study emphasises the importance of specific data labelling when measuring typical engagement states, forming the basis for potential control room improvements. Practitioner summary: This study demonstrates an automatic, real-time, objective, and relatively unobtrusive method for measuring dynamic operator engagement states. Computer vision technologies were used to estimate body posture, then machine learning (ML) was utilised to build the engagement evaluation model. The overall evaluation shows the effectiveness of this framework. Abbreviations: AI: Artificial Intelligence; OpenCV: Open Source Computer Vision Library; SVM: Support Vector Machine; UWES: Utrecht Work Engagement Scale; ISA Engagement Scale: Intellectual, Social, Affective Engagement Scale; DSSQ: Dundee Stress State Questionnaire; SSSQ: Short Stress State Questionnaire; EEG: electroencephalography; ECG: Electrocardiography; VMOE: Video-based Measurement for Operator Engagement; CMU: Carnegie Mellon University; CNN: Convolutional Neural Network; 2D: two dimensional; ML: Machine learning. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|