Evaluation of subjective mental workload: A comparison of SWAT, NASA‐TLX, and workload profile methods

Abstract

The present research evaluates several psychometric properties (intrusiveness, sensitivity, diagnosticity, and validity) of three multidimensional subjective workload assessment instruments: the NASA Task Load Index (TLX), the Subjective Workload Assessment Technique (SWAT), and the Workload Profile (WP). Subjects performed two laboratory tasks separately (single task)and simultaneously (dual task). The results of the ANOVAs performed showed that there are no differences with regard to the three instruments’ intrusiveness, and that among the three subjective workload instruments WP has an outstanding sensitivity to the different task manipulations. To evaluate the diagnosticity of each of the three instruments canonical discriminant analysis was used, and this demonstrated that the three multidimensional ratings provided diagnostic information on the nature of tasks demands that was consistent with the a priori task characterisation. However, the diagnostic power of WP was clearly superior to that obtained using TLX or SWAT. Pearson correlations between each performance and each subjective workload measure were calculated to evaluate the concurrent validity of each instrument with task performance, and to assess the convergent validity of the instruments. The three coefficients were positive and near to one, showing the high convergent validity of the three instruments considered in this research. Implementation requirements and subject acceptability were also compared. Finally, practical implications on the three assessment approaches are mentioned.