Ergonomics International Journal ISSN: 2577-2953 Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception Ergonomics Int J Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception Susana Rubio-Valdehita*, Mª Inmaculada López-Núñez and Eva Mª Díaz-Ramiro Department of Work Psychology, Complutense University of Madrid, Spain *Corresponding author: Susana Rubio-Valdehita, Department of Work Psychology, Complutense University of Madrid, Campus de Somosaguas, 28223, Pozuelo de Alarcón, Madrid, Spain, Email: srubiova@ucm.es Abstract Adaptation to the European Higher Education Area (EHEA) has meant a significant change in the workload of students. The main objective of this study is to evaluate and compare the perception of the workload of adapted and no-adapted to EHEA students in Psychology, and to analyze the nature of the changes produced by this adaptation considering several academic activities: attending theoretical classes, attending practical classes, conducting group work outside the classroom, searching for material and bibliography, studying and personal work, attending tutorials and other activities (attending seminars, conferences, etc.). The sample consisted of 645 students (397 no-adapted and 248 adapted). To measure the perceived workload, the NASA-TLX scale was used. The scale permits to distinguish six workload dimensions: effort, mental demand, temporal demand, physical demand, performance and frustration. The results show a significant increase in the perceived workload of attending practical classes and specially in performing group work outside the classroom. It is concluded that the adaptation to the EHEA has meant an increase in the perception of students' workload, which points out the need to develop training programs and evaluation of the teamwork competence, one of the most important transversal competence indicated in the professional profile of the degree in psychology. Keywords: Mental Workload; NASA-TLX; Psychology; Teamwork; European Higher Education Area Introduction The European Higher Education Area (EHEA), derived from the Bologna Declaration [1] aims to achieve high employability rates, improve the competitiveness of the European higher education system and promote mobility [2]. To achieve these objectives, a transformation was proposed in the structure of higher education through the European Credit Transfer System (ECTS), a system that distributes students' workload at estimated hours through face-to-face activities (assistance to classes, evaluations, etc.), non-face-to-face activities with tutoring (directed work and mentoring) and non-face- to-face activities (study and individual activities). Adaptation to the EHEA has led to important changes both at the institutional level and in the teaching Research Article Volume 1 Issue 1 Received Date: September 06, 2017 Published Date: September 13, 2017 DOI: 10.23880/eoij-16000106 mailto:srubiova@ucm.es https://doi.org/10.23880/eoij-16000106 Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 2 programs and methodologies used [3,4] with more active methods (attend seminars, solve problems, carry out projects and presentations, etc.) that give the student a greater autonomy of learning, with less face- to-face activities and greater weight of his personal work. These modifications also imply a change in the teaching-learning process, which is more constructivists [5], focused on the acquisition of competences and on the student's continuous work [6,7]. This implies a greater non-face-to-face workload (WL) [8,9]. The EHEA model establishes that each academic year consists of 60 ECTS credits. In Spain and in the specific case of Psychology, a “grado” in Psychology (four year qualification adapted to EHEA with 240 ECTS) of a general nature was proposed, with compulsory and optional blocks. This degree replaces the previous system “licenciatura” (five-year qualification reflecting the previous curriculum) [10]. One of the fundamental concepts involved in the new curricula is the students´ WL [11]. This WL can be defined as the working hours that are dedicated to the tasks both inside the educational center (attending classes, tutorials, seminars) and outside it (group work, study, bibliographical consultations, etc.), that is to say, the time required to complete an academic course and acquire the competencies established as learning outcomes. The literature consulted agrees that adjusting the workload to the credits established in the programs is one of the main difficulties that teachers have [12] and that the ECTS system supposes an increase in WL [11,13]. Although this educational reform was officially established in 2010, we do not have empirical studies that analyze the effect of such adaptation from the perspective of students, much less in the teaching of Psychology [14]. In this sense, there is an evident lack of studies that explore the impact of these changes in the perception of WL of the grado in Psychology (EHEA) compared to that of the students of the previous curricular structure (licenciatura). Ortiz, et al. [9] found, from a perspective based on the study of the times dedicated to each task, the differences in the WL of grado and licenciatura´s students and point out that the change in teaching-learning methodologies must be approached from the perspective of student. The level of WL perceived can have a very important influence on both the level of academic achievement reached and the psychological well-being of students, since it is one of the main factors that generate academic stress [15]. An excess of WL makes it difficult to acquire knowledge and promotes a superficial learning [16], failure and academic desertion [17]. The students´ WL has been evaluated from different perspectives. Traditionally, it has been assimilated to the count of hours that students dedicate to the subjects of their degree [18] although a smaller number of cases also analyzed the perception of students´ WL [19,21]. This second approach, which we follow in the present study, assumes that perceived WL is modulated by several factors, and this perception is the main stress- generating factor [22]. In this line, Kember & Leung [23] find that many hours of study are not perceived as high when the course design is adequate, and they show the scarce relation between the time spent on a task and the perception of the load [24,25]. One of the instruments most used to evaluate the perceived WL is the NASA-Task Load Index scale (TLX) [26]. NASA-TLX is a valid and reliable tool for the analysis of perceived WL in different sectors: industrial, psychological, health, aeronautical or transport [27,28]. In educational setting its use has been less frequent. López-Núñez [29] analyzes the generalizability and validity of this scale with a large sample of university students, and concludes that it is a useful tool to evaluate the WL in the educational field. Likewise, Kyndt, et al. [21] use NASA-TLX to evaluate WL with the objective of analyzing its mediating role in the relationship between student motivation and their learning strategy. All changes require an adjustment and more than a decade after the Bologna Declaration and when the Spanish´s University is currently participating in the renewal of accreditation of the adaptation of the studies to the EHEA, it is necessary to evaluate these changes. While we can say that the change has been positive in terms of inclusion of new technologies (ICTs) and methodologies, greater interaction with students, or changes in the evaluation system, other aspects such as increased WL for teachers and students do not seem so positive [2,30]. Therefore, the objective of this study is to analyze and compare the perception of WL among licenciatura students in relation to grado students (new system adapted to the EHEA) to investigate the nature of the changes produced in this variable. Methods Participants The sample consisted of 645 third-year university students of Psychology of whom 397 were licenciatura students and 248 grado students (EHEA). The distribution by sex was very similar between both qualifications (χ2 = 1.70, p = 0.430); in the licenciatura Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 3 studies, the sample consisted of 365 women (83.5%) and 32 men (16.5%), and in the grado studies 232 women (81%) and 16 men (19%). The average age of licenciatura students was 21.56 years (SD = 2.31) and the grado (EHEA) was 21.38 (SD = 2.53). The difference was not significant [F = 0.89, p = 0.382]. All participants generally attended all the classes. Instrument To evaluate WL, the NASA-TLX scale was used [26]. This instrument distinguishes six dimensions each of which is evaluated by the subjects on a scale of 0 to 100: effort, mental demand, physical demand, temporal demand, performance and frustration / dissatisfaction. The student should assess the degree to which each of their academic activities requires each of the dimensions of WL. Seven academic activities were distinguished: attending theoretical classes, attending practical classes, conducting group work outside the classroom, searching for material and bibliography, studying and personal work, attending tutorials and other activities (attending seminars, conferences, etc.). The choice of these activities was made taking into account the changes and the importance given by the EHEA to the autonomous work of the students and the acquisition of competences, both the specific ones of the degree (EHEA) as the transversal ones for their insertion in the socio-labor context [10]. With the assessments provided by each student an overall WL index for each activity was calculated as the arithmetic mean of the scores in each dimension [27]. In addition, all the participants answered a brief questionnaire in which their gender and age were collected. Procedure This is an empirical study of non-experimental cross- sectional type. The data were collected in a single session during one of the practical classes of one of the subjects of the third-year, as much in the of licenciatura studies as grado studies. All students participated anonymously and completely voluntarily, in addition signed an informed consent by accepting their participation in the study and the use of the data for research purposes only. Data Analysis All analysis was performed with the statistical software Statistical Package for the Social Science (SPSS), version 22.00. In the first place, the descriptive statistics of all variables were calculated. Subsequently, and after verifying compliance with the necessary statistical assumptions, variance analysis (ANOVA) was performed to compare the WL scores obtained by the two groups of students (Licenciatura vs. Grado) in each of the academic activities. The value of the F statistic, its significance level (set at .05) was calculated. Results First, we calculated the mean and standard deviations (SD) of the WL scores (for each WL dimension and for the overall score) in both groups (licenciatura y grado) (Figure 1). The total WL rating was obtained by averaging all load ratings of all dimensions and activities. Figure 1: Mental workload scores for each educational system. 20 25 30 35 40 45 50 55 60 Theorical classes Practical clases Group work Search materials Personal work Tutoring Other Total WL Mental workload Licenciatura Grado Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 4 Once compliance with the variance homogeneity hypothesis was verified by the Levene test (p> 0.05 in all cases), we compared the mean scores of the overall WL of each activity according to the qualification. There were statistically significant differences in the performance of group work outside the classroom [F = 28.66, p = .000], attendance to practical classes [F = 4.28, p = .027] and total WL [F = 4.14, p = .042]. In all three cases, the grado student group showed higher WL scores. In order to analyze in more detail what dimension or dimensions of WL were the cause of these differences, comparisons of means between licenciatura and grado in each of the 6 dimensions of NASA-TLX were performed for the activities in which were found significant differences in the previous analysis (Figure 2). For "Attendance to practical classes", there are significant differences only in the mental demand dimension [F = 7.11, p = .010], obtaining higher scores in the grado group. In the activity "Performing group work outside the classroom", there are significant differences in all dimensions (p< .01 in all cases) except performance. Figure 2: NASA-TLX scores for each educational system. Discussion The EHEA establishes a system of teaching and learning based on the acquisition of specific competences of the degree, and of those considered transversal for the adaptation and positive action in the labor context [31]. A student-centered system that requires organizational and planning strategies for autonomous learning, and consequently an increase in WL. The objective of this research was to analyze and compare the perception of WL that licenciatura psychology students have with respect to the new system adapted to the EHEA (grado). Taking as reference the previous theoretical framework, we can conclude that the adaptation to the EHEA has meant an increase in the perception of the WL of Psychology students, an expected result according to the ECTS credit system. One of the strengths of this paper is that, although the result is consistent with the theoretical foundation, there are few studies focused on the empirical measurement of students' perception of WL between the previous institutional curriculum and the new one. The literature on the adjustment of the Bologna Plan has focused, on the one hand, on the evaluation of WL on specific subjects or periods [9,32,33] and to know the opinion of the students about the process of adaptation to the EHEA through surveys or interviews [13,34] but there are no studies on the perceived WL of 20 30 40 50 60 70 80 Ef fo rt M en ta l P h ys ic Te m p o ra ry P er fo rm an ce Fr u st ra ti o n Ef fo rt M en ta l P h ys ic Te m p o ra ry P er fo rm an ce Fr u st ra ti o n Ef fo rt M en ta l P h ys ic Te m p o ra ry P er fo rm an ce Fr u st ra ti o n Practical classes attendance Carrying out group work outside the classroom Total Work-Load (WL) NASA-TLX scores Licenciatura Grado Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 5 the new curricula or on the title of psychology in particular, nor have studies been found on which modes and learning activities generate higher levels of WL. The use of the NASA-TLX scale has allowed us not only to know the load levels perceived by the students, but also to deepen in the knowledge of which activities are those that generate higher levels of WL as well as to recognize the factors that produce this perception. The results show that there are statistically significant differences in the activities: "Performing group work outside the classroom", "Attendance to practical classes" and "TotalWork -Load", obtaining higher scores in the group of grado than the licenciatura. As for the activity "Attendance to practical classes", there are significant differences only in the mental demand dimension, obtaining higher scores in the grado group. In the activity "Performing group work outside the classroom", there are significant differences in all dimensions except performance. These results highlight two important aspects of adaptation to the EHEA that should reflect and open future research perspectives: the first is the evidence of an increase in WL perceived by students in the new institutional curricula of psychology studies at the time of its implementation, and the second, that one of the activities that have more WL in all its dimensions is "Perform group work outside the classroom." Regarding the first consideration, several researches indicate that the WL in the grado studies is adequate to the number of credits, so that the students' WL perception may be because the new methodologies require a continuous temporal distribution in the semesters [33]. Other authors point out that the lack of relationship between dedication and performance cannot be explained by a biggest WL, but by the lack of appropriate study techniques, the scarce use of learning resources such as personalized tutoring or the need for a greater student effort [9]. At a time when new qualifications are in the process of confirmation and both students and teachers are familiar with the ECTS credit system and more active teaching and learning methodologies, we consider it appropriate to assess the perception of WL as an indicator of quality on the process of adaptation to the EHEA, the ECTS system, the teaching coordination between the courses, the evaluation systems, as well as the evolution of the students in the autonomous learning. As for the second consideration, the fact that the activity "group work outside the classroom" is the one that produces the highest WL perceived in all its dimensions except performance, indicates the difficulty that students have to adapt to a system of learning that implies the development of interpersonal skills, planning and organization, that is why they perceive the ECTS credit system with greater WL, in addition this activity feels like a stressful activity, which requires time and whose effort does not correspond to the reward obtained. The EHEA points out that one of the main transversal competences to be achieved is group work. Several studies have evaluated the acquisition and development of this competence in university students. The results indicate that it is perceived as stressful due to WL, time demand and low control, is not considered a resource that helps them in their learning and concludes that this competition is not being adequately acquired [35-37] in line with what was found in the present work. The application of teamwork in the academic field is usually carried out in what the authors call the "black box" model [38]. That is, the usual is that the competence of teamwork is not evaluated, but only the quality of work once completed. Therefore, students who develop teamwork competence are left alone for simple exposure without providing resources or guidance, mistakenly assuming that students have access to university degrees with these skills previously acquired in non-university education. The results indicate the need to develop training programs and evaluation of this transversal competence since it is precisely one of the most important ones indicated in the professional profile of the degree in psychology [10]. At present there are studies, researches and models that have developed methodologies to train in the skills of teamwork and to be able to evaluate it. An example is the Comprehensive Training Model of the Teamwork Competence (CTMTC) [39]. The use of new technologies (ICTs) has also been proposed as a useful pedagogical tool for the development of this and other competences [40]. The application of these methods has proven to be very effective in the development of team work competence, but requires a significant effort on the part of teachers and educational administration [41]. It would be advisable to train teachers in the use of these methodologies, as well as provide them with the necessary resources so that they can be implemented in classrooms, so that the entire acquisition of this competence does not fall exclusively on students. Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 6 Finally, as regards the limitations of the study, it should be pointed out that this is an incidental sample of a single university, so that a larger sample size of different Spanish universities would have allowed a greater generalization of the results obtained. Nevertheless, the implication of the results allows us to open new lines of research on the differences detected and to propose future studies considering the perception of WL as an indicator of quality and evaluation of the process of adaptation to the EHEA. References 1. Bologna Declaration (1999) Joint declaration of the European Ministers of Education. The European higher education area. 2. Ariza T, Quevedo-Blasco R, Bermúdez MP, Buela- Casal G (2013) Analysis of postgraduate programs in the EHEA and the USA. Revista de Psicodidáctica 18(1): 197-219. 3. Ariza T, Bermúdez MP, Quevedo-Blasco R, Buela- Casal G (2012) Evolución de la legislación de doctorado en los países del EEES [Evolution of doctoral legislation in the EHEA countries]. Revista Iberoamericana de Psicología y Salud 3(2): 89-108. 4. Ortega R, Zych I (2013) Professional psychologists and university training in psychology: looking for references to the Degree in Psychology. International Journal of Psychology and Psychological Therapy 13(1): 83-96. 5. Tejada Á, López M (2012) Nuevas metodologías docentes en los títulos de grado: la literatura como recurso pedagógico colaborativo. Aula Abierta 40(3):107-114. 6. De Miguel M (2006) Metodología de enseñanza y aprendizaje para el desarrollo de competencias. Madrid: Alianza editorial. 7. Rodrigo M, Almiron N (2013) Autopercepción de la adquisición de competencias de los estudiantes de periodismo. el caso de la Universitat Pompeu Fabra 41(1): 99-110. 8. Díez MC, García JN, IPDDA (2010) Percepción de metodologías docentes y desarrollo de competencias al EEES. Boletín de Psicología 99: 45- 69. 9. Ortiz VM, Jenaro C, Meilán JG, Zubiauz B, Mayor MA, et al. (2011) Carga de Trabajo en el EEES: La necesidad de coordinación docente entre asignaturas. Proceedings of the IX Conference of Research Networks in University Teaching 1863- 1877. 10. ANECA (2005) Libro blanco del título de grado en Psicología. 11. Roca-Cuberes C (2013) La percepción de los estudiantes sobre su experiencia de aprendizaje a partir de su dedicación temporal y adquisición de competencias: un estudio sobre la implementación del Grado de Traducción e Interpretación de la Universidad Pompeu Fabra. Revista Complutense de Educación 24(2): 359-379. 12. Reyes JR, Valdés A, Castaño S (2006) Prácticum y carga de trabajo. Revista de Investigación Educativa 24 (2): 557-574. 13. Rodríguez-Izquierdo RM (2014) Modelo formativo en el Espacio Europeo de Educación Superior: valoraciones de los estudiantes, Aula Abierta 42(2): 106-113. 14. Ramiro-Sánchez T, Bermúdez MP, Buela-Casal G (2016) University Qualification in Psychology in EHEA: Comparison between Spanish Grado and Licenciatura Qualification. Revista de Psicodidáctica 21(1): 175-189. 15. Cabanach RG, Souto-Gestal A, Franco V (2016) Escala de Estresores Académicos para la evaluación de los estresores académicos en estudiantes universitarios. Revista Iberoamericana de Psicología y Salud 7(2): 41-50. 16. Lam P, McNaught C, Lee J, Chan M (2012) The impact of student workload on learning experiences. Working Paper 12. Hong Kong: Centre for Learning Enhancement and Research, the Chinese University of Hong Kong. 17. Cope C, Staehr L (2005) Improving students’ learning approaches through intervention in an information systems learning environment. Studies in Higher Education 30(2): 181-197. 18. Ruiz-Gallardo JR, Castaño S, Gómez-Alday JJ, Valdés A (2011) Assessing student workload in problem based learning: Relationships among teaching method, student workload and achievement. A case study in Natural Sciences. Teaching and Teacher Education 27(3): 619-627. 19. Karagiannopoulou E, Christodoulides P (2005) The impact of Greek university students’ perceptions of their learning environment on approaches to studying and academic outcomes. International Journal of Educational Research 43(6): 329-350. https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf https://www.eurashe.eu/library/modernising-phe/Bologna_1999_Bologna-Declaration.pdf http://www.ugr.es/~aepc/articulo/3.pdf http://www.ugr.es/~aepc/articulo/3.pdf http://www.ugr.es/~aepc/articulo/3.pdf http://www.ugr.es/~aepc/articulo/3.pdf http://www.ijpsy.com/volumen13/num1/348.html http://www.ijpsy.com/volumen13/num1/348.html http://www.ijpsy.com/volumen13/num1/348.html http://www.ijpsy.com/volumen13/num1/348.html http://www.ijpsy.com/volumen13/num1/348.html https://dialnet.unirioja.es/servlet/articulo?codigo=3994641 https://dialnet.unirioja.es/servlet/articulo?codigo=3994641 https://dialnet.unirioja.es/servlet/articulo?codigo=3994641 https://dialnet.unirioja.es/servlet/articulo?codigo=3994641 http://www.uv.es/RELIEVE/v13n1/RELIEVEv13n1_revbook1.htm http://www.uv.es/RELIEVE/v13n1/RELIEVEv13n1_revbook1.htm http://www.uv.es/RELIEVE/v13n1/RELIEVEv13n1_revbook1.htm https://dialnet.unirioja.es/servlet/articulo?codigo=4097779 https://dialnet.unirioja.es/servlet/articulo?codigo=4097779 https://dialnet.unirioja.es/servlet/articulo?codigo=4097779 https://dialnet.unirioja.es/servlet/articulo?codigo=4097779 http://www.uv.es/seoane/boletin/previos/N99-3.pdf http://www.uv.es/seoane/boletin/previos/N99-3.pdf http://www.uv.es/seoane/boletin/previos/N99-3.pdf http://www.uv.es/seoane/boletin/previos/N99-3.pdf http://www.aneca.es/var/media/150356/libroblanco_psicologia_def.pdf http://www.aneca.es/var/media/150356/libroblanco_psicologia_def.pdf http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.ucm.es/index.php/RCED/article/view/42084 http://revistas.um.es/rie/article/viewFile/97191/93321 http://revistas.um.es/rie/article/viewFile/97191/93321 http://revistas.um.es/rie/article/viewFile/97191/93321 http://www.redalyc.org/articulo.oa?id=17543420010 http://www.redalyc.org/articulo.oa?id=17543420010 http://www.redalyc.org/articulo.oa?id=17543420010 http://www.redalyc.org/articulo.oa?id=17543420010 http://www.redalyc.org/articulo.oa?id=17543420010 http://www.redalyc.org/articulo.oa?id=245145815001 http://www.redalyc.org/articulo.oa?id=245145815001 http://www.redalyc.org/articulo.oa?id=245145815001 http://www.redalyc.org/articulo.oa?id=245145815001 http://www.redalyc.org/articulo.oa?id=245145815001 https://www.cuhk.edu.hk/clear/research/WP12_LMcNLC_2012.pdf https://www.cuhk.edu.hk/clear/research/WP12_LMcNLC_2012.pdf https://www.cuhk.edu.hk/clear/research/WP12_LMcNLC_2012.pdf https://www.cuhk.edu.hk/clear/research/WP12_LMcNLC_2012.pdf https://www.cuhk.edu.hk/clear/research/WP12_LMcNLC_2012.pdf http://www.tandfonline.com/doi/abs/10.1080/03075070500043275 http://www.tandfonline.com/doi/abs/10.1080/03075070500043275 http://www.tandfonline.com/doi/abs/10.1080/03075070500043275 http://www.tandfonline.com/doi/abs/10.1080/03075070500043275 Ergonomics International Journal Susana Rubio-Valdehita, et al. Ergonomic Assessment of Mental Workload in Higher Education. Effects of Education System on Student’s Workload Perception. Ergonomics Int J 2017, 1(1): 000106. Copyright© Susana Rubio-Valdehita, et al. 7 20. Kyndt E, Berghmans I, Dochy F, Bulckens L (2014) “Time is not enough”. Workload in higher education: A student perspective. Higher Education Research & Development 33(4): 684-698. 21. Kyndt E, Dochy F, Struyven K, Cascallar E (2011) The direct and indirect effect of motivation for learning on students' approaches to learning through the perceptions of workload and task complexity. Higher Education Research & Development 30(2): 135-150. 22. Ramsden P (1992) Learning to teach in higher education. Routledge, Londres, UK. 23. Kember D, Leung DY (2006) Characterising a teaching and learning environment conducive to making demands on students while not making their workload excessive. Studies in Higher Education 31(2): 185-198. 24. Hertzum M, Holmegaard KD (2013) Perceived time as a measure of mental workload: Effects of time constraints and task success. International Journal of Human-Computer Interaction 29(1): 26-39. 25. Nosair E, Hamdy H (2017) Total Student Workload: Implications of the European Credit Transfer and Accumulation System for an Integrated, Problem- Based Medical Curriculum. Health Professions Education. 26. Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: PA Hancock y, N Meshkati (Eds.), Human mental workload, Amsterdam, pp: 139-183. 27. Hart SG (2006) NASA-Task Load Index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50(9): 904-908. 28. Young G, Zavelina L, Hooper V (2008) Assessment of workload using NASA task load index in perianesthesia nursing. Journal of Perianesthesia Nursing 23(2): 102-110. 29. López-Núñez MI (2010) Generalización al ámbitolaboral de dos instrumentos de medidasubjetiva de la carga mental (tesis doctoral). Madrid: UCM. 30. Quevedo-Blasco R, Ariza T, Buela Casal G (2015) Evaluación de la satisfacción del profesorado de ciencias con la adaptación al Espacio Europeo de Educación Superior. Educación XX1 18(1): 45-70. 31. Dunn TJ, Baguley T, Brunsden V (2014) From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology 105(3): 399-412. 32. Rychen D, Salganik H (2006) Las competencias clave para el bienestar personal, social y económico. Málaga. 33. Julián J, Zaragoza J, Castejón FJ, López VM (2010) Carga de trabajoen differentes asignaturas que experimentan el sistema ECTS. International Journal of Medicine and Science of Physical Activity and Sport 10(38): 218-233. 34. Llamas M, Caeiro M, Castro M, Plaza I, Tovar E (2012) Work in progress: Preliminary survey results on the first year of the Bologna process in engineering education in Spain. Frontiers in Education Conference, IEEE, pp: 1-2. 35. Alonso-Martín P (2010) La valoración de la importancia de las competenciastransversales: comparación de supercepción al inicio y final de cursoenalumnos de psicología. Revista de investigación educative 28(1): 119-140. 36. Arce R, Fariña F, Novo M, Seijo D (2012) Efecto del sistema de enseñanza en el rendimiento académico, burnout experimentado y estrés academic. Aula abierta 40(2): 3-10. 37. París G, Torrelles C, Mas O (2016) La evaluación de la competencia “trabajo en equipo” de los estudiantes universitarios. RIDU 8: 86-97. 38. Fidalgo-Blanco Á, Lerís D, Sein-Echaluce ML, García- Peñalvo FJ (2013) Indicadores para el seguimiento y evaluación de la competencia de trabajo en equipo a través del método CTMTC. CINAIC. 39. Lerís D, Fidalgo Á, Sein-Echaluce ML (2014) A comprehensive training model of the teamwork competence. International Journal of Learning and Intellectual Capital 11(1): 1-19. 40. Guitert M, Romeu T, Pérez-Mateo M (2007) Competencias TIC y trabajo en equipo en entornos virtuales. International Journal of Educational Technology in Higher Education (ETHE) 4(1). 41. Tobón S (2010) Formación integral y competencias. Editorial Macro, Lima. http://www.tandfonline.com/doi/abs/10.1080/07294360.2013.863839 http://www.tandfonline.com/doi/abs/10.1080/07294360.2013.863839 http://www.tandfonline.com/doi/abs/10.1080/07294360.2013.863839 http://www.tandfonline.com/doi/abs/10.1080/07294360.2013.863839 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 http://www.tandfonline.com/doi/abs/10.1080/07294360.2010.501329 https://eric.ed.gov/?id=EJ734531 https://eric.ed.gov/?id=EJ734531 https://eric.ed.gov/?id=EJ734531 https://eric.ed.gov/?id=EJ734531 https://eric.ed.gov/?id=EJ734531 http://ccc.ku.dk/employees/?pure=en%2Fpublications%2Fperceived-time-as-a-measure-of-mental-workload-effects-of-time-constraints-and-task-success(614faaed-2ffd-4dfb-b698-1facd78bd949)%2Fexport.html http://ccc.ku.dk/employees/?pure=en%2Fpublications%2Fperceived-time-as-a-measure-of-mental-workload-effects-of-time-constraints-and-task-success(614faaed-2ffd-4dfb-b698-1facd78bd949)%2Fexport.html http://ccc.ku.dk/employees/?pure=en%2Fpublications%2Fperceived-time-as-a-measure-of-mental-workload-effects-of-time-constraints-and-task-success(614faaed-2ffd-4dfb-b698-1facd78bd949)%2Fexport.html http://ccc.ku.dk/employees/?pure=en%2Fpublications%2Fperceived-time-as-a-measure-of-mental-workload-effects-of-time-constraints-and-task-success(614faaed-2ffd-4dfb-b698-1facd78bd949)%2Fexport.html http://journals.sagepub.com/doi/abs/10.1177/154193120605000909?journalCode=proe http://journals.sagepub.com/doi/abs/10.1177/154193120605000909?journalCode=proe http://journals.sagepub.com/doi/abs/10.1177/154193120605000909?journalCode=proe http://journals.sagepub.com/doi/abs/10.1177/154193120605000909?journalCode=proe https://www.ncbi.nlm.nih.gov/pubmed/18362006 https://www.ncbi.nlm.nih.gov/pubmed/18362006 https://www.ncbi.nlm.nih.gov/pubmed/18362006 https://www.ncbi.nlm.nih.gov/pubmed/18362006 http://revistas.uned.es/index.php/educacionXX1/article/view/12311 http://revistas.uned.es/index.php/educacionXX1/article/view/12311 http://revistas.uned.es/index.php/educacionXX1/article/view/12311 http://revistas.uned.es/index.php/educacionXX1/article/view/12311 https://www.ncbi.nlm.nih.gov/pubmed/24844115 https://www.ncbi.nlm.nih.gov/pubmed/24844115 https://www.ncbi.nlm.nih.gov/pubmed/24844115 https://www.ncbi.nlm.nih.gov/pubmed/24844115 http://www.agapea.com/libros/Las-competencias-clave-para-el-bienestar-personal-social-y-economico-9788497003759-i.htm http://www.agapea.com/libros/Las-competencias-clave-para-el-bienestar-personal-social-y-economico-9788497003759-i.htm http://www.agapea.com/libros/Las-competencias-clave-para-el-bienestar-personal-social-y-economico-9788497003759-i.htm http://cdeporte.rediris.es/revista/revista38/artcarga151.htm http://cdeporte.rediris.es/revista/revista38/artcarga151.htm http://cdeporte.rediris.es/revista/revista38/artcarga151.htm http://cdeporte.rediris.es/revista/revista38/artcarga151.htm http://cdeporte.rediris.es/revista/revista38/artcarga151.htm http://ieeexplore.ieee.org/document/6462432/?reload=true http://ieeexplore.ieee.org/document/6462432/?reload=true http://ieeexplore.ieee.org/document/6462432/?reload=true http://ieeexplore.ieee.org/document/6462432/?reload=true http://ieeexplore.ieee.org/document/6462432/?reload=true http://revistas.um.es/rie/article/view/97821 http://revistas.um.es/rie/article/view/97821 http://revistas.um.es/rie/article/view/97821 http://revistas.um.es/rie/article/view/97821 http://revistas.um.es/rie/article/view/97821 https://dialnet.unirioja.es/servlet/articulo?codigo=3920846 https://dialnet.unirioja.es/servlet/articulo?codigo=3920846 https://dialnet.unirioja.es/servlet/articulo?codigo=3920846 https://dialnet.unirioja.es/servlet/articulo?codigo=3920846 https://dialnet.unirioja.es/servlet/articulo?codigo=5641682 https://dialnet.unirioja.es/servlet/articulo?codigo=5641682 https://dialnet.unirioja.es/servlet/articulo?codigo=5641682 https://gredos.usal.es/jspui/bitstream/10366/122531/1/GRIAL_PaperIndicadores.pdf https://gredos.usal.es/jspui/bitstream/10366/122531/1/GRIAL_PaperIndicadores.pdf https://gredos.usal.es/jspui/bitstream/10366/122531/1/GRIAL_PaperIndicadores.pdf https://gredos.usal.es/jspui/bitstream/10366/122531/1/GRIAL_PaperIndicadores.pdf http://www.inderscienceonline.com/doi/abs/10.1504/IJLIC.2014.059216 http://www.inderscienceonline.com/doi/abs/10.1504/IJLIC.2014.059216 http://www.inderscienceonline.com/doi/abs/10.1504/IJLIC.2014.059216 http://www.inderscienceonline.com/doi/abs/10.1504/IJLIC.2014.059216 https://www.uoc.edu/rusc/4/1/dt/esp/guitert_romeu_perez-mateo.pdf https://www.uoc.edu/rusc/4/1/dt/esp/guitert_romeu_perez-mateo.pdf https://www.uoc.edu/rusc/4/1/dt/esp/guitert_romeu_perez-mateo.pdf https://www.uoc.edu/rusc/4/1/dt/esp/guitert_romeu_perez-mateo.pdf https://creativecommons.org/licenses/by/4.0/ Abstract Keywords Introduction Methods Discussion References