Martinez-Maldonado, R, Elliott, D, Axisa, C, Power, T, Echeverria, V & Shum, SB 2020, 'Designing translucent learning analytics with teachers: an elicitation process', INTERACTIVE LEARNING ENVIRONMENTS.View/Download from: Publisher's site
Echeverria, V, Martinez-Maldonaldo, R, Buckingham Shum, S, Chiluiza, K, Granda, R & Conati, C 2018, 'Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers’ Attention through Educational Data Storytelling', Journal of Learning Analytics, vol. 5, no. 3, pp. 72-97.View/Download from: Publisher's site
Martinez-Maldonado, R, Echeverria, V, Elliott, D, Axisa, C, Power, T & Shum, SB 2019, 'Making the design of cscl analytics interfaces a co-design process: The case of multimodal teamwork in healthcare', Computer-Supported Collaborative Learning Conference, CSCL, pp. 859-860.View/Download from: UTS OPUS
© ISLS. Multimodal Learning Analytics innovations offer exciting opportunities for Computer-Supported Collaborative Learning (CSCL) practice and research, but they also make more evident the need to make the design of analytics tool into a horizontal, co-design process. The emergence of new algorithms and sensors can be a major breakthrough in the way CSCL research is conducted and automated feedback is provided. However, there still is a lack of research on how these innovations can be used by teachers and learners, as most existing systems are restricted to experimental research setups. This poster paper sheds light on the first steps that can be made towards making the design of CSCL analytics interfaces a co-design process where teachers, learners and other stakeholders become design partners.
Echeverria, V, Martinez-Maldonado, R & Shum, SB 2019, 'Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data', CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, CHI Conference on Human Factors in Computing Systems (CHI), ASSOC COMPUTING MACHINERY, Glasgow, SCOTLAND.View/Download from: Publisher's site
Echeverria, V, Martinez-Maldonado, R, Granda, R, Chiluiza, K, Conati, C & Shum, SB 2018, 'Driving data storytelling from learning design', Proceedings of the 8th International Conference on Learning Analytics and Knowledge, International Conference on Learning Analytics and Knowledge, ACM, Sydney, New South Wales, Australia, pp. 131-140.View/Download from: UTS OPUS or Publisher's site
© 2018 Association for Computing Machinery. Data science is now impacting the education sector, with a growing number of commercial products and research prototypes providing learning dashboards. From a human-centred computing perspective, the end-user’s interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that ‘usable’ visualisations are not necessarily effective from a learning perspective. Since an educator’s interpretation of visualised data is essentially the construction of a narrative about student progress, we draw on the growing body of work on Data Storytelling (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators’ responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined.
Echeverria, V, Martinez-Maldonado, R, Power, T, Hayes, C & Shum, SB 2018, 'Where is the Nurse? Towards automatically visualising meaningful team movement in healthcare education', 19th International Conference, AIED 2018, London, UK, June 27–30, 2018, Proceedings, Part II (LNCS), International Conference on Artificial Intelligence in Education, Springer, London, United Kingdom, pp. 74-78.View/Download from: UTS OPUS or Publisher's site
© Springer International Publishing AG, part of Springer Nature 2018. Providing immediate, effective feedback on team and individual performance in healthcare simulations is a challenging task for educators, such is their complexity. Focusing on emergency procedures on patient manikins, our prior work has demonstrated the feasibility of using multimodal data capture and analysis to generate visualisations of student movement, talk and treatment actions. The limitation to date has been the need for manual steps in the analytic workflow. This paper documents how we have automated several key steps, using new technologies, which were piloted during a nursing simulation. Combining role-based nurses’ movement data with high fidelity manikin logs, we have implemented a zone-based classification model, and are able to automatically visualise movements within an emergency response team, providing the data needed to design near real-time feedback for both educators and students.
Martinez-Maldonado, R, Echeverria, V, Prieto, LP, Rodriguez-Triana, MJ, Spikol, D, Curukova, M, Mavrikis, M, Ochoa, X & Worsley, M 2018, '2nd Crossmmla: Multimodal learning analytics across physical and digital spaces', CEUR Workshop Proceedings.View/Download from: UTS OPUS
© 2018 CEUR-WS. All Rights Reserved. Students’ learning is ubiquitous. It happens wherever the learner is rather than being constrained to a specific physical or digital learning space (e.g. the classroom or the institutional LMS respectively). A critical question is: how to integrate and coordinate learning analytics to provide continued support to learning across physical and digital spaces? CrossMMLA is the successor to the Learning Analytics Across Spaces (CrossLAK) and MultiModal Learning Analytics (MMLA) series of workshops that were merged in 2017 after successful cross-pollination between the two communities. Although it may be said that CrossLAK and MMLA perspectives follow different philosophical and practical approaches, they both share a common aim. This aim is: deploying learning analytics innovations that can be used across diverse authentic learning environments whilst learners feature various modalities of interaction or behaviour.
Martinez-Maldonado, R, Echeverria, V, Santos, OC, Dos Santos, ADP & Yacef, K 2018, 'Physical learning analytics: A multimodal perspective', ACM International Conference Proceeding Series, pp. 375-379.View/Download from: UTS OPUS or Publisher's site
© 2018 Association for Computing Machinery. The increasing progress in ubiquitous technology makes it easier and cheaper to track students’ physical actions unobtrusively, making it possible to consider such data for supporting research, educator interventions, and provision of feedback to students. In this paper, we reflect on the underexplored, yet important area of learning analytics applied to physical/motor learning tasks and to the physicality aspects of ‘traditional’ intellectual tasks that often occur in physical learning spaces. Based on Distributed Cognition theory, the concept of Internet of Things and multimodal learning analytics, this paper introduces a theoretical perspective for bringing learning analytics into physical spaces. We present three prototypes that serve to illustrate the potential of physical analytics for teaching and learning. These studies illustrate advances in proximity, motion and location analytics in collaborative learning, dance education and healthcare training.
© 2017, CEUR-WS. All rights reserved. This work presents a multimodal dataset of 17 workgroup sessions in a collaborative learning activity. Workgroups were conformed of two or three students using a tabletop application in a co-located space. The dataset includes time-synchronized audio, video and tabletop system's logs. Some challenges were identified during the collection of the data, such as audio participation identification, and user traces identification. Future work should explore how to overcome the aforementioned difficulties.
Echeverria, V, Falcones, G, Castells, J, Martinez-Maldonado, R & Chiluiza, K 2017, 'Exploring on-time automated assessment in a co-located collaborative system', 2017 4th International Conference on eDemocracy and eGovernment, ICEDEG 2017, International Conference on eDemocracy & eGovernment, IEEE, Quito, Ecuador, pp. 273-276.View/Download from: UTS OPUS or Publisher's site
© 2017 IEEE. This research explores the effects of providing on-time automated assessment in a co-located collaborative system for Entity-Relationship design. In addition, students' perceptions about the validity and usefulness of this automated assessment, and its potential for reflection were analyzed. Thirty undergraduate students from computer science participated in the study, a quasi-experimental pre-post test design was conducted. Short quizzes for measuring students' learning performance were used and the perceptions of students were gathered by means of questionnaires. Results showed that on-time assessment positively affected students' learning performance in the study. Finally, students perceived that the system assessment is valid and useful and has the potential to generate mechanisms for reflecting about individual learning and group performance. The study concludes with steps for further research.
Echeverria, V, Martinez-Maldonado, R & Shum, SB 2017, 'Towards data storytelling to support teaching and learning', ACM International Conference Proceeding Series, 29th Australian Conference on Computer-Human Interaction 2017, Brisbane, Queensland, Australia, pp. 347-351.View/Download from: UTS OPUS or Publisher's site
© 2017 Association for Computing Machinery. All rights reserved. Data science is now impacting the educational sector, with a growing number of commercial products and research prototypes providing learning dashboards as feedback for both educators and students. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that 'usable' visualisations are not necessarily effective from a teaching and learning perspective. Since an educator's interpretation of visualised data is essentially the construction of a narrative about that student's progress, we draw on the growing body of work on 'Data Storytelling' (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators' responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined. The results suggest that DS elements could add clarity, especially when there are multiple possible stories in a complex visualisation.
Echeverria, V, Martinez-Maldonado, R, Chiluiza, K & Shum, SB 2017, 'DBCollab: Automated Feedback for Face-to-Face Group Database Design', 25TH INTERNATIONAL CONFERENCE ON COMPUTERS IN EDUCATION (ICCE 2017), International Conference on Computers in Education, ASIA PACIFIC SOC COMPUTERS IN EDUCATION, Christchurch, New Zealand, pp. 156-165.View/Download from: UTS OPUS
Martinez-Maldonado, R, Echeverria, V, Yacef, K, Dos Santos, ADP & Pechenizkiy, M 2017, 'How to capitalise on mobility, proximity and motion analytics to support formal and informal education?', CEUR Workshop Proceedings, Multimodal Learning Analytics Workshop & International Learning Analytics and Knowledge Conference, CEUR, Vancouver, Canada, pp. 39-46.View/Download from: UTS OPUS
© 2017, CEUR-WS. All rights reserved. Learning Analytics and similar data-intensive approaches aimed at understanding and/or supporting learning have mostly focused on the analysis of students' data automatically captured by personal computers or, more recently, mobile devices. Thus, most student behavioural data are limited to the interactions between students and particular learning applications. However, learning can also occur beyond these interface interactions, for instance while students interact face-to-face with other students or their teachers. Alternatively, some learning tasks may require students to interact with non-digital physical tools, to use the physical space, or to learn in different ways that cannot be mediated by traditional user interfaces (e.g. motor and/or audio learning). The key questions here are: why are we neglecting these kinds of learning activities? How can we provide automated support or feedback to students during these activities? Can we find useful patterns of activity in these physical settings as we have been doing with computer-mediated settings? This position paper is aimed at motivating discussion through a series of questions that can justify the importance of designing technological innovations for physical learning settings where mobility, proximity and motion are tracked, just as digital interactions have been so far.
Martinez-Maldonado, R, Yacef, K, Santos, ADPD, Shum, SB, Echeverria, V, Santos, OC & Pechenizkiy, M 2017, 'Towards Proximity Tracking and Sensemaking for Supporting Teamwork and Learning', Proceedings - IEEE 17th International Conference on Advanced Learning Technologies, ICALT 2017, 2017 IEEE 17th International Conference on Advanced Learning Technologies, IEEE, Timisoara, Romania, pp. 89-91.View/Download from: UTS OPUS or Publisher's site
© 2017 IEEE. A large number of learning tools offering some sort of personalisation features rely mainly on the analysis of logged interactions between students and particular user interfaces. Much less attention has been given to the analysis of physical aspects so often present in 'traditional' intellectual tasks, although these are both important in the full development of a life-long learner. This paper (1) discusses existing literature focused on supporting learning using proximity and location analytics and sensors, and, based on this, (2) illustrates the feasibility and potential of these analytics for teaching and learning through an study in the context of proximity and location analytics in a team-based health simulation classroom.
Echeverría, V, Domínguez, F & Chiluiza, K 2016, 'Towards a distributed framework to analyze multimodal data', CEUR Workshop Proceedings, pp. 52-57.
© Copyright 2016 for this paper by its authors. Data synchronization gathered from multiple sensors and its corresponding reliable data analysis has become a difficult challenge for scalable multimodal learning systems. To tackle this particular issue, we developed a distributed framework to decouple the capture task from the analysis task through nodes across a publish/subscription server. Moreover, to validate our distributed framework we build a multimodal learning system to give on-time feedback for presenters. Fifty-four presenters used the system. Positive perceptions about the multimodal learning system were received from presenters. Further functionality of the framework will allow an easy plug and play deployment for mobile devices and gadgets.
Falcones, G, Wong-Villacres, M, Echeverria Barzola, V & Chiluiza Garcia, K 2016, 'Enhancing Quality of Argumentation in a Co-located Collaborative Environment through a Tabletop System', 2016 IEEE ECUADOR TECHNICAL CHAPTERS MEETING (ETCM), IEEE Ecuador Technical Chapters Meeting (ETCM), IEEE, Guayaquil, ECUADOR.
Luzardo, G, Echeverria, V, Quinonez, Y & Granda, R 2015, 'Multi-tabletop System to Support Collaborative Design Assessment', TRENDS AND APPLICATIONS IN SOFTWARE ENGINEERING, 4th International Conference on Software Process Improvement (CIMPS), SPRINGER-VERLAG BERLIN, Autonomous Univ Sinaloa, Fac Informatic Mazatlan, Culiacan, MEXICO, pp. 271-281.View/Download from: Publisher's site
Wong-Villacres, M, Barzola, VE, Granda, R & Garcia, KC 2016, 'Portable tabletops: A low-cost pen-and-touch approach', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 241-252.View/Download from: Publisher's site
© Springer International Publishing Switzerland 2016. This paper describes the design and implementation of a lowcost portable tabletop to be used in classrooms. This solution enables pen and touch interactions over a projected canvas. Twelve users participated in a user study that gauged the system’s effectiveness to support drawing and moving objects on the surface. Additionally, a stress test to evaluate users’ identification was conducted. Results showed that the system exhibited a fair effectiveness when users draw or move objects. Average errors of 5.6% to 6.5% were found when differentiating users. In general, the proposed tabletop is a promising solution at an affordable price; nevertheless, three key challenges need to be addressed before a full deployment of the solution: a better precision to draw complex shapes, a better gesture intepretation when rotating objects and achieving a minimum error for user differentiation.
Dominguez, F, Echeverria, V, Chiluiza, K & Ochoa, X 2015, 'Multimodal Selfies: Designing a Multimodal Recording Device for Students in Traditional Classrooms', ICMI'15: PROCEEDINGS OF THE 2015 ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2015 ACM International Conference on Multimodal Interaction, ASSOC COMPUTING MACHINERY, Seattle, WA, pp. 567-574.View/Download from: Publisher's site
Echeverria, V, Guaman, B & Chiluiza, K 2015, 'Mirroring Teachers' Assessment of Novice Students' Presentations Through an Intelligent Tutor System', 2015 ASIA-PACIFIC CONFERENCE ON COMPUTER-AIDED SYSTEM ENGINEERING - APCASE 2015, 2015 Asia-Pacific Conference on Computer Aided System Engineering, IEEE, Quito, ECUADOR, pp. 264-269.View/Download from: Publisher's site
Granda, RX, Echeverria, V, Chiluiza, K & Wong-Villacres, M 2015, 'Supporting the Assessment of Collaborative Design Activities in Multi-tabletop Classrooms', 2015 ASIA-PACIFIC CONFERENCE ON COMPUTER-AIDED SYSTEM ENGINEERING - APCASE 2015, 2015 Asia-Pacific Conference on Computer Aided System Engineering, IEEE, Quito, ECUADOR, pp. 270-275.View/Download from: Publisher's site
Wong-Villacres, M, Chiluiza, K, Ortiz, M & Echeverria, V 2015, 'A tabletop system to promote argumentation in computer science students', Proceedings of the 2015 ACM International Conference on Interactive Tabletops and Surfaces, ITS 2015, pp. 325-330.View/Download from: Publisher's site
© Copyright 2015 ACM. This study explores the design of a tabletop system that seeks to bolster the argumentative skills of Computer Science students. A set of four design guidelines - positive interdependence, stages, interference, and awareness - were derived from user research and used for designing and prototyping a multi-display tabletop application. Four students evaluated a video prototype; the overall results showed that the application's features have great potential to support the design guidelines. Moreover, students' impressions about the prototype's enforcement of positive interdependence indicate possibilities for augmenting argumentation opportunities. Steps for future work are presented.
Echeverría, V, Avendaño, A, Chiluiza, K, Vásquez, A & Ochoa, X 2014, 'Presentation skills estimation based on video and kinect data analysis', MLA 2014 - Proceedings of the 2014 ACM Multimodal Learning Analytics Workshop and Grand Challenge, Co-located with ICMI 2014, pp. 53-60.View/Download from: Publisher's site
© 2014 ACM. This paper identifies, by means of video and Kinect data, a set of predictors that estimate the presentation skills of 448 individual students. Two evaluation criteria were predicted: eye contact and posture and body language. Machine-learning evaluations resulted in models that predicted the perfor- mance level (good or poor) of the presenters with 68% and 63% of correctly classified instances, for eye contact and postures and body language criteria, respectively. Furthermore, the results suggest that certain features, such as arms movement and smoothness, provide high significance on predicting the level of development for presentation skills. The paper finishes with conclusions and related ideas for future work.
Echeverría, V, Gomez, JC & Moens, MF 2013, 'Automatic labeling of forums using Bloom's taxonomy', Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 517-528.View/Download from: Publisher's site
The labeling of discussion forums using the cognitive levels of Bloom's taxonomy is a time-consuming and very expensive task due to the big amount of information that needs to be labeled and the need of an expert in the educational field for applying the taxonomy according to the messages of the forums. In this paper we present a framework in order to automatically label messages from discussion forums using the categories of Bloom's taxonomy. Several models were created using three kind of machine learning approaches: linear, Rule-Based and combined classifiers. The models are evaluated using the accuracy, the F1-measure and the area under the ROC curve. Additionally, a statistical significance of the results is performed using a McNemar test in order to validate them. The results show that the combination of a linear classifier with a Rule-Based classifier yields very good and promising results for this difficult task. © Springer-Verlag 2013.