In workplace based learning, e-portfolios are used more and more to keep track on the development of competencies. These e-portfolios contain the results of assessments at the workplace, reflections and other types of information. Although portfolios are a valuable source of information for the student and the supervisor, the amount of information can be overwhelming and hence be a cause that important signals are not recognized .
In the FP7-project WATCHME (2014-2017, www.project-watchme.eu) student models are developed that, on the basis of the e-portfolio content, can send out signals to the student or supervisor. These signals point to meta-cognitive problems such as a temporary dip in motivation. The developed models are based on pedagogical insights and theories, selected using expert interviews. They make use of scores from workplace assessments such as MiniCEX and OSATS and translate patterns in these scores to meta-cognitive concepts. Because a certain level of uncertainty is involved, the models are probabilistic . The signals that the models produce are displayed as messages within the e-portfolio.
These models have been applied by 7 Dutch and international partners (Germany, Hungary, Estonia, USA) in three domains: residence training medicine, veterinary medicine and teacher training. Sometimes students already had some experience with e-portfolios, at other places those are introduced during the project. Two rounds of evaluation have been held among 318 students and 50 supervisors, first formative to allow for improvements and then summative.
The project showed that it is possible to catch metacognitive processes into functioning probabilistic student models, and that workplace assessments are suitable as input for these models. The project also learned that the design of the interaction with the student and the supervisor is crucial: the selected shape and content of the messages appeared not always to be received as intended. Further it appeared that the diagnosis of metacognitive processes contained in the models need a longer period of data points than anticipated in the evaluation period of the project, which eventually caused less messages to be generated by the models than we would like.
The project showed that pedagogical student models can be applied in a complex en and changing environment such as workplace based learning. The technology we developed can also be used in other application areas in learning-analytics, but also in digital assessment, gaming and simulation.
Jeroen Donkers is assistant professor at the department of educational development and research in the Faculty of Health, Medicine and Life sciences of Maastricht University. He is trained in mathematics and computer science and wrote a Ph.D thesis in artificial intelligence on the subject of opponent modelling. Currently his research interest is in learning analytics, digital assessment and rich feedback.
The STELA Erasmus+ project, Successful Transition from secondary to higher Education using Learning Analytics (562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD) aims at exploring how learning analytics can be used to support the transition from secondary to higher education.
Within the project, KU Leuven focuses on the development of learning dashboards that provide actionable feedback within a traditional first-year bachelor program. The additional focus on scalability introduces the challenge to primarily focus on data that is or could easily be made available in any higher education institute.
So far, three learning dashboard were deployed within the first-year of twelve bachelor programs at KU Leuven. The first learning dashboard provides students feedback on their learning and study strategies based on a paper-and-pencil questionnaire. The second and third learning dashboard provides feedback after the first and second examination period. The student-facing dashboards have some characteristics in common:
- a visualization of the individual scores with respect to the scores of their peers in the program,
- a “predictive” part that shows the study progress of a similar norm group in previous years,
- specific tips and recommendations and referral to counseling services.
The learning dashboards have been well-received by students and staff. Next academic year the dashboard will be deployed at more than 20 programs within KU Leuven and one program at TU Delft. The decision to use easily available grade and survey data for the learning dashboards has resulted in high acceptance among students and staff. The attitude of students and staff toward learning analytics has evolved from skeptical to demanding more wider application. We therefore argue that such “low-level“ analytics creates essential buy-in and acceptance for later extension and further application of learning analytics. Additionally, the interaction with the learning dashboards themselves creates additional learning traces that can serve as input data for future learning analytics interventions.
Tom is a PhD candidate at the Faculty of Engineering Science, Tutorial Services & Department of Computer Science at KU Leuven, where he focuses on scalable learning analytics interventions.
Jorge Mañana-Rodríguez - Centre for Human and Social Sciences, Spanish National Research Council, Madrid, Spain
Elena Martín-Monje is a lecturer at UNED (Spain), where she teaches mainly in the areas of English for Specific Purposes and CALL (Computer-Assisted Language Learning), also her fields of research as a member of ATLAS (http://atlas.uned.es). Both her research and teaching practice have received official recognition, with a Prize for Doctoral Excellence at UNED and a University Excellence in Teaching Award. Author of numerous papers in national and international journals, two of her most prominent publications are Language MOOCs: Providing Learning, Transcending Boundaries (2014), which is the first published book on Language MOOCs, and the Routledge monographic volume Technology-Enhanced Language Learning for Specialized Domains (2016).
María Dolores Castrillo is a senior lecturer at UNED, in the areas of German Studies and CALL (Computer-Assisted Language Learning). Member of the ATLAS (Applying Technology to LAnguageS) group, her current research interests include Computer-Mediated Communication, e-learning, Mobile Assisted Language Learning (MALL), Open Educational Resources (OERs) and MOOCs (Massive Open Online Courses). Her publications include papers in indexed journals and book chapters both at national and international level. She has won two prizes related to Open Learning: Best Open CourseWare (2008) and best MOOC (2013), both awarded by the Spanish Ministry of Education. Her latest initiative is GLOBE (Group for Languages in Open and Blended Environments).
Jorge Mañana-Rodríguez, Ph.D. is researcher at the Spanish National Research Council, holding a degree in Education and a doctorate in Information Science and Library Science. He is specialised in Information Science and Data Analysis.
The ABLE Erasmus+ project, Achieving Benefits from LEarning analytics (2015-1-UK01-KA203-013767), explores how learning analytics can be used to support both staff and students in higher education. The partners, Nottingham Trent University (UK), KU Leuven (Belgium) and Universiteit Leiden (Netherlands) focus on the first-year experience.
The project team developed LISSA: the Learning dashboard for Insights and Support during Study Advice. LISSA aims at supporting and facilitating the communication between a study advisors and student. To this end, LISSA provides an at-a-glance overview of the “facts”:
- the obtained grades for the different test moments,
- the positioning of the individual grades with respect to the peers (histogram),
- the overall position with respect to the peers, and
- a “predictive” part that shows the number of years students with similar results needed to obtain the bachelor degree.
Depending on the time LISSA is used a planning module for resits or the next academic year is added.
LISSA was designed, developed, and evaluated in collaboration with study advisers. In the meanwhile LISSA is deployed in 12 programs at KU Leuven and will be deployed in about 20 programs at KU Leuven next year and in two programs at Universiteit Leiden.
Based on interviews, observations, and questionnaires we conclude that LISSA supports the adviser-student dialogue, helps study advisors to motivate students, triggers conversation, and provides the study advisors tools to add personalization, depth, and nuance to the advising session. It provides insights at a factual, interpretative, and reflective level and allows both adviser and student to take an active role during the session.
Besides presenting the results of LISSA we will elaborate on the process we followed, which resulted in maximal acceptance.
Katrien Verbert received the doctoral degree in computer science from KU Leuven, Belgium, in 2008. She is an assistant professor at the HCI Research Group, KU Leuven. Her research interests include learning analytics, visualization techniques, recommender systems for learning, and digital humanities. She has been involved in several European and Flemish projects on these topics, including the EU FP7 ROLE and STELLAR projects. She is also involved in the organization of several conferences and workshops (general chair EC-TEL 2017, program co-chair EC-TEL 2016, workshop co-chair EDM 2015, program co-chair LAK 2013, and program co-chair of the RecSysTEL workshop series).
PXL started a research project about learning analytics in 2016. The aim of the project is to develop a dashboard that teachers can use to keep track of their students, and to stimulate teachers to make use of learning analytics in their courses. There has been a pilot project in 2016, teacher discussions, and a student evaluation. This has led to a new Dashboard design and guidelines for course development. Teachers want to be able to identify which students are at risk, and want to be able to interact with them in an easy way. Tobe Baeyens will present the current state of this research project and where it is going next.
Tobe Baeyens is an e-learning advisor. He is part of a research project about learning analytics at PXL. He also works for Erasmushogeschool Brussel as an advisor in blended learning. Before he was teaching ICT in Summerhill School in the UK and in Lutgardiscollege in Brussels. He has a strong interest in self-directed learning and in educational technology.
Whereas French used to be the first second language in Flanders, it has gradually become the first foreign language. As a consequence, more and more university students whose curriculum includes a French course do not pass the exam. At KU Leuven, almost 2000 bachelor students (Faculty of Economics, Lax, Social Sciences …) may face this problem.
Recently, the French teaching staff of the Leuven Language Institute has built an online tool, PAZAPA (pas à pas: step by step) to help students address gaps in their basic knowledge of French grammar. Using learning analytics (LA), we managed to turn a traditional online environment combining tests and exercises into a powerful tool which not only makes it possible to identify at-risk students but also to work on each student’s specific language problems. In addition, students can compare levels and are encouraged to work on their grammar issues throughout the academic year.
LA not only allows teachers to monitor students’ progress. A thorough analysis of log files reveals specific grammatical pitfalls which can be tackled (again) in classroom training. It is even possible to generate individualized tests as exam preparation.
All these data are synthesized in various dashboards for teachers and students, showing all relevant information and actions to be undertaken at a glance.
Serge Verlinde is Professor of French for Specific Purposes (Business French, Legal French) and Director of the Leuven Language Institute. His main research interests are corpus linguistics, pedagogical lexicography and CALL (Computer-assisted language learning).
Jordi Heeren is a research associate and lecturer of Academic Dutch at the Faculty of Social Sciences. His main research interests are language testing, writing research and academic literacy skills.
Nathalie Nouwen is a lecturer of French for Specific Purposes at the Faculties of Law and Social Sciences.