Jock Boyd PhD Research Proposal Data Mining Methods to Study the Engagement of Students in E-Learning Environments

Statement of Topic:
Developing systems that can reliably assess how students use e-learning environments, (moodle, blackboard) and the attitudes and aspirations, which underlie these decisions. Colleges and Universities store vast amounts of data about students. Most e-learning systems allow access to this data via the learners log files. Data mining methods can give meaning to these data and provide valuable information for assessment not only of the student but also of the elearning environment. One area that is very important in the design and implementation of e-learning systems is learner engagement as it is this that will improve the quality of the e-learning environment and prevent student attrition rates.

Aims:
This research aims to design, develop and test, learners level of engagement and apply a framework to e-learning environments to improve their effectiveness and sustainability. Student disengagement can be predicted by monitoring the learners click data such as the pages they have read, quizzes taken and time spent in the learning environment. Prediction analysis techniques would in turn allow interventions with failing students at appropriate times and may potentially be used for automatic flagging of students in crisis.

Review of the literature:

Educational designers try to meet the learners wants and needs in order to enhance learning. However, most systems do not consider the learners engagement in the learning environment, despite its impact on learning being generally acknowledged and of research showing that lack of engagement is correlated with learning rate decrease (Barker 2004) Most e-learning environments try to engage students by the design of the elearning system, through attractive designs that incorporate multimedia materials or by including game dynamics (Connolly 2006) and have been shown to be successful in a number of cases (Chen 1998). In spite of these efforts, students are not always focused on learning and even try to play the systems Barker (2004) calls it gaming the system where they try to manipulate the learning environment by exploiting the system rather than by attempting to learn the material.

Click data files are used in commercial web-page design to assess customer engagement with the website, recently educational systems have been using this information gauge student motivation with their learning. An important advantage of click data files analysis is the unobtrusiveness of the assessment process. Several efforts to detect motivational aspects from learners actions are reported in the literature Vincent (2002), Barker (2004), Beck (2005), Arroyo (2005), Cocea (2007). However, these studies focus on intelligent tutoring systems where the students learning environments are much more constricted. On-line content-delivery systems such as moodle and blackboard are increasingly used in formal education; there is a need to extend this research to encompass this type of systems as well. The learner is free to interact with these systems freely, posing several difficulties to an automatic analysis of learners activity. This research will be restricted to identifying information from a learner use of the learning environment and how this affects their engagement and disengagement and will develop a system of prediction.

Methodology: The research will be conducted in Australian College and Universities moodles, in three phases. I will construct an evaluation model, based on the literature review, of how learners use e-learning systems based on their click log files. I will test the models capacity for prediction using simulation tools to model learners use of e-learner environments. I will refine the evaluation model and the simulation tools over time to create a prediction model.

Research Questions:
How do we evaluate the engagement of learners in an e-learning environment?

How well can an evaluation model predict the effectiveness of e-learning environments on learners engagement or disengagement?

References:
R. Baker, A. Corbett and K. Koedinger, Detecting Student Misuse of Intelligent Tutoring Systems, Proceedings of the Seventh International Conference on Intelligent Tutoring Systems, pp. 531540, 2004

T. Connolly and M. Stansfield, Using Games-Based eLearning Technologies in Overcoming Difficulties in Teaching Information Systems, Journal of Information Technology Education 5, 459476, 2006

G.D Chen, G.Y. Shen, K.L. Ou and B. Liu, Promoting motivation and eliminating disorientation for web based courses by a multi-user game, Proceedings of the EDMEDIA/ED-TELECOM 98 World Conference on Educational Multimedia and Hypermedia and World conference on Educational Telecommunications, June 20-25, Germany (1998)

A. de Vicente and H. Pain, Informing the Detection of the Students Motivational State: an empirical Study, In S.A. Cerri, G., Gouarderes and F. Paraguau (eds.) Intelligent Tutoring Systems, 6th International Conference, pp. 933943. Springer, Berlin, 2002

J. Beck, Engagement tracing: Using response times to model student disengagement. In C. Looi, G. McCalla, B. Bredeweg and J. Breuker (eds.) Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology, pp. 8895. IOS Press, Amsterdam, 2005

I. Arroyo and B.P. Woolf, Inferring learning and attitudes from a Bayesian Network of log file data. In C.K. Looi, G. McCalla, B. Bredeweg and J. Breuker (eds.) Artificial Intelligence in Education, Supporting Learning through Intelligent and Socially Informed Technology, pp. 3334. IOS Press, Amsterdam, 2005

M. Cocea and S. Weibelzahl, Eliciting Motivation Knowledge from Log Files towards Motivation Diagnosis for Adaptive Systems. In C. Conati, K. McCoy and G. Paliouras (eds.) User Modelling 2007. Proceedings of 11th International Conference, UM 2007, pp. 197-206, Springer, Berlin, 2007