 |
| Image by Gerd Altmann from Pixabay |
Learning analytics involves reviewing the Educational Data that is available and then taking appropriate action based on what you discover (Corrin, 2021; Da Silva et al., 2022; Lockyer et al., 2013; Wise & Vytasek, 2017).
Before implementing Learning Analytics with our (Chisholm's) Moodle data, we need to be clear on what questions we want answered and then what actions we might need to take. There is so much information that is generated with online learning that there needs to be a focus to the analysis, or you can easily lose track of what you are trying to achieve or learn.
What do I want to know about my students regarding their learning and interactions with the learning resources? How can we determine success? What actions do I want to take? What effect will this have on the design for our 2023 Moodles for the Certificate III in Business trainees?
Questions
- How long do students spend online, how often and what are they accessing?
- Predicting "At Risk" students?
- Do the students interact with each other and/or their trainer?
- Have the students enhanced their knowledge in the subject?
To determine the Data and Reports available, I will be using Moodle data from the unit BSBTWK502 Manage Team effectiveness. This unit was delivered during 2022 in our Diploma of Business course.
1. How long do students spend online and what activities?
Moodle has some built in reports that you can view on screen or you can download the actual logs into a CSV or Excel file. The Course Administration page allows a reviewer to access a variety of reports that can provide information for each student or as a summary.
Summary Reports
 |
| Moodle Course Administration Page - Reports |
 |
| Activity Report Summary for the Unit 1 |
 |
| Activity Report Summary for the Unit 2 |
Recent Activity Reports
This can provide information on what activities students are more likely to use.
 |
| Recent Activity Report |
The Recent Activity report allows the reviewer to select any particular part of the unit Moodle and choose what activities they want the report to focus on. Selection of dates can be used for different groups or classes or to focus on a timeframe within the course.
The resulting list can be viewed or used to send messages to students, for example those that may not have accessed the course for a designated amount of time.
Full activity Logs
This report also allows you to filter the results. The screen below shows that you can select the course, the group, individual students, or individual activities. For example, you may need to find out what a particular student has completed on the Moodle site, how often they log in and when, and the last activity for that student.
As well as being able to create a full activity log, the filter screen below also allows you to select a particular group, participant or activity.
 |
| This filter screen allows you to choose the settings for the Activity Logs |
The Pivot Table example below is for a particular group and is based on number of views of the assessment guide. The Assessment Guide details what the students need to do to successfully complete the unit.
 |
| Demonstration of spreadsheet pivot table for access to the Assessment Guide |
Using the spreadsheet of ALL activity for ALL students in a particular group, I created a Pivot Table that summarises the activity by student.
 |
| Pivot Table summarises the activity by student |
 |
| Student views on Moodle Announcements |
 |
| Student Views on the Assessment Guide PDF |
Conclusion
Using student logs in Moodle I can review the activity by student and activity type. I can see their last date of activity as well as being able to summarise the data using Excel. While this isn't ideal, and is very labour intensive, it does provide the information to determine how long students spend online, when are they most likely to login, and what resources and activities they are accessing. This is valuable information that can indicate the level of student engagement with the resources (Lockyer et al., 2013; Gibson et al., 2014) as well as overall participation at the course level (Gibson et al., 2014). Having the Moodle data summarised and presented in a readable manner easily would be a huge advantage and save lots of time. This information can be used to evaluate the course activity design and sequencing, and make adjustments to the course design as required.
There is enough information within the Moodle Log information to answer Question 1 - How long do students spend online and what activities?
2. Predicting "At Risk" students.
Predicting students that may be at risk of not completing the course is often the main reason for the use of Learning Analytics, but focuses on those failing rather than providing information that can benefit all students (Gibson et al., 2014).
Activity logs generated through an LMS provides a teacher with real-time data that can be used to gauge activity and make judgements. Using the data provided by Moodle activity logs, analysis can highlight whether a particular student is engaging with the resources and submitting assessments on time. If a student is falling behind, an email or message can be sent to remind the student about upcoming tasks or to see if there are any issues that can be resolved with some form of intervention (da Silva et al., 2022; Ifenthaler et al., 2021; Jawthari & Stoffa, 2022; Lawson et al., 2016). These predictions are only as good as the data that is obtained. The more data, the better the ability to predict success is. Jawthari & Stoffa (2022) recommend adding regular activities to the online component to increase the amount of good quality data.
The aspects of student engagement that can be reviewed to determine students at risk include:
- General activity - how often do they engage with the site?
- When was the last time the student accessed the site?
- What are they doing within the Moodle - what activities/resources are they accessing?
- Have they viewed the Assessment Guide?
- Are they up to date with quizzes or assessment uploads?
- Are they successfully completing any graded activities?
Conclusion
The Moodle activity logs will provide all of the above information, but care needs to be taken to ensure that "at risk" students are being identified accurately and that no prejudices are taking place by using engagement data rather than assessment results (Jawthari & Stoffa, 2022). Students work at a varying pace with some that will work consistently throughout the unit timeframe, whereas others prefer to do more in depth work towards the end of the unit.
There is enough information within the Moodle Log information to answer Question 2 - Predicting "at risk" students. "At risk" students are those that, based on past data trends, may drop out of the course. The disclaimer for this is that a student could look at and review a number of activities regularly - does this mean that they are NOT at risk of dropping out? Students that are more engaged in the course, are more likely to successfully complete the course (Jawthari & Stoffa, 2022).
Analysing the data, even with tools such as Microsoft Excel, is very time consuming. There are a number of add-ins for Moodle that could automate this task and produce visual representations of the data that can be reviewed easily without too much added expertise (Vitkare, 2022). If this could automatically be produced weekly with results available for teaching staff and any relevant support staff such as our disability or LLN support staff.
3. Do the students interact with each other and/or their trainer/instructor?
In our current Moodles, there is really not enough information to determine this. There is an Announcements forum as well as a Discussion Forum. There are also forums within each chapter of the resources on Moodle, but most students don't see them or engage as they have the option to download the resource as a single PDF.
Based on the data and reports from the BSBTWK502 Moodle, there were no posts in the general discussion that took place on the Moodle and only one student contributed to a chapter forum. The chapter "books" are SCORM activities, that were not widely used. Most students just downloaded the PDF Learner Resource document.
Conclusion
Currently, there is not enough data to answer this question.
How can we get more data?
Moore (1989) proposed that there were three types of interactions that can be incorporated into online or distance learning. He identifies them as firstly learner-content interaction, followed by learner-instructor interaction and finally learner-learner interaction. Learner-content interaction is catered for by providing resources that students can view, but learner interaction with their instructor(s) and with other learners needs to be included in the learning design. This can be achieved by incorporating forums, discussions or other social interaction tools into the design of the course. They need to be an integral component of the course design and students encouraged to contribute. They could be included as a formative assessment, or could be an area where students can share ideas or things that they may have learnt, or build up a portfolio. The more activities that are placed in the LMS, the more data that will be provided (da Silva et al., 2022; Jawthari & Stoffa, 2022; Lockyer et al., 2013)
Other Moodle tools such as OU (Open University) Blogs, which has been included as a plugin on Chisholm's Moodle instance, provides a place where students can collaborate with each other, in groups, or privately with their instructor. There are some other added benefits provided by the OU Blogs, including making them public, adding tags, allowing comments, and students can change the name of their Blog to personalise it (Bird, 2022). This is an area worth exploring.
There is much discussion on how Learning Analytics can be used to enhance and guide the course design framework (Lockyer et al., 2013; West et al., 2020a; Wilkinson et al., 2019). This is a good example of how Learning Analytics can influence the design and structure of a course or unit.
4. Have the students enhanced their knowledge in the subject?
The only assessment of knowledge is in a report and an observation assessment, both undertaken towards the end of the learning process. Once a student has been graded, Moodle is updated with the result along with feedback, which can then be viewed by the student. This determines whether a student has gained competency in the unit, but there is no record of knowledge gained along the way.
Conclusion
To determine the level of learning or knowledge gained while working through the resources, use of discussion and/or activity forums could provide some useful information along with multiple quizzes.. For example, compulsory or graded forum discussion posts could be scanned for keywords or concepts that will demonstrate some level of understanding of the course material.
Inclusion of quizzes, forums and discussions need to be considered at the Learning Design stage and mapped to the unit accordingly (Lockyer et al., 2013; West et al., 2020a; Wilkinson et al., 2019; Wise & Vytasek, 2017).
