EBook Interaction Logs as a Tool in Predicting Learner Performance in Reading

The present study assessed the effectiveness of ebook interaction logs in the reading performance of Saudi EFL learners. It employed an experimental research design to a total of 30 male students at Qassim University, Saudi Arabia, spanning twelve weeks. We used the BookRoll system and asked the EFL teachers to upload textbook materials, allowing learners free access anytime, anywhere. Apart from availability, what set the system apart from classroom teaching was the kind of privacy and individual learning pace that the system provides to learners: They managed to revisit relevant sections marked by the teacher, checked out the notes and engaged (even repeatedly) in other reading-related activities that the teacher incorporated into the ebook. The study analysed the system logs to develop prevalent reading patterns and predict reading behaviours in the participants. It shed light on the strategies that learners employed in reading the texts. The result showed that BookRoll could be used to predict learners ’ reading behaviours with a reasonable degree of accuracy which would be beneficial to a large number of interested parties including, but not restricted to, educational advisors keen to integrate technology into the EFL classroom in the Kingdom of Saudi Arabia, the teaching community which, for the most part, ignores reading as a skill by resorting to quick summaries in students ’ mother tongue at times and learners who need to find their individual learning pace and environment.


Introduction
Information technology has long been used to perform basic classroom functions such as recording of attendance and conducting of tests, not to mention the use of tablets and laptops for real-time note-taking.For the past two decades, Electronic Books (ebooks) have also come to be used, by both students and the general public, for various specialised and general reading purposes.Electronic books have many names in the literature, such as ebook, ebook reader, ebook device and digital book.Ebooks are a great tool available for teachers to monitor students' performance.Also, from the perspective of content publishers, ebooks are convenient to update (Yin et al. 286;Yin et al. 31).In a more specific context, for pedagogical purposes, ebooks with individual interaction logs have been introduced, and are great tools to engage students in learner groups with varied abilities, a phenomenon very common in EFL courses in Saudi Arabia, and to keep tabs on learner performance as they can predict early how each learner is likely to perform in achievement tests.As a teaching tool, it gives the teacher the added advantage of "catching them early" as logs and their analysis indicate the students' individual performance.Digital textbooks record individual reading behaviours apart, but not isolated, from students' learning activities for present and future reference by both the teacher and the learner.
According to Al-Ahdal and Al-Awaid, language teachers face various challenges, such as adopting new teaching strategies, aligning with new theories, and keeping up with new tools and technology (759).In the case of English Language Teaching (ELT), teachers must be focused on developing an environment conducive to the enhancement of students' learning.With the incorporation of technology in the education sector, many teachers have started using tools and technology in the learning environment to improve teaching and learning (Magulod).In this study, the ebook as a component of Information and Communications Technology has been used and gauged as to its potential efficacy in teaching ELT.It is also essential that students are regularly monitored and corrected based on their performance, which would help them to grow and improve their language.
Data is believed to be the most crucial driver of growth of countries in the future and it entails wide use of technology in all spheres of life, especially education.As such, the Saudi Data and Artificial Intelligence Authority launched its brand identity under the theme "Data is the Oil of the 21st Century" on March 4, 2020.The stated emphasis of the organisation is on the importance of data in the twenty-first century and introduces Saudi Vision 2030 as a roadmap to transform the Kingdom into an outstanding data-driven economy.
As an active language skill, reading does not generally figure highly on the "to-do" list of EFL teachers and, by the same token, learners in the Kingdom of Saudi Arabia (KSA).There are many reasons for this including poor learner proficiency, constraints of time and syllabi on the teachers' part, demotivated learners (as reading is seen as insignificant, or less significant, to their main and coveted aim of being able to speak fluent English) and the painfully slow and laborious nature of the activity.Yet, it is a skill that remarkably enriches learners' vocabulary and grammatical structures and is, thus, indispensable.One major impediment in helping poor readers is the very identification, or absence thereof, of the fact that reading is not actively taken up as an in-class activity.This study believes that using ebooks can help propel teaching and learning in this direction to a large extent, as they not only provide teachers with data on students' poor reading performance but also improve the reading strategies of good readers and help them strengthen their skills.
The present study assessed the effectiveness of ebook interaction logs in the reading performance of Saudi EFL learners.It aims to help language teachers to gain a better insight into language learners' reading behaviours, which is not practical and possible in a face-to-face classroom, especially with larger classes.It further aims to identify recurrent patterns in learner behaviour, for instance, the frequency of students' online visits to check teacher's notes in lessons, comprehension strategies frequently used and the ability to recall and reproduce information when the experience is e-based.Another objective is to compare assessment scores on reading tests in an e-based reading environment where reading can be done at learners' own pace.This is unlike the normal classroom where learners need to be urged to read aloud, answer questions related to what they read and generally face the daunting task of being watched and heard by their peers.This can be an uncomfortable and unmotivating experience, especially when they are struggling with the answers.

Research Questions
The present study attempts to answer the following questions: 1. Can learners' performances on reading be predicted via the use of the Bookroll, which electronically records their reading behaviours? 2. What are the attributes of the reading behaviours of good and poor readers in an e-based reading environment?3. How can pedagogical modifications improve learners' reading output in an e-based reading environment?
The BookRoll: Digital Learning Material Reading System Motivation is a key factor in improving students' learning.Second language learners of English, in particular, face the added challenge of needing to understand a foreign language that is lexically and, more importantly, structurally different from their mother tongue.To mitigate the issue of the English language as a barrier for learning and a cause for students' poor learning in the classroom, this study introduces the BookRoll System or ebook Interaction Logs as a new tool for enhancing learning and predicting performance.The ebook interaction logs allow students to mark, write a memo, re-read and bookmark the content as specific study strategies, at their own pace.Digital systems not only serve as distribution platforms for digital content and course material, but are also useful for learners as they enable them to access the study material from anywhere anytime over personal devices such as their computers and smartphones.The Learning Management System (LMS) is one of the most common digital systems that can be extremely beneficial to both teachers and learners.LMS also generates Big Data of the behaviour and pattern of each learner's interaction with the LMS as an event log.Big Data refers to large data sets, which can be computerised to expose trends, patterns and relations, particularly on human behaviour and interactions (Sivarajah et al. 263).The analysis of these event logs would generate valuable insight into both the personalised behaviour of individual learners and the behaviour of the community of learners and may be useful to all the stakeholders involved, including students, teachers and policymakers, for future improvements in teaching and learning.
The BookRoll digital learning material reader used in the present study allows teachers to upload the learning material in the PDF format for students to access later via a web browser in their devices.The user interface of the BookRoll has many functions for easy accessibility, including: Jump page: This feature allows the learner to move to the previous page, the next page or to any intended or random page.Bookmark: This feature allows a user to bookmark a page for ready reference at any future time.Marker: This is a handy feature of the BookRoll as it allows the user to highlight a section that has not been understood in yellow and any part of the page in red to indicate its importance.Memo: It is possible to create a memo on any page as needed by the learner.A memo can also be attached to any section of a page with the marker.Search: As in any digital systems, this interface helps learners to search for information quickly and easily in the data.

Literature Review
In the past decade, a great deal of interest has been shown by researchers to analyse educational data generated by digital learning materials.This interest in the analysis of educational data has led to a new area and era of research, now called Learning Analytics, which is likely to play a significant role in teaching and learning in future (Chatti et al. 10).One of the main functions of digital learning systems is to provide data on Learning Analytics, to be used by policymakers and educators.Learning Analytics aid in understanding and enhancing students' learning behaviour based on the data generated by the digital learning systems (Baker and Yacef 3;Yin et al. 319).Ebooks, in combination with learning platforms such as the learning management systems, are capable of generating considerable amounts of data on different aspects of students' learning behaviour.As Verbert et al point out, the digitisation of the learning environment is taking place at an increasing rate, and learning management systems collect data on every interaction of the learner with the content as "event logs" (133).Closely related to Big Data is the idea of educational data mining (EDM).In EDM, the aim is to understand Big Data with the help of specialised mining techniques, and in turn, to understand and provide solutions to issues related to Learning Analytics and digital learning (Romero and Ventura 601).
According to Daniel and Woody (18), and Junco and Clem (54), students' academic performance can be predicted by the use of ebooks and the event logs that are generated.Learning Analytics is useful because it may not only study the performance and behaviour of a learner at a given time, but also predict and assign a probability to the performance of a learner in the future.Learning Analytics can predict the chances of passing an examination by any particular learner based on data generated on their interaction pattern with the LMS.It also helps educators to identify learning stages and channel their efforts in the direction that will result in maximum benefits and learning for their students.Many researchers believe that Learning Analytics not only tracks and predicts learners' performance, but also diagnoses the issues that affect their performance (Troussas et al 16;Karthikeyan and Palaniappan 935;Acharya and Sinha 37).
Researchers at Purdue University in the USA experimented with an early digital system, called The Signal Project, which is often cited as one of the most successful examples of early, pioneering digital systems in the literature (Arnold 33).In The Signal Project, teachers were provided with a "signal" plug-in as a feature in their online system.Two hundred and twenty students participated in the pilot scheme and based on their performance in the first week, 45 students were placed in the "red" signal, while 32 were put in the "yellow."Each student was assigned a risk indicator based on the analysis of their interactions.The risk indicator was structured on the pattern of traffic lights, to provide them with feedback on their performance.A "red" light sign would mean a high probability of failure, "yellow" a medium likelihood of failure and "green" a high chance of success.The Signal Project showed a significant improvement in the performance of the students who participated in its pilot scheme.In the subsequent weeks, the number of students in the red category declined considerably as 55% of the students progressed to the yellow group and 24.4% moved up to the green group, thus leaving only 10.6% in the red group.Similarly, 69% of the students in the yellow group progressed to the green group.
An essential aspect of learning which the application of Learning Analytics could help unveil is hidden or unobserved learning behaviours among learners .Understanding unobserved behaviour would lead to significant improvement in learning outcomes.Lopez et al. used the Moodle LMS, an open source online learning to analyse the correlation between the interactions in learners' discussion and their academic performance.With the Naive Bayes algorithm, the performance of the students could be predicted with 89.4% accuracy.In this study, the Moodle LMS and its discussion forum were proven to be reliable tools to predict students' academic performance.Romero and Ventura also carried out a study using the Moodle LMS to predict students' academic performance.They used different algorithms, and in their research the accuracy of different algorithms varied from 60 to 70 per cent.The analysis attributed the poor accuracy percentage to the relatively low use of LMS.
In an attempt to build a prediction model, Howard et al. conducted a study with the aid of data generated by undergraduate students to identify at-risk students (those dropping out or failing) in large classes and online courses.The study mainly focused on the identification of the optimal time to apply an early warning system in any academic courses to "detect" poor performance and dropouts among students.The authors concluded that the second half of the classroom session was the appropriate time for any predictions to be made on an early warning system with a fair amount of accuracy.The method outlined in the study discussed how to develop an accurate prediction model for an early warning system for a course and how to recognise an optimal time to provide students with corrective measures during a session.Consequently, this knowledge would allow teachers to design appropriate intervention to help students improve before it is too late.
Despite the varied benefits associated with Learning Analytics, there are specific dilemmas and ethical issues to be considered before its application.According to Slade and Prinsloo, ethical considerations, e.g., taking consent, deidentification and opting out of the procedure, are concerns when applying Learning Analytics in the education system (1510).Their research found that students tended to opt out of the process when their performance was not monitored.It is also essential that, before starting its application, teachers try to understand Learning Analytics and the concepts associated with it.

Ebooks
An ebook can be defined as a text for reading material in a digital form, a complete book in the format of a computer file or a collection of images and words in an electronic format readable on computers, laptops, tablets, smartphone or specialised electronic devices such as ereaders (Lam et al. 509).Rao considers the advent of the ebook as the single most significant development in the history of the publishing industry since the establishment of the Gutenberg Press in 1450 (85).Compared with print books, ebooks are convenient, accessible anywhere and affordable, thus offering an excellent alternative to print textbooks for many stakeholders (Baker and Yacef 3).For students, ebooks have the potential to enhance their learning, thanks to their various functions such as information recall, search, copy-paste, highlighting and bookmarking.Ebooks also facilitate interaction among students, contact with teachers and prompt feedback from teachers (Chickering and Ehrmann 3).

The BookRoll System
According to Duke et al., theories develop with changing learning practices.One such theory is connectivism, which focuses on understanding learning (4).The main emphasis of this theory is on learners, who connect to the learning community and also feed information to the community.In connectivism, knowledge is believed to be stored and distributed across a network of data stored in the digital format.At the same time, the digital information network can be accessed by individuals to learn and enhance their knowledge (Duke et al. 4).The BookRoll System can also be considered a connectivist system in that it allows teachers and supervisors to upload educational material online, thus connecting them with the students and their peers and providing them with a learning space conducive to their overall learning.The teacher keeps an online record of all attendance and students' classwork.Since it is a system, it can receive user input, process data and, with the processed data, create information for storage and/or analysis.
In considering cognitivism as a learning theory, it is the learning processes, rather than the observed students' behaviour, that play an essential role in learning.The learning environment can also play a significant role in facilitating, or obstructing, learning (Ertmer and Newby 50).Within this learning environment, the practices done by the students and the corrective feedback given by the teacher are also crucial and determining factors in the process.Ertmer and Newby hold that the main focus of the cognitive approach in pedagogy should be on encouraging students to test, experiment with and pick different learning strategies that can help them improve their overall learning.In the case of the BookRoll System, a private, but not isolated, learning environment is provided to the student, wherein s/he can read the material, allocating as much time and effort needed for learning.Also, students' activities and patterns can be analysed, thus enabling the teacher to help enhance their learning and, in the case of students with low grades in achievement tests, provide them with corrective feedback and newer strategies.A graphical depiction of the BookRoll System is presented below in figure 1.

Methodology
Educational institutions in Saudi Arabia, in general, tend to take a lenient and "forgiving" approach to weaker performing students.Consequently, the problem of dropping out, which higher education organisations in other parts of the world might deal with as a serious issue, is either ruled out or kept at a bare minimum in KSA.The ultimate aim of any higher education organisation is to eliminate dropouts or keep them reasonable checked, and the Bookroll and similar elogs would be helpful in the early identification of poor performers.
The study employed the pre-test and post-test quasi-experimental research design to assess the effectiveness of the ebook interaction logs in predicting learners' performance in reading.This method is considered appropriate to examine the effects of innovation in an educational setting and considered as a standard method in educational research (Bloomfield and Fischer 27)

Searc h
The outcome is enhanced students' learning within their private space, along with new learning strategies main concerns of policymakers in higher education in Saudi Arabia is to improve learner output by providing extraneous academic support, and it is for this purpose that the use and testing of Bookroll logs was conducted.Learners' performance on a standardised reading test (one of the many held by the university as part of the assessment system) constituted the starting point of the study and the behaviour of learners, identified as "poor readers" following the administration of the ebook intervention, was analysed to determine performance trends and learning strategies.

The Population of the Study
The current study was conducted on 30 male students majoring in the English Language at Qassim University.The learners, all in the age group of 21-22 years with an equal number of years of training in EFL, were well versed with the use of ebooks, whether on their ereaders, such as Kindle, or the ereaders available on their smart devices.Even so, they were all provided with adequate training in the use of the BookRoll digital reading system and given individual access to the system, which meant instant access to the system by the learners anytime from anywhere and on any device.Three lessons in three different genres (a poem, a factual description and a biographical note) were uploaded in the LMS by the researcher in PDF format.In addition, notes prepared by the teacher were also uploaded as support materials and the learners were informed that they were free to read up on the texts on other websites.

Data Collection
Questionnaires and manual methods of data collection are popular methods of data collection but are deemed highly subjective in that the data are collected through criteria selected by the researcher.In this study, data were collected automatically, as the the learner's interactive behaviour is simultaneously recorded while s/he interacts with the digital reading system.Thus, the procedure was more objective, which renders it more reliable and authentic.The BookRoll digital reading system is used to collect simultaneous data during the participants' interaction with the system.The data used were for weeks 3, 9, 12 to cover the initial, median and concluding phases of learning via the BookRoll.

Data Analysis and Results
Using inferential statistics, a correlation was sought between the number of times the BookRoll was accessed in a week, the average time spent on the BookRoll in a week and the number of times students visited the teacher's notes and the online assessment exercises.While a strong correlation at 0.735 using the Pearson's r was found between the number of times the teacher's notes were accessed and the assessment score, a slightly lower but positive correlation at 0.692 was found between the number of times the BookRoll was used in a week and learners' output on the three reading assessments.In other words, poor readers tended to use the BookRoll and teacher's notes less frequently than good readers, where a strong positive correlation was established on both of the parameters (i.e., the average number of times the BookRoll was used and the reference to the teacher's notes) with a correlation of 0.821 and 0.787, respectively.Similarly, a positive relationship was found between the frequency of mark-up or note-taking by the learners and their performance in online assessment exercises.For poor readers, this stood at 0.573 and for good readers, it stood at 0.619, indicative of their respective reading performances.Table 1 which summarises the data retrieved from the elog shows nine indicators.
It can be seen here that the average sessions attended per student in a week is seven with an average time of 22 minutes in a week spent using the elog.Consequently, ten minutes was the average time spent on reading the teacher's notes.There was also an average frequency score of seven on the sequential recall events using e-assignment.They also showed an average of eight out of fifteen referential questions.An average of ten e-write ups were produced out of 15 text summaries.Similarly, on the post-activity assessment score out of 50, the learners showed an average rating of 25.Finally, the pre-activity assessment score, out of 50, showed an average score of 24.
We took the rating of 50 per cent as the baseline to identify poor readers.The data revealed that 12 of the participants consistently scored poorly both on the online assessment and pre-activity university examination, with average scores hovering at 11.5 (23%) in the university examination and a slightly higher score of 12 (24%) in the online assessments.The learners visited the ebook five or fewer number of times a week and spent as little as an average of 3.5 minutes a week on reading the text.The average time devoted to referring to the teacher's notes was also low, at 6 minutes per week.The texts used carried detailed annotations in addition to the teacher's notes, and these annotations were all in English, unlike classroom sessions where teachers tended to translate the text to students' mother tongue to save time.The average number of note-taking or mark-up by poor readers was as low as 1.3 times per week.
For good readers, the average score in the university examination stood at 32.7 (65%), and in the online assessment, it was 32.9 (65.4%).These learners spent an average of 32.2 minutes a week on reading the BookRoll text, an average of 11 minutes a week on reading the teacher's notes and 8.9 minutes a week on marking up or note-taking of the text.Figure 1 below shows the comparative values between poor and good readers.In the group of good readers, the strategy that was most frequently used with a good amount of time spent too was reading the teacher's notes on the text.At least six of the good readers also conducted searches on the Internet for additional notes on the texts included in the BookRoll.

Poor reader
Good reader

Conclusions and Recommendations
The present study assessed the effectiveness of ebook interaction logs in the reading performance of Saudi EFL learners.The study answered the research questions with the following conclusions: RQ 1: The mean test scores for each of the two groups, viz., poor and good readers are 11.5 and 12 for the former and 32.7 and 32.9 for the latter.
The difference between the pre-test and post-test scores in the case of the poor readers is statistically insignificant at a p-value of 0.5.Similarly, the pvalue for the two assessment scores of the good readers is 0.49, also showing a statistically insignificant difference.In other words, the assessment scores, as obtained from the use of the BookRoll are close to those obtained via the traditional university-based reading tests.Thus, the BookRoll can be used to predict learners' reading behaviours with a reasonable degree of accuracy.
RQ2: Poor readers typically spend much less time on the BookRoll textual materials as well the teacher's support material than good readers.
RQ3: Good readers frequently referred to the teacher's notes, and as recorded by the elog, they did so simultaneously while reading the text.Further, they actively used the mark-up tool and took notes on whether to seek the teacher's clarification later or for future reference.
Reading is an ongoing skill and one which can benefit foreign language learners in the long run.It is, therefore, imperative that efforts be made to foster this skill and provide learners with the necessary tools to enhance reading.Pedagogically, greater intervention of ereaders is unavoidable as learners are best engaged in the learning process if it allows them access anytime and anywhere to read materials.However, uploading all textual content in PDF format can sometimes be boring and monotonous to the learners given the fact that with ereaders, they are wholly on their own.Backing up tools such as BookRoll with active WhatsApp collaborative learning groups or even a Facebook page may reduce the boredom.A system like BookRoll, however, can help significantly with timely interventional mechanisms in the case of poor readers, as learning behaviours even within short durations can be used as predictive tools for eventual success or failure.Accordingly, these may be used for better learning outcomes.
The BookRoll is just one of the many available elog-enabled readers and before recommending the use of any of these, more significant research needs to be carried out in the EFL environment.Another limitation of the current study was the total absence of a control group to verify the usefulness of BookRoll in enhancing reading skills.Other studies on the subject may keep this in view.Finally, future studies are needed with the inclusion of females and broader age groups and proficiency levels to fully understand the implications of the use of ereaders and e-logs.

Figure 1 :
Figure 1: The BookRoll System up or note making by a participant 6 Sequential recall of events (using e-assignments) out of 15 7 Precision of referential questions (based on e-activities) out of 15 8 Summarising the text (producing an e-write up) out of 20 10 Post-activity Assessment score out of 50=15+15+20 25 Pre-activity Assessment score out of 50 24

Figure 2 :
Figure 2: Comparative Scores of the BookRoll System

Table 1 Data Retrieved from the ELog
Table2shows the pre-test scores of the 30 learners (with pre-mean test score is 24).The majority of the learners scored 31-40 with the frequency of 13 (43%), followed by eight learners who scored 11-20 (27%), five learners scored at 21-30 (17%) and four learners obtained the least score from 0-10 (13%).The finding suggests that almost 45% of the learners were able to earn scores of 31-40 in the pre-test assessment.