Ontological scenario model for learners with disabilities in a recommender framework based on assessment analytics

Abstract

Technology Enhanced Learning Environments (TELE) have attracted many learners to acquire knowledge and skills at their own pace. But the majority of these environments are not accessible for all categories of learners including learners with disabilities. In fact, some environments may provide content during the learning process that does not meet the profiles of every type of disability. Much research has been developed in the area of personalizing e-learning for people with disabilities. The use of assessment analytics, on the other hand, remains largely unexploited despite its great informative potential, which is elated to assessment data generated by online learning environment. Our proposal focuses on the design of a scenario model for Assessment Analytics to develop a recommendation framework for learners with disabilities. This framework is conceived to retrieve and select relevant learning and assessment resources to learners with disabilities based on their preferences, accessibility needs, and assessment trace data in the context of online learning.

Keywords- Assessment analytics; Recommender System; Disability.

1. Introduction

The term “disability” refers to any type of limitation that results from the interaction between a person with health problems and the environment [1]. Limitations can influence the quality of life of people with disabilities. This reflects the inability to access education. E-learning devices are a solution to include people with disabilities in education. These devices must be accessible and respond to all profiles of students with disabilities. The accessibility can be defined as the ability of the learning environment to adjust to the needs of all learners and is determined by the flexibility of the environment [2]. Our main objective is to select learning resources appropriate to various profiles of students with disabilities. To achieve this objective, we focus on recommendation techniques. Indeed, there are several recommendation techniques that are designed to provide relevant resources to a learner using certain information about the users and resources.  According to [3] recommender systems in TELE must take into account features that are specific to the learning context. These characteristics are: learning goal, prerequisite knowledge, learner characteristics and preferences, learner grouping, learning resources, learning path and learning strategies. Much research has been developed in the area of personalizing e-learning for people with disabilities. Categories of systems developed include Virtual Learning Environment (VLE), Computer Assisted Instruction (CAI) and Intelligent Tutoring System (ITS). Many research works focused on the importance of accessibility in eLearning environments to provide a digital and inclusive education. Learning Environments should include components that are accessible and properly and universally designed to scaffold and enable each and every learner to learn effectively considering her/his abilities, disabilities, and individual learning preferences [4]. This work falls under the heading of learning analytics or educational data mining. The use of assessment analytics, on the other hand, remains largely unexploited despite its great informative potential. Assessment analytics is the analysis of assessment data generated by the online learning environment to improve the field of learning analytics. The added value of this research work is to focus on assessment analytics to develop a Recommendation Framework for learners with disabilities. To tackle our objective, the main research questions are:

• How can e-learning and  e-assessment data be used to recommend educational resources?
• Which learner model covers the accessibility needs of each learner?
• What recommendation model based on the e-assessment analytics could be proposed for an accessible  and personalized learning environment?

This paper is organized as follows: In section 2, we describe the Background and Related Work. A Recommendation Scenario Model is presented in Section 3. Section 4 provides the Proposed Recommendation Framework. In section 5, we describe the algorithms; and finally section 7 brings the conclusion.

2. Related Work

The aim of learning personalization is to provide a learner with online learning resources that are relevant to his characteristics. It is therefore needed to  have recommender system capable of providing personalized recommendations or guiding the user to interesting or useful resources within a large data space [5]. E-learning platforms used recommender systems to “recommend relevant learning materials to learners and help them make decisions”[6]. There are mainly three approaches recommended in the field of e-learning research: content-based, collaborative-based, and hybrid-based. The majority of work related to recommendations in e-learning “focus on these conventional recommendation techniques” [7]. Content-filtering is one of the oldest recommendation approaches; it builds the user profile according to the characteristics of selected or preferred objects [8]. Content-based recommenders could be defined as systems that recommend according to the attributes or descriptors or characteristics or properties or even variables that represent the recommended items. Collaborative-based recommendation systems try to recommend items similar to those a given user has liked in the past, whereas collaborative filtering recommenders are based on the principle that users with the same personal features will probably like the same items. This approach is based on the tracking of other users’ interactions with the system. The Knowledge-based approach matches the user’s needs with the user’s characteristics, based on items suggestion through logical inferences about the user’s preferences and needs [9].  Using analytics, would enable to provide an accurate personalization process. Learning analytics is about measuring, collecting, analyzing, and reporting on data from learners in learning contexts for the purpose of understanding and optimizing learning and the environments in which they take place.

In our research work we focus mainly on assessment analytics, where data related to assessment is considered. The majority of the review research papers focus on Learning analytics in considering a personalization or a recommendation process

In our research work our target learning group is composed of learners with disabilities. It is therefore important to consider digital accessibility.

In this section, we discuss research work related to our research topic. We first focus on recommendation/personalization approaches. In [10] the authors propose an intelligent recommendation system for an online learning environment (EST-in-Line) to provide personalized courses and guide students to choose the most appropriate courses to their profiles. The recommendation technique used in this system is based on association rules. The authors do not address disability and do not exploit assessment analytics. The disability profile includes social, cognitive, and a disability classification model. An ontology-based personalization approach applied in an online learning environment for students with disabilities in higher education is proposed in also [11]. These two works do not address the analytics of assessment for personalization generation. The authors in [12] also proposed a recommendation technique by combining collaborative filtering and ontology to recommend personalized e-learning material to learners by considering learner characteristics. The learning materials are filtered according to the preconditions of the learner’s request and the learner’s knowledge. A mechanism for personalized search and recommendation of educational objects has been proposed in [13]. The approach is based on an existing trace-based system, and proposes an implementation of personalization services within the ARIADNE tool. The new features implemented concern the filtering of educational objects with respect to the user’s preferred language and file format, as well as with respect to the resources consulted by the user. In [14] the authors proposed a hybrid and context-sensitive recommendation approach for museum visits. This approach combines three different methods: demographic, semantic and collaborative, each method being adapted to a specific stage of the visit. A recommendation system based on hybrid filtering of semantic information in communities of practice for e-learning (CoPE) is proposed in [15]. Collaborative filtering has been treated in [16], where authors proposed a recommender system using collaborative filtering of online learning resources. The proposed model contains five main components, namely, learner ontology, learning resource ontology, recommendation engine, Decision Algorithm (DA), and final recommendation component. An ontology-based personalized recommendation system for e-learning to recommend appropriate learning content to learners using collaborative filtering was also proposed [17]. OERs (Open Educational Resources) have been also used in [18] to design a customized OER recommendation approach that takes into account learners’ skills, occupations, and accessibility properties to find the most appropriate and high quality OERs. In [19] a novel open ICT accessibility and inclusive design ICT-AID competency framework with a view to support individuals, and education and training institutions on delimiting the required relevant ICT accessibility competencies has been proposed. None of these works has provided a Recommendation Framework for learners with disabilities based on assessment analytics.

3. Ontological Scenario model

In this section we discuss the strengths and weaknesses of the proposed models and approaches that are cited in the related works.

From the comparative study between the personalization approaches proposed by the researchers we found that there is no personalization or recommendation framework for learners with disabilities that take into account both e-accessibility and e-assessment analytics. In this work, the recommendation system suitable for learners with disabilities is the one proposed by [18] since the authors have integrated the preferences and accessibility needs of each learner in the recommendation process. On the other hand, this approach has adopted the specifications of the IMS Access For All standard in order to define preferences, accessibility needs and digital resources.

Some authors proposed a new recommendation service includes two steps of filtering existing resources on the trace server: (1) filtering with respect to the user’s preferred language and file format, and (2) filtering with respect to the number of occurrences of the keywords in the title of the resource currently viewed by the user in the title and description of each resource. In the literature, there are models to analyze assessment trace data like [20]. This model relies solely on assessment data to extend the xAPI model. We propose therefore to use this model to exploit the assessment data in the recommendation process, through a scenario model in an accessible environment. To allow reusability and sharing, our recommendation scenario model is built upon an ontological structure enabling the formalized representation of the recommendation process.

The whole scenario model is oriented to learners with disabilities, at each phase we need to take into consideration the profile of the learners, their preferences, to be able to select and recommend the most appropriate resources. Figure 1 shows the proposed scenario model which is composed of the following five main phases:

• Phase 1: Pre-processing

In this phase two steps are necessary to feed the initial profile of the learner in order to analyze the behavior of the learner by the system:

• Registration: In this step the learner’s personal information and preferences are established via a graphical interface.
• Analysis of accessibility needs: The accessible environment will analyze the learner’s initial behavior and detect his handicap.

• Phase 2: Recommendation process based on pre-assessment analytics

In this phase three steps must be carried out which are:

• Pre-assessment: An initial test is set by the system to determine the learner’s level of prerequisites in the field.
• Pre-assessment analytics Based on the previous step, the system analyzes the pre-assessment traces to identify the learner’s prerequisites level.
• Provision of recommended resources for learning: In this step a list of resources is established to the learner according to the result of the pre-assessment analytics process.

• Phase 3: Recommendation process based on learning analytics

• Learning process: This step represents the learner’s engagement in the recommended learning process.
• Learning analytics
• The system collects and stores learning traces and updates the learner’s profile (preferences, learning style, etc.).
• Provision of recommended resources for learning
• This step represents the provision of recommended resources to the learner based on the learning analytics process.

• Phase 4: Recommendation process based on assessment analytics

• Assessment process: This step represents the learner’s engagement to complete an assessment test.
• Assessment analytics: The system collects and stores assessment traces and updates the learner’s profile.
• Provision of recommended resources for learning: This step represents the provision of recommended resources to the learner based on the assessment analytics process.

• Phase 5: Update of learner profile

In this final phase, the learner’s profile is updated based on the learning analytics and assessment analytics process.

Figure 1. Ontology-based Recommendation Scenario Model

4. Recommendation Framework for assessment of learners with disabilities

The goal of our proposed recommendation framework is to retrieve and select relevant learning and assessment resources to learners with disabilities based on their preferences, accessibility needs, and assessment trace data in the context of online learning. This Framework can be composed mainly of three models: a model that covers the preferences and accessibility needs of each learner, a model that allows exploiting the assessment trace data and finally a recommendation model.

In the literature we found that the suitable recommender system to recommend resources to learners with disabilities compared to other works is the approach proposed by [18]. Since the authors take into account the preferences and accessibility needs of each learner. The authors do not consider learning and assessment trace data in the recommendation process.

First, we propose to extend this approach by an assessment analytics model from [20] to exploit assessment trace data in the recommendation process. Secondly, we use the recommendation technique (collaborative filtering) based on the learners, the resources and the assessment trace data (the score obtained, number of correct answers, recording time, etc.) instead of the learners’ assessments on the learning resources. We have adopted this recommendation technique since it will allow us to focus on the history of users’ actions with the system. The framework has been organized in four basic components: a learner profile, an accessibility assessment analytics model, an accessibility learning analytics model and a recommendation model. Figure 2 illustrates the components of the proposed framework. In the following we explain the framework in detail.

Figure 2. Ontology-based Recommendation Framework for assessment of learners with disabilities

1. Learner Profile: During registration, a new user is asked to enter the following information: 1) personal information (name, gender, date of birth) and current occupation, 2) accessibility preferences, and 3) current education level and skill level (beginner, intermediate, advanced).
2. An accessibility assessment analytics model: This model is the result of combining the two models Accessibility model and the Assessment Analytics model. This combination is done by the relationship between the Learner class of the accessibility model and the Assess_Actor class of the Assessment Analytics model. This model provides assessment trace data to the recommendation model in order to provide relevant learning resources to learners with disabilities.
3. An accessibility learning analytics model: We propose to extend the accessibility model by a subclass Learner_History which is inherited from the Learner class. This subclass contains properties Log_Time, Visied_Digital_Ressource, Duration and LogOut_Time in order to exploit these properties in the recommendation process. The recommendation model can also use knowledge of this model when the learner does not enter his accessibility preferences.
4. Recommendation Model: This model will use accessibility preferences, learning trace data, assessment trace data and the learner’s profile to generate personalized recommendations to the active learner. This model works in three steps:

• Step 1: We consider this step as a first level of filtering. This step is based on the principle proposed by but we use the initial evaluation results instead of the learners’ evaluation (opinion) of the resources. This principle consists of incorporating accessibility preferences and needs into the recommendation process.
• Step 2: we consider this step as a second level of filtering by exploiting learning traces data. We will use collaborative filtering based on users. The working principle of user-based filtering is very simple [21]: determine which users are similar to the current user, and then compute a prediction value for each candidate item by analyzing the ratings that the current user’s neighbors have expressed on that item. In our work, the evaluations represent the data of learning traces (duration of consultation, number of visits,…) which are collected in an implicit way. This data is used by a matrix (Learners, Resources, Learning Trace) to calculate the similarity between learners and the prediction.

Computing similarity between users:

Similarity between two users (learners) x and y can be measured using Cosine similarity or  Pearson Correlation Coefficient (PCC) , we use the PCC similarity measure since it is the best performing measure in user_based collaborative filtering  [21], [22] as illustrated in the following formula (1):

Rx,j and Ry,j the learning trace data of learner x and the learning trace data of learner y.

_x is the mean score of all learning traces data provided by learner x.

After the calculation of similarity between learners a matrix of size NxN will be generated, where N is the number of learners. Then, to predict the learning resource j not evaluated in the matrix by the active learner x, the K learners who have the highest similarities with this learner will be used as inputs to compute the prediction of j for learner x as illustrated in the following formula (2):

Ry,j represents the learning trace data of learner y on learning object j.

• Step 3: we consider this step as a third level of filtering by exploiting the assessment trace data. This time the recommendation model will use the matrix (Learners, Resources, Score) with the rows representing the learners, the columns representing the resources and the boxes representing the assessment results to compute the similarity and predict the recommendations.

To generate recommendations to learners with disabilities we have several criteria to be taken into account (accessibility preferences, learning traces data and assessment traces data) which are structured in ontological models. That’s why we have proposed two recommendation algorithms for the second and third recommendation steps mentioned above:

a. User_based recommendation algorithm (exploiting learning traces):

1: For each new learner x, retrieve his/her profile and create a vector

2: While there are learners to compare there

3: create a vector for y

4: calculate the similarity between x and y

5: Sort the list of nearest neighbors

6: Get the first 10 nearest neighbors

7: For each learner in the top 10 list do

8: For each resource in a learner’s history do

9: If Rate>= 5 Then

10: add resource R to the KNN list

11: EndIf

12: EndFor

13: EndFor

The Rate represents either the duration of consultation of a resource or the number of accesses to this resource.     

b. User_based recommendation algorithm (exploiting assessment traces):

1: For each new learner x, retrieve his/her profile and create a vector

2: While there are learners to compare there

3: create a vector for y

4: calculate the similarity between x and y

5: Sort the list of nearest neighbors

6: Get the first 10 nearest neighbors

7: For each learner in the top 10 list do

8: For each resource in a learner’s history (assessment result) do

9: If assessment_result(Score) >= 5 Then

10: add resource R to the KNN list

11: End If

12: EndFor

13: EndFor

Score represents the score obtained by the learner after an assessment test.  The results of these algorithms will be combined to avoid cold start problems.

5. Conclusion

In this paper, we are interested in developing a Recommendation Framework for assessment of learners with disabilities. From the state of the art and the comparative study between research works related to our subject we found that the majority of these works do not take into account e-accessibility, and do not focus on e-assessment analytics despite its great informative potential. For this reason, we have proposed a Recommendation Framework for learners with disabilities based on e-assessment analytics. The advantage of this Framework is to ensure accessibility based on the IMS Access for All specification and to exploit the e-assessment analytics that is very little exploited in the related research work in order to recommend learning resources to learners with disabilities. As a next step we intend to design use case scenarios as an instantiation of the proposed scenario and to deploy the framework with learners with various types of disabilities to validate our proposed recommendation process.

References

• WHO (2011). World Report on Disability. Technical report, World Health Organization: https://www.who.int/publications/i/item/9789241564182
• Cooper, M., Treviranus, J., & Heath, A. (2005, August). Meeting the diversity of needs and preferences–a look at the IMS Access For All specifications’ role in meeting the accessibility agenda efficiently. In Accessible Design in the Digital World Conference 2005(pp. 1-3).
• Drachsler, H., Hummel, H., & Koper, R. (2009). Identifying the goal, user model and conditions of recommender systems for formal and informal learning. Journal of Digital Information, 10(2), 4-24.
• Khribi, M. K.,, Inclusive ICTs in Education, Nafath Workshop, Issue 17 – May 2021 https://doi.org/10.54455/MC.NAFATH17.03
• Zimmermann, A., Lorenz, A., & Oppermann, R. (2007). An operational definition of context. In Modeling and Using Context: 6th International and Interdisciplinary Conference, CONTEXT 2007, Roskilde, Denmark, August 20-24, 2007. Proceedings 6(pp. 558-571). Springer Berlin Heidelberg.
• Burke, R. (2002). Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction, 12, 331-370.
• Aguilar, J., Valdiviezo-Díaz, P., & Riofrio, G. (2017). A general framework for intelligent recommender systems. Applied computing and informatics, 13(2), 147-160.
• Tarus, J. K., Niu, Z., & Mustafa, G. (2018). Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning. Artificial intelligence review, 50, 21-48.
• Balabanović, M., & Shoham, Y. (1997). Fab: content-based, collaborative recommendation. Communications of the ACM, 40(3), 66-72.
• Karampiperis, P., & Sampson, D. (2005, May). Designing learning systems to provide accessible services. In Proceedings of the 2005 international cross-disciplinary workshop on web accessibility (W4A)(pp. 72-80).
• Lancheros-Cuesta, D. J., Carrillo-Ramos, A., & Pavlich-Mariscal, J. A. (2014). Content adaptation for students with learning difficulties: design and case study. International Journal of Web Information Systems, 10(2), 106-130.
• Nganji, J. T., Brayshaw, M., & Tompsett, B. (2011). Ontology-based e-learning personalisation for disabled students in higher education. Innovation in Teaching and Learning in Information and Computer Sciences, 10(1), 1-11;
• Butoianu, V., Catteau, O., Vidal, P., & Broisin, J. (2011). Un systéme à base de traces pour la recherche personnalisée d’objets pédagogiques: le cas d’ariadne finder. Atelier” Personnalisation de l’apprentissage: quelles approches pour quels besoins?”, EIAH 2011.
• Benouaret, I. (2017). Un système de recommandation contextuel et composite pour la visite personnalisée de sites culturels(Doctoral dissertation, Université de Technologie de Compiègne).
• Berkani, L., Nouali, O., & Chikh, A. (2012). A Recommendation-based Approach for Communities of Practice of E-learning. In ICWIT(pp. 270-275).
• Mbaye, B. (2018). Recommender System: Collaborative Filtering of e-Learning Resources. International Association for Development of the Information Society.
• Agbonifo, O. C., & Akinsete, M. (2020). Development of an ontology-based personalised E-learning recommender system. International Journal of Computer (IJC), 38(1), 102-112.
• EL Aissaoui, O., & Oughdir, L. (2020, April). A learning style-based Ontology Matching to enhance learning resources recommendation. In 2020 1st international conference on innovative research in applied science, engineering and technology (IRASET)(pp. 1-7). IEEE.
• Khribi, M. K., Othman, A., & Al-Sinani, A. (2022, July). Toward Closing the Training and Knowledge Gap in ICT Accessibility and Inclusive Design Harnessing Open Educational Resources. In 2022 International Conference on Advanced Learning Technologies (ICALT) (pp. 289-291). IEEE.
• Nouira, A., Cheniti-Belcadhi, L., & Braham, R. (2017). An Ontological Model for Assessment Analytics. In WEBIST(pp. 243-251).
• Aggarwal, C. C. (2016). Recommender systems(Vol. 1). Cham: Springer International Publishing.
• Jannach, D., Zanker, M., Felfernig, A., & Friedrich, G. (2011). An introduction to recommender systems. New York: Cambridge, 10, 1941904.