Enhancing Student's Performance Classification Using Ensemble Modeling

in the quality of education and the academic achievements of students [2]. In the context of higher education, the prediction and understanding of student performance considers a significant process that leads to achieving academic goals and enhancing the overall quality of education. in addition, it allows the institute to support individuals and students at risk within an early stage in a timely manner, This leads to better learning experiences and overall improvement of educational efficacy [3]. Therefore, this paper proposes a unique approach to classify student performance using an ensemble model with a stacking classifier. This approach utilizes the capabilities of several machine learning (ML) algorithms with ensemble model to improve categorization [4]. In addition, this research creates a prediction model in order to classify students separate groups a similar performance at ABSTRACT: A precise prediction of student performance is an important aspect within educational institutions to improve results and provide personalized support of students. However, the predication accuracy of student performance considers an open issue within education field. Therefore, this paper proposes a developed approach to identify performance of students using a group modeling. This approach combines the strengths of multiple algorithms including random forest (RF), decision tree (DT), AdaBoosts, and support vector machine (SVM). Afterward, the last ensemble estimates as one of the bets logistic regression methods was utilized to create a robust and reliable predictive model because it considers The experiments were evaluated using the Open University Learning Analytics Dataset (OULAD) benchmark dataset. The OULAD dataset considers a comprehensive dataset containing various characteristics related to the student’s activities thereby five cases based on the utilized dataset were investigated. The experiment results showed that the proposed ensemble model presented its ability with accurate results to classify student performance by achieving 95% of accuracy rate. As a result, the proposed model exceeded the accuracy of individual basic models by using the strengths of various algorithms to improve the generalization by reducing the potential weaknesses of individual models. Consequently, the education institutes can easily identify students who may need additional support and interventions to improve their academic performance.


INTRODUCTION
The prediction of student performance is considered an important role within education institutions, which helps these institutes to make strategic decisions and better results.In addition, an early precise prediction aids the educational institutes to identify the exact needs of students, which leads to use right resources effectively and reduces the rate of failure.This prediction also helps to build strategic planning approaches to enable customized programs targeting weaknesses and improving academic capabilities [1].The academic staff can prepare lessons and instructional methodologies based on student performance to increase their understanding and achievement.Performance prediction plays a vital tool helping educators to recognize strengths and weaknesses within specific areas of individuals.Furthermore, it facilitates the process of evaluation by assessing the efficacy of educational methodologies and the caliber of instruction.By applying sophisticated predictive tools and technologies, educational institutions will be able to achieve long-lasting improvements in the quality of education and the academic achievements of students [2].
In the context of higher education, the prediction and understanding of student performance considers a significant process that leads to achieving academic goals and enhancing the overall quality of education.in addition, it allows the institute to support individuals and students at risk within an early stage in a timely manner, This leads to better learning experiences and overall improvement of educational efficacy [3].Therefore, this paper proposes a unique approach to classify student performance using an ensemble model with a stacking classifier.This approach utilizes the capabilities of several machine learning (ML) algorithms with ensemble model to improve categorization [4].In addition, this research creates a prediction model in order to classify students separate groups a similar performance at ABSTRACT: A precise prediction of student performance is an important aspect within educational institutions to improve results and provide personalized support of students.However, the predication accuracy of student performance considers an open issue within education field.Therefore, this paper proposes a developed approach to identify performance of students using a group modeling.This approach combines the strengths of multiple algorithms including random forest (RF), decision tree (DT), AdaBoosts, and support vector machine (SVM).Afterward, the last ensemble estimates as one of the bets logistic regression methods was utilized to create a robust and reliable predictive model because it considers The experiments were evaluated using the Open University Learning Analytics Dataset (OULAD) benchmark dataset.The OULAD dataset considers a comprehensive dataset containing various characteristics related to the student's activities thereby five cases based on the utilized dataset were investigated.The experiment results showed that the proposed ensemble model presented its ability with accurate results to classify student performance by achieving 95% of accuracy rate.As a result, the proposed model exceeded the accuracy of individual basic models by using the strengths of various algorithms to improve the generalization by reducing the potential weaknesses of individual models.Consequently, the education institutes can easily identify students who may need additional support and interventions to improve their academic performance.
the start of semester.This allows to educators and administrators using an accurate tool to identify students in need for additional academic help.These tools such as data mining (DM) and machine learning (ML) techniques have the potential impact to enhance educational procedures.
To enhance predictability, a stacking classifier and a meta-learning approach were utilized within the proposed ensemble model.These approaches include the basic RF, Adaboost, DT, and SVM models, each having distinct capabilities to deal with data and learning patterns.The stacking classifier minimizes individual flaws while increasing prediction ability.Due to its interpretability and efficiency, logistic regression is employed as the final estimator, allowing educators to gain insight into aspects contributing to student performance forecasts.

Related work
This section delves into the existing studies related to knowledge surrounding factors that influence academic achievement, the methodologies utilized to assess student performance, and the interventions implemented to enhance educational outcomes.In this regard, Wahid et al. (2019) present a discussion on the use of learning analytics in virtual learning environments to anticipate at-risk students and provide early intervention strategies.This study used a deep artificial neural network with manual characteristics collected from click data.Their experiments achieved classification accuracy ranging from 84% to 93%.In addition, the neural network outperforms the logistic and base regression models with accuracy ranging from 79.82% to 85.60% and from 79.95% to 89.14%, respectively.The purpose of this article is to help institutions develop the basic framework for educational assistance and facilitate decision-making processes in higher education toward sustainable education [5].
Hassan et al. (2020) suggested the DM methods with video learning analytics approaches to forecast the overall performance of 772 students registered in e-commerce technology modules and e-commerce at a higher education institution (HEI).The study used eight different classification algorithms to consider data from a student knowledge system, a mobile application, and a learning management system.To reduce the characteristics, the data was transformed and preprocessed, and then genetic search and principal component analysis were performed.RF correctly predicted successful students at the end of the semester with an accuracy of 88.3% by using equal proportions of presentation and information gain.This study demonstrates how video learning analytics and DM methods can be used to predict student success in higher education institutions [6].
Rodriguez Hernandez et al. (2021) presented an ANN in order to predict academic achievement in higher education.They also investigated the importance of several determinants of academic achievement in higher education.This study involved 162,030 students of all races from Columbia's private and public colleges.They illustrated that the scientists have discovered the possibility to use ANN to identify students' academic performance as high or low, with an accuracy of 82% and 71%, respectively.They also discovered that in evaluation metrics such as the F1 score and recall, ANN beat current ML techniques [7].Gaftandzhieva, S. et al. (2022) proposed Moodle Learning Management System (LMS) and Zoom data to estimate the final grades of students enrolled in the object-oriented programming course at the University of Plovdiv.The dataset comprises final grades, online course activities, and lecture attendance records for a total of 105 students.To examine the association between scores and the online activity environment, the study used chi-square tests and logistic regression tests.ML algorithms, including RF, XGBoost, KNN, and SVM, are applied.In particular, RF exhibits the highest prediction accuracy at 78%.Research underscores the significance of data-driven predictions in helping educators and decision-makers identify at-risk students and improve overall academic performance [8].
This study by Yac et al. (2022) looked at the use of educational DM to predict final test marks for undergraduate students based on midterm exam outcomes.It employed many machine learning methods on a dataset of 1854 students from a Turkish state institution.Using only three parameters: midterm test marks, department data, and faculty data, the model obtains a classification accuracy rate of 70-75%.The study stresses the importance of data-driven research in higher education in terms of helping decision-making processes and identifying people at high risk of failing.It also determines the most effective machine-learning approaches for this prediction problem [9].
Al-Azizi et al. (2023) focused on Learning Analytics, specifically tracking student performance trends in virtual learning environments within Massive Open Online Courses (MOOCs).It introduced a multiclass day-wise prediction model called ANN-LSTM, which uses Long-Short-Term Memory (LSTM) and Artificial Neural Networks.The model outperformed baseline models, achieving an accuracy of approximately 70% by the course's third month, compared to 53% for a Recurrent Neural Network (RNN) and 57% for a Gated Recurrent Unit (GRU).It also presented a greater accuracy compared to state-of-the-art models, with improvement rates located between 6% and 14%.This highlighted that LSTM's has the ability in early predictions for student performance in MOOCs [10].2023) assessed the integrated academic information system in higher education that used to improve education quality using DM and ML approaches.This study utilized the CRISP-DM methodology and classification tools to develop a prediction model.This leads to rate students' prospects of passing a course early.The findings show that the prediction model is trustworthy, with an accuracy of 0.7619 and an f1 score of 0.8571.These findings suggest the acceptability and effectiveness of analyzing and predicting student performance.The study recommended that the fully automated prediction results for rapid decision-making should be considered.In addition, it suggested further research to incorporate demographic data and additional indicators of criteria for student success [11].Waheed et al. (2023) study on self-paced education programs found that the long-term memory deep learning (DL) technique (LSTM) can predict students at risk of failing a course.The study used data from 22,437 students, with 69% passing and 31% failing instances.The LSTM algorithm achieved an accuracy of 84.57%, a precision of 82.24%, and a recall of 79.43%.It also outperformed conventional algorithms with up to 71% diagnostic accuracy in the first five weeks of training.The study highlights the importance of interpretability in deep learning techniques [12].

Doctor et al. (
Bin Roslan et al. ( 2023) employed DM approaches to predict secondary school students' success in English and mathematics.It found the determination of student success, understanding the characteristics of students with varying degrees of performance, determining the most effective DM strategy, and investigating the link between English and mathematics achievement.Archival data from Malaysian Certificate of Examination (MCE) students aged 16 in 2021 was used.Past academic achievement, demography, and psychological factors were also considered.For prediction, Decision Tree (DT) rules and Nave Bayes (NB) approaches were applied.The study emphasized the importance of previous academic success, demography, and psychological characteristics in predicting performance.The study also demonstrated that a student's previous Mathematics performance can predict their MCE English performance concluding that the two disciplines were interrelated [13].
Poudyal and colleagues (2022) emphasized the expanding use of data mining techniques in educational data analysis to improve learning results.Educators can utilize the increasing amount of data created by technology, elearning tools, and online courses to forecast student success, including academic results.However, there has been little study on the application of convolutional neural networks (CNNs) in educational contexts.To predict academic success, the authors created a hybrid 2D CNN model by integrating two separate models.In terms of precision, the model surpassed established baseline models such as k nearest neighbor, naive Bayes, decision trees, and logistic regression.This shows that CNN-based models can improve academic performance forecasts, indicating their use in educational DM applications [14].

Research Methodology
This study investigates the effectiveness of using Ensemble model architectures with machine learning algorithms to predict student performance within the academic field.Additionally, the goal is to evaluate their combined ability to categorize and forecast student performance in five examples from a specified dataset.The study starts with data collection followed by the preprocessing phase, which includes four consecutive steps: data purification, data transformation, merging datasets, and feature selection.Afterward, the extracted features are used to construct ML algorithms capable of prognosticating student performance based on novel measurements.To evaluate the performance of the proposed model, the model is subjected to new data with associated labels.The data are used to build the ML model, constituting the training set of 70%, while the remaining 30% is reserved for assessing the efficacy of the model, forming the test set.Then apply balancing for the dataset, and after that the final estimator via logistic regression.Upon rigorous testing of the models, the outcomes are juxtaposed to discern which yields the highest accuracy.The workflow of the proposed method is shown in Figure 1.204 207

Dataset
This study utilizes the OULAD benchmark dataset to evaluate the proposed model [15].The OULAD dataset was collected from the Open University database located at Walton Hall, Milton Keynes, UK, in the year 2015.The dataset encompasses information related to courses, students, and their interactions within a virtual learning environment (VLE) across seven specifically chosen modules.The dataset comprises a comprehensive extraction of a relational database, composed of interconnected data tables using distinct identifiers, that is supplied in the form of many commaseparated values (.csv) files.Figure 2 provides a concise summary of the OULAD database schema, illustrating three  3 Classified Withdraws, where withdraws are labeled as 0, failures as 1, passes as 1, and distinctions as 1. 1 Classified Failures among Courses that have been Officially Concluded.Where failures are labeled as 0, passes as 1, distinctions as 1 The dataset contains various information related to 32,593 students, each student was assigned one of the following labels: "fail", "withdrawn", "pass", or "distinct".In the present investigation, a binary classification procedure was conducted in which the labels "fail" and "withdrawn" were combined and treated equivalently, both of which were assigned the label "fail".Similarly, the labels "pass" and "distinct" were aggregated and assigned the label "pass".Upon completion of the aggregation process, a total of 17,208 students were categorized as "fail" and 15,385 students were categorized as "pass".The OULAD dataset is utilized to extract various pieces of information, including details about the course, the number of credits studied, the student's profile, the student's previous attempts at the course, the assessments conducted by either a tutor or a computer, the examination, and the number of interactions with the learning platform.The report includes categorized information for each variable, such as minimum, maximum, sum, and average values over time.In this manner, a comprehensive set of 97 features was algorithmically clustered and extracted in order to delineate the classification quandary.The extraction of features is carried out by a PHP 7.0 script.This proposed uses five cases shown in Figure 3.

Preprocessing
Data preprocessing is a fundamental step in the data analysis and ML pipeline, ensuring that the data fed into a model is accurate, relevant, and optimized.This process involves several key steps, including data cleaning, data transformation, data merging, feature selection, and balanced data.

Data Cleaning
This section tries to substitute the missing values that refer to the absence of data in a particular cell or field within a dataset.This due to various reasons such as human errors, data entry issues, or data corruption.Therefore, collected or recorded data should be completed and solved at this stage.

Data Transformation
This section performs transforming to the data from its initial format to an alternative representation.Generally, It includes normalization process to adjust the data to adapt to a range spanning from 0 to 1. Additionally, binning process will be applied to consolidate numerical values into smaller groups, and the conversion of categorical variables into numerical representations.These methods are crucial in ensuring the maintenance of consistency and accuracy during the process of data analysis.

Data Merging
This section performs a data merging of multiple datasets into a unified entity because data that is gathered from diverse sources in varying forms.The critical factor for achieving successful inclusion is to ensure the compatibility of the data and minimize the introduction of errors or bias during the merging process.This can be achieved by using unique identifiers for each data point and checking for inconsistencies after the merger, as shown in Figure 4.

Feature Selection
Feature selection is the process of identifying the most relevant features for a model.This is crucial as irrelevant or redundant features can negatively impact the model's performance.

Balanced data
In the context of ML, this term refers to a dataset where each class (or category) of the target variable has roughly an equal number of instances.Having balanced data is important because it helps avoid the issues that can arise from imbalanced datasets, where one class dominates the others, leading to biased model performance.

Proposed Ensemble Model
This study used an ensemble model of that stacking classifier that combines AdaBoost, SVM, DT, and Random Forest used an ensemble model of which is a final estimate using logistic regression for classification is a powerful technique for improving the accuracy classifier.Figure 5 depicts the proposed ensemble model.The RF, a widely used approach based on decision trees, creates a collection of decision trees.Each tree is trained on a randomly selected subset of both training data and features.This deliberate randomness helps to introduce diversity and address overfitting concerns.During the prediction phase, individual trees automatically generate class predictions.The ultimate prediction is established through either majority voting or by averaging the predictions from all trees.By taking advantage of the amalgamated insights of multiple trees, the RF algorithm improves accuracy and robustness [16].

AdaBoost (adaptive boost):
AdaBoost operates as an ensemble technique that iteratively trains a sequence of weak classifiers (typically decision stumps, characterized by their shallow structure with just one split) on distinct subsets of the training dataset.Each of these weak classifiers is designed to emphasize instances that were previously misclassified or exhibited elevated error rates.Throughout the training process, AdaBoost assigns higher weights to these challenging instances, thereby enabling subsequent weak classifiers to pay them greater attention.The final prediction made by the AdaBoost model is a weighted amalgamation of the predictions generated by individual weak classifiers [17].

Decision Tree
The DT algorithm is a flexible method that builds a model that resembles a tree through iterative data partitioning, utilizing various features and their corresponding thresholds.This algorithm offers a straightforward and interpretable framework, adept at capturing intricate patterns within the data while accommodating both categorical and numerical attributes.However, it is susceptible to overfitting, prompting the use of regularization methods such as pruning to alleviate this potential concern [18].

Support Vector Machine
SVM stands as a powerful algorithm primarily applied to binary classification endeavors, though its capabilities extend to include multiclass classification.The core objective of SVM is to identify an optimal decision boundary that maximizes the gap between different classes.It demonstrates proficiency in dealing with linearly separable data and can tackle nonlinearly separable data by employing the kernel trick.SVM boasts commendable generalization attributes and exhibits efficacy within feature spaces of high dimensionality [19].However, challenges may arise when faced with overlapping or indistinct classes, underscoring the significance of meticulous kernel and hyperparameter selection.Using a stacking classifier that combines the strengths of AdaBoost, SVM, DT, and Random Forest, this ensemble model can provide improved classification accuracy and generalization performance compared to using either algorithm individually.Ensemble models have a proven track record in enhancing predictive performance by combining the outputs of multiple models, effectively mitigating biases and errors.Each base model in the ensemble plays a role in the final prediction, leading to a more robust and precise result.Through the amalgamation of RF, SVM, DT, and AdaBoost, with Final Estimator using logistic regression for classification is a powerful technique for improving the accuracy classifier, the ensemble model capitalizes on its unique capabilities to address diverse aspects of the student performance detection challenge [20].

Evaluation
Recall, precision, accuracy, and F-score are commonly used to evaluate the effectiveness of ML models.These metrics provide valuable information on different aspects of model performance.
Precision assesses the accuracy of identifying positive instances among the total predicted true positives(TP), indicating a low rate of false positives(FP) [21], [22].
Recall evaluates the ability of the model to identify all positive instances of total actual positives(TP), indicating a low rate of false negatives(FN) [23].
Accuracy, on the other hand, measures the general accuracy of the model by calculating the ratio of correct predictions as a TP and true negative (TN) to the total number of predictions made [24], [25].
Lastly, the F-measure combines recall and precision to provide a single metric that balances both measures [26].
In the provided equations, the various metrics are expressed through specific calculations that rely on the values of TP, FP, TN, and FN.These values are obtained from the model's predictions and the actual ground truth [27].

Result and discussion
Student performance was classified using the Ensemble model for five cases.This study proposed an ensemble model through voting of four algorithm classifiers AdaBoost and RF, DT, and SVM.
As shown in Table 1, the accuracy results of the classifier using the proposed ensemble model were the best in diagnosing students' performance in case1.It can be seen that the accuracy of the EM had the highest value of accuracy 95% while the performance of case2 based on accuracy achieved 93%, case3 achieved 91, case4 achieved 88, and case5 achieved 94.As shown in Table 2, a comparison of results with related work studies, it is crucial to explore state-of-the-art techniques that employ ML or DL methods.Waheed et al. [5] utilized DANN and achieved an impressive accuracy of 0.93%.Hasan et al. [6] developed and employed the DM technique to identify student performance, achieving an accuracy of 88.3%.Rodríguez et al. [7] utilized ANN to extract student performance resulting in an accuracy of 82%.Gaftandzhieva et al. [8] used ML algorithms such as RF, XGBOOST, KNN, and SVM and achieved an f-measure of 78% for the detection of student performance.Yağcı et al. [9] utilized RF, SVM, KNN, LR, and NB and achieved an impressive accuracy of 86.4%.Al-Azazi et al. [10] developed the ANN-LSTM method to classify student performance.They achieved an accuracy of 71.35%.Doctor et al. [11] used DT to identify student performance and gain an accuracy of 76.19%.Waheed et al. [12] employed the deep LSTM approach to identify student performance with accuracy of 84.57%.
It is notable that differences in the datasets were used; it can direct the comparisons with the suggested approach to not be feasible.Various factors cab affect the performance of these DL techniques such as the size which can be a significant factor to influence the performance.However, the proposed ensemble model remains competitive.

Conclusion
In this study, the effectiveness of ensemble model was presented to classify student performance using a stacking classifier with RF, AdaBoost, DT, and SVM as base models, and logistic regression as the final estimator.A robust predictive model was proposed to categorize students into distinct performance groups at the early stage of semester.
The experimental results demonstrated that the proposed ensemble model outperformed individual base-models with higher accuracy, precision, recall, and F score.This due to the advantages of using strengths of various techniques.
Additionally, the ensemble model effectively combined its predictive power, mitigated individual weaknesses, and produced more reliable and consistent performance predictions.The stacking classifier played a vital role in optimizing the model's overall accuracy and generalization capabilities.

Figure 3 :
Figure 3:Cases used in proposed.