Difference between revisions of "Course Grade and GPA Prediction"

From Penn Center for Learning Analytics Wiki
Jump to navigation Jump to search
m
 
(25 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Lee and Kizilcec (2020) [[https://arxiv.org/pdf/2007.00088.pdf pdf]]
Lee and Kizilcec (2020) [https://arxiv.org/pdf/2007.00088.pdf pdf]
* Model predicting college course grade of median or above
* Out-of-the-box random forest model violates demographic parity and equality of opportunity for URM (underrepresented minority: American Indian, Black, Hawaiian or Pacific Islander, Hispanic, and Multicultural) than for non-URM students (White and Asian)


Yu et al. (2020) [[https://files.eric.ed.gov/fulltext/ED608066.pdf pdf]]
* Models predicting college success (or median grade or above)
*Random forest algorithms performed significantly worse for underrepresented minority students (URM; American Indian, Black, Hawaiian or Pacific Islander, Hispanic, and Multicultural) than non-URM students (White and Asian), for male students than female students
*Random forest algorithms performed significantly worse for male students than female students
* The fairness of the model, namely demographic parity and equality of opportunity, as well as its accuracy, improved after correcting the threshold values from 0.5 to group-specific values<br />


* Model predicting undergraduate course grades and average GPA
*students of several racial backgrounds were inaccurately predicted to perform worse than other students


Riazy et al. (2020) [[https://www.scitepress.org/Papers/2020/93241/93241.pdf pdf]]
Yu et al. (2020) [https://files.eric.ed.gov/fulltext/ED608066.pdf pdf]


* Model predicting course outcome
* Models predicting undergraduate course grades and average GPA
* Fairly marginal differences were found for prediction quality and in overall proportion of predicted pass between groups
 
* Inconsistent in direction between algorithms.
* Students who are international, first-generation, or from low-income households were inaccurately predicted to get lower course grade and average GPA than their peer, and fairness of models improved with the inclusion of clickstream and survey data
Li and colleagues (2021) [[https://arxiv.org/pdf/2103.15212.pdf pdf]]
*Female students were inaccurately predicted to achieve greater short-term and long-term success than male students, and  fairness of models improved when a combination of institutional and click data was used in the model
*Model predicting student achievement on the standardized examination PISA
 
*Inaccuracy of the U.S.-trained model was greater for students from countries with lower scores of national development (e.g. Indonesia, Vietnam, Moldova)
 
 
Riazy et al. (2020) [https://www.scitepress.org/Papers/2020/93241/93241.pdf pdf]
 
* Models predicting course outcome of students in a virtual learning environment (VLE)
* More male students were predicted to pass the course than female students, but  this overestimation was fairly small and not consistent across different algorithms
*Among the algorithms, Naive Bayes had the lowest normalized mutual information value and the highest ABROCA value, or differences between the area under curve
* Students with self-declared disability were predicted to pass the course more often
 
 
Jiang & Pardos (2021) [https://dl.acm.org/doi/pdf/10.1145/3461702.3462623 pdf]
* Predicting university course grades using LSTM
* Roughly equal accuracy across racial groups
* Slightly better accuracy (~1%) across racial groups when including race in model
 
 
Kung & Yu (2020)
[https://dl.acm.org/doi/pdf/10.1145/3386527.3406755 pdf]
* Predicting course grades and later GPA at public U.S. university
* Five algorithms and three metrics (independence, separation, sufficiency) analyzed
* Poorer performance for Latinx students on course grade prediction for all three metrics; poorer performance for Latinx students on GPA prediction in terms of independence and sufficiency, but not separation
* Poorer performance for first-generation students on course grade prediction for independence and separation, and for some algorithms for GPA prediction as well
* Poorer performance for low-income students in several cases, about 1/3 of cases checked
 
 
Jeong et al. (2022) [https://fated2022.github.io/assets/pdf/FATED-2022_paper_Jeong_Racial_Bias_ML_Algs.pdf]
* Predicting 9th grade math score from academic performance, surveys, and demographic information
* Despite comparable accuracy, model tends to overpredict Asian and White students' performance, and underpredict Black, Hispanic, and Native American students' performance
* Several fairness correction methods equalize false positive and false negative rates across groups.
 
 
Sha et al. (2022) [https://ieeexplore.ieee.org/abstract/document/9849852]
* Predicting course pass/fail with random forest in Open University data
* A range of over-sampling methods tested
* Regardless of over-sampling method used, course pass/fail performance was moderately better for males
 
 
Deho et al. (2023) [https://files.osf.io/v1/resources/5am9z/providers/osfstorage/63eaf170a3fade041fe7c9db?format=pdf&action=download&direct&version=1]
* Predicting whether course grade will be above or below 0.5
* Better prediction for female students in some courses, better prediction for male students in other courses
* Generally worse prediction for international students

Latest revision as of 09:09, 23 February 2023

Lee and Kizilcec (2020) pdf

  • Models predicting college success (or median grade or above)
  • Random forest algorithms performed significantly worse for underrepresented minority students (URM; American Indian, Black, Hawaiian or Pacific Islander, Hispanic, and Multicultural) than non-URM students (White and Asian), for male students than female students
  • Random forest algorithms performed significantly worse for male students than female students
  • The fairness of the model, namely demographic parity and equality of opportunity, as well as its accuracy, improved after correcting the threshold values from 0.5 to group-specific values


Yu et al. (2020) pdf

  • Models predicting undergraduate course grades and average GPA
  • Students who are international, first-generation, or from low-income households were inaccurately predicted to get lower course grade and average GPA than their peer, and fairness of models improved with the inclusion of clickstream and survey data
  • Female students were inaccurately predicted to achieve greater short-term and long-term success than male students, and fairness of models improved when a combination of institutional and click data was used in the model


Riazy et al. (2020) pdf

  • Models predicting course outcome of students in a virtual learning environment (VLE)
  • More male students were predicted to pass the course than female students, but  this overestimation was fairly small and not consistent across different algorithms
  • Among the algorithms, Naive Bayes had the lowest normalized mutual information value and the highest ABROCA value, or differences between the area under curve
  • Students with self-declared disability were predicted to pass the course more often


Jiang & Pardos (2021) pdf

  • Predicting university course grades using LSTM
  • Roughly equal accuracy across racial groups
  • Slightly better accuracy (~1%) across racial groups when including race in model


Kung & Yu (2020) pdf

  • Predicting course grades and later GPA at public U.S. university
  • Five algorithms and three metrics (independence, separation, sufficiency) analyzed
  • Poorer performance for Latinx students on course grade prediction for all three metrics; poorer performance for Latinx students on GPA prediction in terms of independence and sufficiency, but not separation
  • Poorer performance for first-generation students on course grade prediction for independence and separation, and for some algorithms for GPA prediction as well
  • Poorer performance for low-income students in several cases, about 1/3 of cases checked


Jeong et al. (2022) [1]

  • Predicting 9th grade math score from academic performance, surveys, and demographic information
  • Despite comparable accuracy, model tends to overpredict Asian and White students' performance, and underpredict Black, Hispanic, and Native American students' performance
  • Several fairness correction methods equalize false positive and false negative rates across groups.


Sha et al. (2022) [2]

  • Predicting course pass/fail with random forest in Open University data
  • A range of over-sampling methods tested
  • Regardless of over-sampling method used, course pass/fail performance was moderately better for males


Deho et al. (2023) [3]

  • Predicting whether course grade will be above or below 0.5
  • Better prediction for female students in some courses, better prediction for male students in other courses
  • Generally worse prediction for international students