Black/African-American Learners in North America - Revision history

Shruti at 01:01, 29 June 2023

2023-06-29T01:01:09Z

← Older revision		Revision as of 21:01, 28 June 2023
Line 83:		Line 83:

	Zhang et al.(2023) [https://learninganalytics.upenn.edu/ryanbaker/ISLS23_annotation%20detector_short_submit.pdf pdf]		Zhang et al.(2023) [https://learninganalytics.upenn.edu/ryanbaker/ISLS23_annotation%20detector_short_submit.pdf pdf]
	* Models developed to detect attributes of student feedback for other students’ mathematics solutions, reflecting the presence of three constructs:1) commenting on process,2) commenting on the answer, and 3) relating to self.		* Models developed to detect attributes of student feedback for other students’ mathematics solutions, reflecting the presence of three constructs:1) commenting on process, 2) commenting on the answer, and 3) relating to self.
	* Models have approximately equal performance for African American, Hispanic/Latinx, and White students.		* Models have approximately equal performance for African American, Hispanic/Latinx, and White students.

Shruti at 01:00, 29 June 2023

2023-06-29T01:00:36Z

← Older revision		Revision as of 21:00, 28 June 2023
Line 80:		Line 80:
	* Despite comparable accuracy, model tends to underpredict Black students' performance		* Despite comparable accuracy, model tends to underpredict Black students' performance
	* Several fairness correction methods equalize false positive and false negative rates across groups.		* Several fairness correction methods equalize false positive and false negative rates across groups.


	Zhang et al.(2023) [https://learninganalytics.upenn.edu/ryanbaker/ISLS23_annotation%20detector_short_submit.pdf pdf]		Zhang et al.(2023) [https://learninganalytics.upenn.edu/ryanbaker/ISLS23_annotation%20detector_short_submit.pdf pdf]
	* Models developed to detect attributes of student feedback for other students’ mathematics solutions, reflecting the presence of three constructs:1) commenting on process,2) commenting on the answer, and 3) relating to self.		* Models developed to detect attributes of student feedback for other students’ mathematics solutions, reflecting the presence of three constructs:1) commenting on process,2) commenting on the answer, and 3) relating to self.
	* Models have approximately equal performance for African American, Hispanic/Latinx, and White students.		* Models have approximately equal performance for African American, Hispanic/Latinx, and White students.

Shruti: Addition

2023-06-29T01:00:21Z

Addition

← Older revision		Revision as of 21:00, 28 June 2023
Line 80:		Line 80:
	* Despite comparable accuracy, model tends to underpredict Black students' performance		* Despite comparable accuracy, model tends to underpredict Black students' performance
	* Several fairness correction methods equalize false positive and false negative rates across groups.		* Several fairness correction methods equalize false positive and false negative rates across groups.

			Zhang et al.(2023) [https://learninganalytics.upenn.edu/ryanbaker/ISLS23_annotation%20detector_short_submit.pdf pdf]
			* Models developed to detect attributes of student feedback for other students’ mathematics solutions, reflecting the presence of three constructs:1) commenting on process,2) commenting on the answer, and 3) relating to self.
			* Models have approximately equal performance for African American, Hispanic/Latinx, and White students.

Shruti at 21:06, 27 June 2023

2023-06-27T21:06:24Z

← Older revision		Revision as of 17:06, 27 June 2023
Line 58:		Line 58:


	Zhang et al. (~~in press~~) [https://www.upenn.edu/learninganalytics/ryanbaker/EDM22_paper_35.pdf pdf]		Zhang et al. (2022) [https://www.upenn.edu/learninganalytics/ryanbaker/EDM22_paper_35.pdf pdf]
	* Detecting student use of self-regulated learning (SRL) in mathematical problem-solving process		* Detecting student use of self-regulated learning (SRL) in mathematical problem-solving process
	* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.		* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.

Ryan: Added Jeong et al (2022)

2022-08-04T20:04:39Z

Added Jeong et al (2022)

← Older revision		Revision as of 16:04, 4 August 2022
Line 38:		Line 38:
	* E-rater gave African American test-takers significantly lower scores than human raters when assessing their written responses to argument prompts		* E-rater gave African American test-takers significantly lower scores than human raters when assessing their written responses to argument prompts
	* The shorter essays written by African American test-takers were more likely to receive lower scores as showing weakness in content and organization		* The shorter essays written by African American test-takers were more likely to receive lower scores as showing weakness in content and organization



Line 73:		Line 74:
	* Automated essay scoring models inferring text evidence usage		* Automated essay scoring models inferring text evidence usage
	* All algorithms studied have less than 1% of error explained by whether student is Black		* All algorithms studied have less than 1% of error explained by whether student is Black


			Jeong et al. (2022) [https://fated2022.github.io/assets/pdf/FATED-2022_paper_Jeong_Racial_Bias_ML_Algs.pdf]
			* Predicting 9th grade math score from academic performance, surveys, and demographic information
			* Despite comparable accuracy, model tends to underpredict Black students' performance
			* Several fairness correction methods equalize false positive and false negative rates across groups.

Ryan: Added Litman et al. (2021)

2022-07-04T16:30:57Z

Added Litman et al. (2021)

← Older revision		Revision as of 12:30, 4 July 2022
Line 68:		Line 68:
	Black/African American)		Black/African American)
	* Multiple fairness approaches lead to ABROCA of under 0.01 for overrepresented versus underrepresented students		* Multiple fairness approaches lead to ABROCA of under 0.01 for overrepresented versus underrepresented students


			Litman et al. (2021) [https://link.springer.com/chapter/10.1007/978-3-030-78292-4_21 html]
			* Automated essay scoring models inferring text evidence usage
			* All algorithms studied have less than 1% of error explained by whether student is Black

Ryan at 13:20, 4 July 2022

2022-07-04T13:20:30Z

← Older revision		Revision as of 09:20, 4 July 2022
Line 66:		Line 66:
	* Models predicting whether two students will communicate on an online discussion forum		* Models predicting whether two students will communicate on an online discussion forum
	* Compared members of overrepresented racial groups to members of underrepresented racial groups (over 2/3		* Compared members of overrepresented racial groups to members of underrepresented racial groups (over 2/3
	Black/African American		Black/African American)
	* Multiple fairness approaches lead to ABROCA of under 0.01 for overrepresented versus underrepresented students		* Multiple fairness approaches lead to ABROCA of under 0.01 for overrepresented versus underrepresented students

Ryan: Added Li, Xing, & Leite (2022)

2022-07-04T13:17:47Z

Added Li, Xing, & Leite (2022)

← Older revision		Revision as of 09:17, 4 July 2022
Line 61:		Line 61:
	* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.		* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.
	* No racial/ethnic group consistently had best-performing detectors		* No racial/ethnic group consistently had best-performing detectors


			Li, Xing, & Leite (2022) [https://dl.acm.org/doi/pdf/10.1145/3506860.3506869?casa_token=OZmlaKB9XacAAAAA:2Bm5XYi8wh4riSmEigbHW_1bWJg0zeYqcGHkvfXyrrx_h1YUdnsLE2qOoj4aQRRBrE4VZjPrGw pdf]
			* Models predicting whether two students will communicate on an online discussion forum
			* Compared members of overrepresented racial groups to members of underrepresented racial groups (over 2/3
			Black/African American
			* Multiple fairness approaches lead to ABROCA of under 0.01 for overrepresented versus underrepresented students

Ryan: added link

2022-06-10T10:13:34Z

added link

← Older revision		Revision as of 06:13, 10 June 2022
Line 57:		Line 57:


	Zhang et al. (in press)		Zhang et al. (in press) [https://www.upenn.edu/learninganalytics/ryanbaker/EDM22_paper_35.pdf pdf]
	* Detecting student use of self-regulated learning (SRL) in mathematical problem-solving process		* Detecting student use of self-regulated learning (SRL) in mathematical problem-solving process
	* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.		* For each SRL-related detector, relatively small differences in AUC were observed across racial/ethnic groups.
	* No racial/ethnic group consistently had best-performing detectors		* No racial/ethnic group consistently had best-performing detectors

Ryan: re-ordering of examples

2022-06-10T09:59:35Z

re-ordering of examples

← Older revision		Revision as of 05:59, 10 June 2022
Line 13:		Line 13:
	Christie et al. (2019) [https://files.eric.ed.gov/fulltext/ED599217.pdf pdf]		Christie et al. (2019) [https://files.eric.ed.gov/fulltext/ED599217.pdf pdf]
	* Models predicting student's high school dropout		* Models predicting student's high school dropout
	* The decision trees showed little difference in AUC among White~~, Black~~, Hispanic, Asian, American Indian and Alaska Native, and Native Hawaiian and Pacific Islander.		* The decision trees showed little difference in AUC among Black, White, Hispanic, Asian, American Indian and Alaska Native, and Native Hawaiian and Pacific Islander.


	Lee and Kizilcec (2020) [https://arxiv.org/pdf/2007.00088.pdf pdf]		Lee and Kizilcec (2020) [https://arxiv.org/pdf/2007.00088.pdf pdf]
	* Models predicting college success (or median grade or above)		* Models predicting college success (or median grade or above)
	* Random forest algorithms performed significantly worse for underrepresented minority students (URM; American Indian~~, Black~~, Hawaiian or Pacific Islander, Hispanic, and Multicultural) than non-URM students (White and Asian)		* Random forest algorithms performed significantly worse for underrepresented minority students (URM; Black, American Indian, Hawaiian or Pacific Islander, Hispanic, and Multicultural) than non-URM students (White and Asian)
	* The fairness of the model, namely demographic parity and equality of opportunity, as well as its accuracy, improved after correcting the threshold values from 0.5 to group-specific values		* The fairness of the model, namely demographic parity and equality of opportunity, as well as its accuracy, improved after correcting the threshold values from 0.5 to group-specific values

Line 42:		Line 42:
	Bridgeman et al. (2009) [https://www.researchgate.net/publication/242203403_Considering_Fairness_and_Validity_in_Evaluating_Automated_Scoring pdf]		Bridgeman et al. (2009) [https://www.researchgate.net/publication/242203403_Considering_Fairness_and_Validity_in_Evaluating_Automated_Scoring pdf]
	* Automated scoring models for evaluating English essays, or e-rater		* Automated scoring models for evaluating English essays, or e-rater
	* The score difference between human rater and e-rater was significantly smaller for 11th grade essays written by ~~White and~~ African American students		* The score difference between human rater and e-rater was significantly smaller for 11th grade essays written by African American and White students