Reducing the Filtering Effect in Public School Admissions: A Bias-aware Analysis for Targeted Interventions
0
🎯
Sign in to get full access
Overview
- The paper addresses the problem of fair and effective access to top-quality education in New York City's public schools.
- Traditionally, admission to these schools has been based solely on Specialized High School Admissions Test (SHSAT) scores, which are influenced by socioeconomic factors.
- This has led to a lack of diversity and exacerbated issues like school segregation and class inequality.
- The paper takes an operations research approach, modeling a bias in the performance of disadvantaged students and analyzing the impact on the admissions process.
- It explores how centrally planned interventions, such as scholarships or training programs, can help mitigate the effects of this bias.
Plain English Explanation
The paper looks at the challenge of ensuring fair and equal access to the best public schools in New York City. Historically, these schools have selected students based entirely on their scores on a specialized admissions test, the SHSAT. However, these test scores are known to be influenced by the socioeconomic status of the students and the quality of test preparation they receive in middle school.
This has led to a situation where the student population in these top schools is not representative of the broader New York City student population. The schools tend to be dominated by students from more privileged backgrounds, while students from disadvantaged economic situations are underrepresented. This contributes to issues like school segregation and a lack of diversity and class mixing.
To address this problem, the researchers take a different approach, using operations research techniques to model the situation. They find that there is a systematic bias in the test scores of disadvantaged students, where their true potential is underestimated. This bias then has a significant impact on the admissions process and the resulting student population.
The researchers show that targeted interventions, such as scholarships or training programs aimed at disadvantaged students with average performance, can help reduce the effects of this bias and create a more equitable and representative student body in these top schools.
Technical Explanation
The paper takes an operations research approach to the problem of fair and effective access to top-quality public schools in New York City. Using data from the Department of Education (DOE), the researchers model a bias in the performance of students classified as disadvantaged, primarily based on economic factors.
This bias is characterized as an underestimation of the true potential of disadvantaged students, which then impacts the assortative matching process used to assign students to schools. The researchers analyze the effects of this bias on the resulting student population and school diversity.
Through their analysis, the paper demonstrates that centrally planned interventions, such as scholarships or training programs, can significantly reduce the impact of the bias when targeted at the segment of disadvantaged students with average performance. This suggests that carefully designed policies can help address the challenges of school segregation and lack of class diversity in New York City's top public schools.
Critical Analysis
The paper presents a thoughtful and rigorous approach to a complex problem, but it also acknowledges several limitations and areas for further research. One key limitation is the reliance on DOE data, which may not fully capture the nuances of the student population and their socioeconomic circumstances.
Additionally, the model used to characterize the bias in student performance could be further refined and validated with additional data sources. The paper also does not delve deeply into the potential unintended consequences or implementation challenges of the proposed interventions, which would be an important consideration for policymakers.
While the paper's findings suggest that targeted interventions can help mitigate the impact of bias, it would be valuable to explore a range of policy approaches and their relative effectiveness, as well as to consider the broader systemic factors that contribute to educational inequities.
Conclusion
The paper offers a novel perspective on the challenge of ensuring fair and equitable access to top-quality public education in New York City. By modeling the bias in the performance of disadvantaged students and analyzing its impact on the admissions process, the researchers identify targeted interventions as a promising approach to address school segregation and lack of diversity.
The insights from this research could inform the development of more effective and inclusive policies, helping to create a more level playing field for students from diverse socioeconomic backgrounds. As the scientific community continues to grapple with these complex issues, this paper contributes valuable analytical tools and evidence-based strategies for improving educational opportunities and outcomes.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🎯
0
Reducing the Filtering Effect in Public School Admissions: A Bias-aware Analysis for Targeted Interventions
Yuri Faenza, Swati Gupta, Aapeli Vuorinen, Xuan Zhang
Problem definition: Traditionally, New York City's top 8 public schools have selected candidates solely based on their scores in the Specialized High School Admissions Test (SHSAT). These scores are known to be impacted by socioeconomic status of students and test preparation received in middle schools, leading to a massive filtering effect in the education pipeline. The classical mechanisms for assigning students to schools do not naturally address problems like school segregation and class diversity, which have worsened over the years. The scientific community, including policymakers, have reacted by incorporating group-specific quotas and proportionality constraints, with mixed results. The problem of finding effective and fair methods for broadening access to top-notch education is still unsolved. Methodology/results: We take an operations approach to the problem different from most established literature, with the goal of increasing opportunities for students with high economic needs. Using data from the Department of Education (DOE) in New York City, we show that there is a shift in the distribution of scores obtained by students that the DOE classifies as disadvantaged (following criteria mostly based on economic factors). We model this shift as a bias that results from an underestimation of the true potential of disadvantaged students. We analyze the impact this bias has on an assortative matching market. We show that centrally planned interventions can significantly reduce the impact of bias through scholarships or training, when they target the segment of disadvantaged students with average performance.
Read more7/17/2024
0
Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability
Jinsook Lee, Emma Harvey, Joyce Zhou, Nikhil Garg, Thorsten Joachims, Rene F. Kizilcec
Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-2024 admissions cycle, these colleges faced unprecedented challenges. First, the number of applications has been steadily growing. Second, test-optional policies that have remained in place since the COVID-19 pandemic limit access to key information historically predictive of academic success. Most recently, longstanding debates over affirmative action culminated in the Supreme Court banning race-conscious admissions. Colleges have explored machine learning (ML) models to address the issues of scale and missing test scores, often via ranking algorithms intended to focus on 'top' applicants. However, the Court's ruling will force changes to these models, which were able to consider race as a factor in ranking. There is currently a poor understanding of how these mandated changes will shape applicant ranking algorithms, and, by extension, admitted classes. We seek to address this by quantifying the impact of different admission policies on the applications prioritized for review. We show that removing race data from a developed applicant ranking algorithm reduces the diversity of the top-ranked pool without meaningfully increasing the academic merit of that pool. We contextualize this impact by showing that excluding data on applicant race has a greater impact than excluding other potentially informative variables like intended majors. Finally, we measure the impact of policy change on individuals by comparing the arbitrariness in applicant rank attributable to policy change to the arbitrariness attributable to randomness. We find that any given policy has a high degree of arbitrariness and that removing race data from the ranking algorithm increases arbitrariness in outcomes for most applicants.
Read more7/17/2024
0
Learning treatment effects while treating those in need
Bryan Wilder, Pim Welle
Many social programs attempt to allocate scarce resources to people with the greatest need. Indeed, public services increasingly use algorithmic risk assessments motivated by this goal. However, targeting the highest-need recipients often conflicts with attempting to evaluate the causal effect of the program as a whole, as the best evaluations would be obtained by randomizing the allocation. We propose a framework to design randomized allocation rules which optimally balance targeting high-need individuals with learning treatment effects, presenting policymakers with a Pareto frontier between the two goals. We give sample complexity guarantees for the policy learning problem and provide a computationally efficient strategy to implement it. We then apply our framework to data from human services in Allegheny County, Pennsylvania. Optimized policies can substantially mitigate the tradeoff between learning and targeting. For example, it is often possible to obtain 90% of the optimal utility in targeting high-need individuals while ensuring that the average treatment effect can be estimated with less than 2 times the samples that a randomized controlled trial would require. Mechanisms for targeting public services often focus on measuring need as accurately as possible. However, our results suggest that algorithmic systems in public services can be most impactful if they incorporate program evaluation as an explicit goal alongside targeting.
Read more7/11/2024
🔮
0
Inside the Black Box: Detecting and Mitigating Algorithmic Bias across Racialized Groups in College Student-Success Prediction
Denisa G'andara, Hadis Anahideh, Matthew P. Ison, Lorenzo Picchiarini
Colleges and universities are increasingly turning to algorithms that predict college-student success to inform various decisions, including those related to admissions, budgeting, and student-success interventions. Because predictive algorithms rely on historical data, they capture societal injustices, including racism. In this study, we examine how the accuracy of college student success predictions differs between racialized groups, signaling algorithmic bias. We also evaluate the utility of leading bias-mitigating techniques in addressing this bias. Using nationally representative data from the Education Longitudinal Study of 2002 and various machine learning modeling approaches, we demonstrate how models incorporating commonly used features to predict college-student success are less accurate when predicting success for racially minoritized students. Common approaches to mitigating algorithmic bias are generally ineffective at eliminating disparities in prediction outcomes and accuracy between racialized groups.
Read more7/12/2024