Yesterday, EducationNext discussed new research demonstrating a persistent 50 year achievement gap between “Haves and Have Nots”, and we wanted to chime in about what the gap looks like for students in Arkansas.
While the national and Arkansas-specific research use different data sources and methodologies, the conclusions are the same: the gap in academic performance between students with fewer and greater economic resources is large and isn’t getting smaller.
The national research used an index of student socioeconomic status (SES) developed from student-reported information about the education level of their parents and possessions that they have in their home. The achievement data come from four testing programs: National Assessment of Educational Progress (NAEP) Long-Term Trend; Main NAEP; the Trends in International Mathematics and Science Study (TIMSS); and the Program for International Student Assessment (PISA). The researchers found that the gap in achievement between the highest 10% SES and the lowest 10% SES students has remained consistent over the last 50 years. The conclusion holds when the ranges are expanded to the highest and lowest SES quartiles as well.
What about Arkansas’ Gap?
Here at OEP, we looked into the achievement gap for Arkansas students and found similarly disheartening results.
For the Arkansas analysis, we use the student-level indicator of Free-Reduced Lunch (FRL) program participation as the indicator of student socioeconomic status because we don’t have information on parental education or home possessions. FRL eligibility is based on family income, and is often used as an imperfect proxy for a student’s socioeconomic status. We group students into two groups: Not participating in FRL, or participating in FRL. In 2008-09, the first year of our analysis, 56% of Arkansas students participated in FRL. The percentage increased to 59% in 2009-10, and has remained fairly consistent since then, ranging as high as 61% before declining slightly to 60% in 2017-18 .
For the achievement data, we use ten years of the annual Arkansas state assessments, which have changed over time. For the first six years of our analysis we use data from the Benchmark and End-of-Course exams. In the 2014-15 school year, students in Arkansas completed the PARCC exam, before switching to the ACT Aspire exam for 2016-17 and beyond.
All of these test changes have made it essentially impossible for Arkansas schools to examine achievement gaps over time, because each test scores and reports performance in different ways. Although the assessments have varied over time, each provides a measure of how students statewide perform in literacy and mathematics. We think it is very important for education stakeholders to be able to examine performance over time, so we used a common standardizing procedure to track the relative performance of different groups. We transform these scores into percentiles to aid in interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Note that the percentiles are standardized within year and state, meaning that they are not indicators of how ‘true’ achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed (details about the methodology can be found at the end of this blog).
We found that in Arkansas, the difference in achievement between “Haves and Have Nots” has remained essentially unchanged over the past decade.
As seen in Figure 1, Arkansas’ literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 39th percentile in literacy, while students who were not participating scored at the 65th percentile on average. Over time, the performance on non-participating students remained consistent, while FRL-participating student performance increased slowly over time to the 42nd percentile by 2017-18. The literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years.
As seen in Figure 2, Arkansas’ mathematics achievement gap increased slightly from 20 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 42nd percentile in mathematics, while students who were not participating scored, on average, at the 62nd percentile. Student performance fluctuated over time for students in both groups, but the performance of non-FRL participating student increased slightly over time, while the performance of their FRL participating peers remained steady, leading to the slight widening of the achievement gap in mathematics.
It is interesting to note that the achievement gaps in both literacy and mathematics were smallest in the 2014-15 school year, the first (and only) year that Arkansas students completed the PARCC assessment. The gaps closed due to an increase in the average performance of FRL-participating students AND to a decline in the average performance of non-FRL participating students. It is tempting to hypothesize causes behind this decreased gap, but futile because we have only a single data point, and the gap then returns to established levels.
Well Shoot!
As with the national research, we feel these are important things to consider as education/ the level of education attained is key to upward mobility. Both the national and Arkansas results show that certain subgroups are limited in their ability to achieve success academically in comparison to their peers, and that the achievement gaps have persisted over time.
Therefore, it is seen that schools in Arkansas and across the country are not truly meeting the challenge of reducing those disparities among these subgroups. How can that be addressed?
We need to keep digging in order to get to the root of this achievement gap issue. OEP is digging into these gaps more in depth, and will share more information about trends in grade-level achievement gaps, as well as gaps between racial/ethnic and gender groups. We look forward to highlighting the Arkansas schools and districts that HAVE closed the achievement gaps for some student populations.
About the methodology:
Students in grades 3-8 were consistently assessed in both content areas, but there was variation in when high school students were assessed. For our analysis we used all grades assessed in literacy and/ or math in a given year, and limited our analyses to students completing the general assessment.
Scale scores were standardized for each year within grade level and content area, creating a Z-score (mean of 0, standard deviation of 1). These z-scores were then averaged across student groups and transformed into percentiles to ease interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Percentiles are standardized within year and state, meaning that they are not indicators of how actual achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed.