It’s been a big week in Arkansas education, with A-F Letter Grades and ACT scores being released, but both scores are generally a repeat of last year’s results.
As expected, the majority of schools (67%) received the same letter grade as last year. There was some movement up a level (16% of schools) or down a level (17%) of schools, but these were generally more reflective of schools crossing over a classification threshold than significant changes in performance. ACT scores for Arkansas’ 2018 graduates were just the same as they were for the Class of 2017. The average composite score of 19.4 and the determination that 17% of Arkansas graduates met the readiness benchmarks in all four tested areas are both below the national average and exactly the same as last year’s results.
How are school leaders, parents, and policy makers to interpret the static results? Here at the OEP, we hope to share some tools for interpreting the new data. We also want to get into the weeds about how the Achievement and Growth scores interact in the ESSA model.
Before we jump into the details, we have some good news to highlight!
First- we congratulate ADE on getting the school performance information out so quickly this year. This helps schools evaluate how their school is serving students and make changes in a timely manner to address areas where the numbers are low.
Second- we celebrate the transparency that ADE has built into the system, and we fully support Arkansas’ decision to provide all students the opportunity to take the ACT for free. These are positive decisions for students! We love the public availability of the school performance reports through myschoolinfo.arkansas.gov– especially the option to compare achievement and growth performance with similar schools. This option is available within the Reports tab by selecting similar schools based on %FRL, racial percentages and/or geographic proximity (see images below). If you haven’t checked this out, you should!
The data is a lot to take in, and it can be tough to try to figure out what’s important and what the patterns are. We have posted this interactive data viz of the Letter Grades and associated scores to help you see a statewide picture. You can find your school, and filter by academic growth score and poverty rate to see how schools that are similar to yours have scored.
OEP’s Interactive 2017-18 ESSA Data Visualization
Even with all the data out there, however, we are concerned that it may be difficult for many stakeholders to understand how their school is performing. In addition, we think each indicator provides insight independently, and so we made these simple one-pagers for easy reference. To ease interpretation for stakeholders, we assigned each school a percentile rank (within their assigned grade span) for the Achievement, Growth, and School Quality indicators. While this is too superficial for school personnel to act upon, we hope you find it a helpful communication tool. You can get them from our website officeforeducation policy.org, or here:
OEP’s One-Page School Summary of 2017-18 ESSA Data
Now we are going to dig into the school performance data, and consider the new letter grades. Don’t worry – there are pictures, and you can download the data we used here!
As expected, the majority of schools (67%) received the same letter grade as last year. There was some movement up a level (16% of schools) or down a level (17%) of schools, but these were generally more reflective of schools crossing over a classification threshold than significant changes in performance. Overall, the percentage of schools receiving each letter grade was similar to last year.
Figure 1: School Letter Grade Percentages, 2016-17 and 2017-18.
Certain types of schools are more likely to get A’s and B’s.
Once again this year, we find that schools serving a lower percentage of students who participate in Free/Reduced Lunch (FRL) generally get better grades than schools who serve a more disadvantaged population. As you can see in Figure 2 below, there is a strong negative relationship between FRL and ESSA scores (r=-0.69) although schools with the same %FRL have a range of about 20 ESSA points. Take note that there are a lot of schools reporting 100% FRL, which is the result of districts participating in the community eligibility provision. We like the CEP, but some of these schools reported much lower FRL rates in years past, so we should use caution when considering the performance of these schools since it may not be an apples-to-apples comparison.
Figure 2: School ESSA Index Score and %FRL, 2017-18.
As we’ve said before, the ESSA index is mainly driven by achievement, and two scores are almost perfectly correlated at r=.97. This is frustrating because the model was supposed to consider growth more heavily than achievement, but in reality achievement scores overwhelm growth. In addition, you can see in Figure 3 that this year’s achievement is almost perfectly correlated with prior year achievement (r=.94). So high achieving schools (generally lower FRL schools) tend to get better grades both years.
Figure 3: Weighted Achievement, 2016-17 and 2017-18.
What about GROWTH?
The good news is that academic growth (our favorite) is less associated with school FRL rates than achievement (see in Figure 4 (r=-.25)). Growth is the indicator that measures how students scored compared to how well we predicted they would score based on prior achievement. We feel it is very important to examine this indicator carefully, as it is the best reflection of the learning occurring in our classrooms.
Figure 4: Academic Growth and %FRL, 2017-18.
Interestingly, as you can see in Figure 5, academic growth is strongly correlated with prior year growth (r=.80). This means that schools with high growth in 2016-17 also had high growth in 2017-18. The reverse is also true, unfortunately, as schools where students performed lower than predicted in 2016-17 demonstrated the same pattern in 2017-18.
Figure 5: Academic Growth 2016-17 and 2017-18.
This is great news for Arkansas educators! We have an indicator of student learning that is not very correlated with school poverty, but seems to be consistently identifying schools as high, average, or low growth. Keep an eye out for the OEP awards, which will celebrate the high growth school!
But does increased growth relate to increased achievement, as we would think it should? The figure below represents the change in growth score from 2016-17 to 2017-18 and the change in achievement score over those same two years. Although increased growth does not always correlate with positive changes in achievement (because students can be achieving at a higher level than predicted but not necessarily making it into another proficiency category), there are few schools in the lower right quadrant where growth decreased and achievement increased. Of greatest concern are the schools in the lower left quadrant, where growth and proficiency both decreased since the prior year.
Figure 6: Change in Academic Growth and Achievement, from 2016-17 to 2017-18.
So what should school leaders, parents, and policy makers do?
- Get into the data! Understand and communicate how your school is performing on each indicator compared to similar schools. If there is an indicator where scores are relatively high, build on that success. If there is an indicator where scores are relatively low, consider what might be contributing to that score and work to change students’ school experiences. Stakeholders and policy makers should celebrate school successes, and work to support development in low-performing indicators.
- Focus on the learning every day! The ESSA Index, the ACT Aspire, and the ACT tests are once-a-year snapshots of performance. Teachers should be using high quality ongoing formative assessments to understand where their students are and work to help them move them forward in their learning. Stakeholders and policy makers should support schools as they work to make an opportunity for growth for their students.
- Stay the course! Change takes time. To move the needle on student achievement and success in Arkansas, we need to put in the work and give the work time. Changing assessments or backing away from rigorous and high-quality school performance analyses will only add instability into the system.
Arkansas education is on the right track, so let’s keep playing that tune.