[sorcerer-mar]

No it wasn't (or at least that is not stated/proven by the article).

It says they were adjusting the scores, not in any one direction.

If kids were doing worse and you have flat scores, that'd indicate the test being adjusted to be easier.

If kids were doing better and you have flat scores, that'd indicate the test being adjusted to be harder.

Because they've designed this to produce flat scores, it obscures which of those two statements is true.

[riffraff]

The shortened title on HN is misleading, "masking improvement" vs "masking whether the students improved".

The former seems to say the students did improve but this was not visible.

[1659447091]

The article makes statements that the test were getting more difficult:

>> In addition, norm-referenced tests [STAAR] are designed so that a certain share of students always fail, because success is gauged by one’s position on the “bell curve” in relation to other students. Following this logic, STAAR developers use practices like omitting easier questions and adjusting scores to cancel out gains due to better teaching.

Followed by:

>> Ultimately, the STAAR tests ... were not designed to show improvement. Since the test is designed to keep scores flat, it’s impossible to know for sure if a lack of expected learning gains following big increases in per-student spending was because the extra funds failed to improve teaching and learning, or simply because the test hid the improvements.

>> Texas’ educational accountability system has been in place since 1980, and it is well known in the state that the stakes and difficulty of Texas’ academic readiness tests increase with each new version, which typically come out every five to 10 years. What the Texas public may not know is that the tests have been adjusted each and every year – at the expense of really knowing who should “pass” or “fail.”

What is not clear or known for sure is the gains, gained, from an increase of money spent per-student because the test, by it's nature, is designed to be flat and not show improvements. But the test itself is designed by the developers to cancel out gains by making it harder. Therefore, the test could not be used to assess that. It is just written a bit funny.

This hurt the lower income students most as the wealthier schools with better teaching resources improved faster, it made the test more difficult for poorer performing schools. This is used as a measure to close or take over schools by the state. There's a bit of a kerfuffle going on currently about some schools getting F ratings and the state wanting to take them over, which in other states that might be okay, by here -- they are about to mandate that the 10 Commandments be displayed in classrooms[0]. They are also on track to getting rid of the STAAR test[1]. Hopefully the new assessment they replace it with will be better, supposedly it will be to show improvements, I fear it will then fall into the "no child left behind" trap that STAAR may have prevented by keeping scores flat, even with major investment in education; but it may also help lower income students who are hurt by the current testing practice in that they will receive the funds they earned for their improvement instead of losing them because wealthy districts outpaced them in gains -- even thou they both gained. Texas is weird sometimes.

[0] https://www.texastribune.org/2025/05/24/ten-commandments-tex...

[1] https://www.texastribune.org/2025/05/23/staar-test-texas-sch...

[resoluteteeth]

The author does imply that the test is effectively getting more difficult, but it seems like they're simply assuming that the adjustment must be happening in that direction without actual evidence.

In reality, because the test was normalized, it is useless for determining the change in academic achievement over time, and it could just as easily end up being normalized in the direction of effectively making the test easier. Because the scores can't be used for this purpose, other evidence would be needed to determine how achievement is actually changing over time (and therefore whether the normalization is effectively increasing or decreasing scores) and the author doesn't present any evidence that would show which is the case.

However, either way the decision to normalize the scores on a test that's apparently supposed to be used to show changes over time seems odd if the author is correct that normalized scores are being used for that purpose.

[1659447091]

She did a peer-reviewed deep dive into the test design documentation [0] Stated in the second sentence of this article with the link.

Where she found:

>> According to policies buried in the documentation, the agency administering the tests adjusted their difficulty level every year.

She goes on to say that difficultly is increased. As shown in my original quotes of the article above.

I have not read her paper or the test design documents; but it appears she has and that she does have the evidence.

[0] https://doi.org/10.1080/15427587.2024.2415618