[dwater]

I was a public school teacher that worked in a district that attempted to measure teacher value add. There was limited pay for performance based on that metric. And the district I worked in was majority racial minority and majority living in poverty.

"Using the expiration of preexisting collective bargaining agreements as a source of exogenous variation in the timing of changes in pay, I show that the introduction of flexible pay raised salaries of high-quality teachers, increased teacher quality (due to the arrival of high-quality teachers from other districts and increased effort), and improved student achievement."

"The main dataset contains information on the universe of Wisconsin teachers, linked to student test scores to calculate teacher VA."

"Student Test Scores and Demographics.—Student-level data include math and reading test scores in the Wisconsin Knowledge and Concepts Examination (WKCE 2007–2014) and the Badger test (2015–2016), for all students in grades 3 to 8, as well as demographic characteristics such as gender, race and ethnicity, socioeconomic (SES) status, migration status, English-learner status, and disability."

What this study is saying is that paying teachers more for improved performance by their students on standardized tests results in higher standardized test scores of their students. That does not surprise me. In my experience, this system encouraged teaching to the test (focusing the majority of instruction on test prep rather than traditional instruction) and widespread cheating. Both of these efforts raise standardized test scores. The study assumes that standardized test scores are a direct measure of teacher quality and student achievement. In my experience that is not true.

[jorvi]

A secondary question is: is the pay increase high enough to induce more talented people to become a teacher?

If all it does is pull higher quality teachers from other places, it is zero sum and all you eventually end up with is the same quality schooling for a higher price.

I do think teachers as a whole should be paid well, but that is a separate discussion.

[dwater]

When I left teaching my salary went up 50% on day 1, to 100% after about a year, to 150% after 2 or 3 years, and has continued climbing significantly since. If I had stayed my salary would have lagged inflation. The pay for performance bonuses I've heard of have never been more than 20% of a teacher's salary. I'm not typical because I have CS and Math degrees, but still, I have no intention of ever considering returning to teaching.

Not part of this discussion, but I did work with people who left public schools to work at charter schools because you could get 25-50% more there, but they rarely stayed more than 2 years because of the working environment being more toxic than at public schools.

[brodouevencode]

What if the standardized tests aren't administered by the teacher? That seems like it would be an easy solution to get around the cheating problem.

As for teaching to the test, what's the real downfall of doing that? Colleges use standardized test scores as the biggest or one of the biggest criteria in admission.

[lahvak]

> What if the standardized tests aren't administered by the teacher? That seems like it would be an easy solution to get around the cheating problem.

In order for the test scores to be granular enough to be used as measure of performance of individual teachers, the testing would have to be done fairly often. The schools and school districts that are already lacking funds for basic stuff like maintenance and teacher and staff salaries would have hard time to pay an additional subject to administer the test.

With the frequency and volume of testing, it would likely be too costly to do the testing in an external testing facility (which most districts do not have), so it would have to be done on site. I remember my kids telling me that every time before they took a standardized test (it happens once a year around here), the teachers told them basically "we cannot give you the answers, but look around, the answers may already be there, there may be a poster hanging on the wall or something else like that that may be helpful."

[ufo]

The downfall is that in theory the purpose of school is to educate, not merely get people enough points to be admitted to college.

[dmix]

The schools are free to design their own pay packages right? The state isn't mandating that pay be tied to tests. This is mostly just an assumption that that would be the only metric used.

[RecycledEle]

> The schools are free to design their own pay packages right?

No. Most if the time, state laws set limits on what school districts can do.

[thaumasiotes]

And the purpose of the test is to see whether the education took. Teaching to the test is the whole point of the test.

[bena]

> As for teaching to the test, what's the real downfall of doing that?

This comment is a perfect encapsulation of AI hype.

Recitation is not competence. It doesn't matter if Bobby can say "4" when I say "What is 2 + 2?". What matters is if Bobby can add.

[ryathal]

Test questions are written to test varying levels of competence from basic recall to true mastery. It's partially why so many people hate word problems, they don't actually understand the concepts. Many others just can't read.

[Hizonner]

Test questions can be written to assess true mastery, although it's not so easy as you're trying to make it sound... especially if you want the test to take a sane amount of time, be consistently gradable by different graders, and have any kind of reasonable signal to noise ratio. But sure, you can get there, or at least go a long way toward it.

If you try to actually create tests that way, you will discover that the institutions and political atmosphere around you will punish you harshly. It's "unfair" to give a test that you can't pass by memorizing things, you see. And the people who will ultimately make the decisions about what you're allowed to do with your test are rarely going to care much about actual mastery, and often wouldn't be able to recognize mastery if they did care.

In practice, standardized tests are always going to be easily gamed, so if you make people's and institutions' rewards dependent on them, you will end up diverting more time, energy, and other resources away from actually teaching mastery and into meaningless gaming.

[ryathal]

Standardized tests spend the money and time to get those quality questions that are consistently graded. It's not some random folks making minimum wage coming up with questions.

[paulryanrogers]

Teaching the test narrows the subject matter significantly. Some of the most important things I learned in school weren't directly on tests: critical thinking, fallacies, cultural studies, how to avoid being exploited.

That said, in some circumstances only teaching the test could be an improvement.

[readthenotes1]

I was in a meeting once where an outside advisor asked the test group to share the test cases with the developers.

"But if we do that," the test manager complained, "the code will only work for the tests."

"Well, wouldn't that be a nice change." observed the Director.

[somethingsome]

One of the problem is that you can train the students to overfit any test, and that's worse than cheating which represent only a small number of students usually.

So by incentiving the teachers to make the students perform well in some tests, why teach real lessons when you can just throw at them cheat codes for the test.

[dwater]

If a teacher knows or suspects what questions will be on the test, they can drill their students to memorize the answers the week before the test. That's cheating and it doesn't matter who proctors the test.

[readthenotes1]

I once taught a class for teachers and after so many failed to be able to apply the quadratic equation, I told him that it would be the bonus question for the next test.

After so many then failed I told them that just writing the formula would be the bonus question on the next test.

Very few people got the bonus question.

In order to drill students to memorize, that students have to be present enough and interested enough for it to stick...

[dwater]

When I was in my teaching Masters program, we talked a lot about intrinsic vs. extrinsic motivation. Intrinsic motivation is when you want to do something for yourself, e.g. you enjoy the satisfaction of solving a problem. Extrinsic motivation is when your reason for doing it comes from outside yourself, e.g. your parents tell you they'll give you $20 for every A you get on your report card. The district I worked in experimented with extrinsic motivators to get students to attend school and perform well on the test with rewards like food, gift cards, and electronics. That was enough to get some students to try to do better, but usually only the students who already had some intrinsic motivation. The students that they were really trying to motivate were the ones with no intrinsic motivation, and they largely ignored the rewards offered.

[readthenotes1]

_Punished by Rewards_ is a great book on the topic. For your experiment, I curious how you controlled for suppressing counter motivations (e.g., peer pressure, pride, etc.)

The class was a required class for teachers to get their education degree. Back then, they had to have that degree in order to teach in the schools.

I doubt that many of them were thrilled for math, especially the more foundational topics we were going through so yes, their intrinsic motivation was mostly not high.

However their extrinsic motivation should have been off the charts. They literally couldn't get their degree without passing the course. Many were repeating the course having failed it before, so their chances were running out.

I do not recall any of the students being the normal 18 to 21 college students - - they were all older. They better understood why they were choosing their degree (looking back, based on demographics, I suspect many were divorcees prepping for a career or moms whose children had started school).

I gave the memorization question as a way to see how much of it was the work they were putting in and much was my inability to explain the difficult material in a way they could understand.

[IOT_Apprentice]

It may be that the manner in which you are teaching that topic is itself failing. Your students don’t understand, investigate how to improve your approach in explaining which results in them understanding. Decompose the steps to see where that understanding is not happening.

[readthenotes1]

It was decades ago so the advice is too late. As I taught other students successfully at the time (in that most seemed to grasp the material), there may be some other problems.

I boiled it down to "just write the equation down" to see if it were my teaching or their desire.

The inability for many to successfully replicate the formula indicated to me that they weren't putting the time in to memorize.

Since these were more adult learners going for their teaching degree, and many repeating the class having failed to pass it the first time, my observations beyond "I wasn't capable" are: - the course material was difficult (a rapid survey of math foundation, including some group theory) - they couldn't make the time because of other life commitments, exacerbated by being a summer course where we met every day.

I had one student, who had never scored higher than 50 on a test, show up after the final to ask if there was anything she could do to pass the class as it was required and she had flunked now 3 times and couldn't repeat it. "All I want to do is teach 3rd grade, I don't need any of this" she explained. There was nothing i could do...

[waveBidder]

when a metric becomes the target, it ceases to be a good metric

[azinman2]

My fear as well. Is there a better way to assess teacher quality, especially in challenging student environments?

[dwater]

First you have to define teacher quality. What should a good teacher provide to their students? Knowledge, ability, curiosity, safety, integrity, self-worth, honor, respect, kindness? Can these be quantified and compared against each other?

[soco]

None of those other ways can be done without effort and extra costs. And as nowadays everything has to be done cheaper and quicker, it will stay as that.

[readyman]

Don't assess teacher quality. Eliminate poverty by eliminating capitalism, a system that literally depends on it.

[waveBidder]

if all you have is a hammer…

[robocat]

HN guidelines:

  Please don't use Hacker News for political or ideological battle.

Please read guidelines link at bottom of page. So many of your past comments are simple anti-capitalist opinions that are not gratifying intellectual curiosity. Not a mod: just hoping for better discourse here.

[driscoll42]

Even in a non-capitalist society, you still need high quality teachers and ideally some way to measure them. And it's not like eliminating capitalism in any way would prove to end poverty.

[CoastalCoder]

> Eliminate poverty by eliminating capitalism, a system that literally depends on it.

Could you give an example of where this has worked?

[snapplebobapple]

If you define pverty as earning less than 15 % of the median country wage and then pay 90 % of people 2 dollars a day and 10 % of the people infinity dollars a day you have no poverty by definition. Some variation of that is how every other system of government/economy has beaten poverty. There is no point engaging these people unless you are a psychiatrist trying to help them with their mental illness.

[lupusreal]

Even easier than that, simply define poverty as a property of capitalism and if anybody says that poverty also exists in your communist society, you throw them into the gulag for spreading capitalist propaganda.

[dmix]

Why even engage this sort of thing? It's just going to turn into a flame thread.

[CoastalCoder]

Just in case I had something to learn.

Apologies for the risk of conflagration :)

[lupusreal]

Assessing teacher quality sounds a lot less destructive. You know, fewer people getting summarily executed for "crimes" such as owning property, being disliked by their neighbors, etc. That kind of bloodshed is what makes commies so red.

[bitshiftfaced]

The question is whether test scores as a measure of teacher quality is better than the next best alternative. The authors suggest that this would be seniority, but it could also be subjective measures like complaints from parents, likeability, and office politics.

I agree that there are problems with test-based evaluation, but overall I believe it's the least bad solution. It gives teachers a tangible incentive to figure out why what they're doing may not be working and try something different.

[gadders]

>>and widespread cheating

It amazes me that teachers would be so unethical. They seem worse than the general population.

[dwater]

They are the general population, look around. Teaching no longer provides a stable middle class life in most areas. They live paycheck to paycheck, struggle to afford housing, feel crippled by student loan payments, and have little opportunity for advancement.

[jddj]

"You get what you measure" is broadly applicable and systems need to accomodate that.

There's nothing about teachers that makes them less ethical than the the average <s>car emissions system engineer</s> <s>KPMG auditor</s> person

[gadders]

The expectation is that they would be more ethical, in the same way you would expect EG a doctor to be.

[Clubber]

It's easier to be incorruptible by money when you already have a lot of money.

One of the big societal problems with pushing down wages for 50 years is that it makes society less ethical because people aren't comfortable enough in their finances. When people aren't comfortable with their finances, they are less likely to be able to quickly ignore unethical things that might help their financial situation even a little bit.

Quick example: You see a guy drop a $20 when stuffing his ATM withdrawal into his wallet. I would say, "hey bud, you dropped a 20." Someone else might just let him walk out and pick it up for themselves because it would help them a lot more than me.

[cess11]

Where I live schools are commonly for profit, so shareholders and bosses exert pressure to get higher grades.

It works, without the students learning more. Our universities are complaining that students arrive that can hardly read even though their grades say they are able to.

And yeah, we have standardised tests administered centrally so we can actually to some degree measure 'grade inflation' too, and know very well that employment/pay based on grading undermines whatever slim value grading would have otherwise.

Edit: Oh, and parents also exert such pressure.

[bena]

Bingo.

My wife is a special education teacher and so much of her work does not get reflected on a test.

[readyman]

The sick idea that profit is the onlyway to motivate humans is historically idiotic. It is an outcome of pathetic efforts to legitimate capitalism, a system of exploitation.

[jackcosgrove]

I agree but I wouldn't pin all the blame on the big C.

Marxism places a lot of emphasis on materialism, which was always one of the biggest targets for its critics. One man's profit is another's stolen labor value. Both systems of economic (that should be the clue here) thought are very materialist and don't do a good enough job assigning value to intrinsic motivation, believing in something, following a passion, etc.

[CoastalCoder]

You were downvoted, but I think you're correct.

When I worked for the DoD, most of us were focused on doing the right thing for the country, and we worked diligently towards that.

The pay was mediocre, and potential performance bonuses were minimal.

But the job stability let us focus on the big picture, and we did.

I haven't worked anywhere else that had the same dynamic, even much better paying jobs in the private sector.