[SicSemperUranus]

> Test scores are highly predictive

Any study in this area that doesn't control for socio-economic factors is near useless, because it mainly predicts that children of successful (ie rich) people will likely grow up to be successful (ie rich). This is almost tautological.

[xyzzyz]

You seem to be completely unfamiliar with the research in the field. It is well known that controlling for socioeconomic factors does not substantially impact predictive validity of standardized tests. See for example

https://journals.sagepub.com/doi/abs/10.1177/095679761243873...

> Scores on the SAT (a test widely used in the admissions process in the United States), secondary school grades, college grades, and SES measures from 143,606 students at 110 colleges and universities were examined, and results of these analyses were compared with results obtained using a 41-school data set including scores from the prior version of the SAT and using University of California data from prior research on the role of SES. In all the data sets, the SAT showed incremental validity over secondary school grades in predicting subsequent academic performance, and this incremental relationship was not substantially affected by controlling for SES.

Or this

https://eric.ed.gov/?id=ED562860

> (a) SES is related to SAT scores (r = 0.42 among the population of SAT takers), (b) SAT scores are predictive of freshman grades (r = 0.47 corrected for school-specific range restriction), and (c) statistically controlling for SES reduces the estimated SAT-grade correlation from r = 0.47 to r = 0.44.

There are more studies on this, with rather consistent results. In short, poor students do about as well as rich students when matched on SAT score.

[majormajor]

Why would one expect socioeconomic factors to do one or both of:

a) influence grades in primary and secondary school

b) influence standardized test scores

but not also do:

c) influence college performance (esp. freshman year when you're least removed from those same effects in HS)?

[xyzzyz]

Not sure what your point is. My point is that controlling for SES is useless in context of the predictive validity of SAT. Regardless of the degree in which socioeconomic factor casually influence test scores and educational outcomes, you get no substantial gain from learning them above what you learn from test scores alone, as long as the goal is to select students who are most likely to learn and succeed at your university.

This means that admission officers who want to learn about SES of candidates in addition to learning their test scores usually either are looking for something else the student’s aptitude for learning and ability to succeed, or are completely unfamiliar with state of the research (this is sadly quite common, and this is often willful ignorance too).

[throwaway09223]

They certainly do all three.

This doesn't mean that SAT scores can't still be similarly predictive across socioeconomic gradients. Both of these things can be true without any inherent conflict.

[majormajor]

Sure, it just means it's a bit of a pointless debate...

Questions that are much more interesting to me are:

* should any level of the education system attempt to correct for underlying socioeconomic factors?

* if so, how? (I would love more discussion about this because it both seems like an extremely difficult problem and because otherwise we waste all our time and energy on non-effective things like removing standardized tests from admissions requirements.)

In the software engineering interview world, for instance, a whole lot of active discussion is spent around interviewing methods. Let's make that bigger! Let's figure out how to get the people who never even make it to the interview stage today a better opportunity.

Otherwise it's like taking two plants, reducing the amount of sunlight one gets to 50% of what the other one gets for several years, then moving them both to a new environment, waiting four more years, and saying "huh, the one that was taller before we moved them 4 years ago is still taller, guess that's a good measurement to use to pick plants, since even controlling for that, the taller ones from the light-limited cohort are also ended up taller than their cohort-mates after the last 4 years!"

[pannSun]

> Questions that are much more interesting to me are

Sure, but it's still worthwhile to point out factually inaccurate articles and deceptive papers. If they want to argue for how the education system should correct socioeconomic factors, they should do so honestly, not by lying to us that tests are useless.

[throwaway09223]

What debate?

The test scores are useful. That's the point. There's nothing to debate.

[SicSemperUranus]

I am actually familiar with some of the research actually, but thank you for your condescendence.

I didn't mean to imply that the SAT or grades are not predictive of future performance, only that lower socio-economic statuses will correlate to lower SAT grades in the first place, ie your (a) proposal in the second source.

From this same source:

> The SAT-grade relationship is not an artifact of common influences of SES on both test scores and grades

To substantiate my initial comment, test scores are highly predictive yes, but as you just linked, SES is highly predictive of test scores (and this is only for people who get to the damn test). This is the tautological part. High incomes make people test well which lead to jobs with high incomes. But since this correlation is not 100% but merely very high, we can pat ourselves on the back and say hey it's accurate!

Articles that defend the accuracy of the system tend to implicitly defend its legitimacy, which I think is a much more important question than "is SAT indicative of future performance" when considering whether the SAT is useful.

[xyzzyz]

> I didn't mean to imply that the SAT or grades are not predictive of future performance, only that lower socio-economic statuses will correlate to lower SAT grades in the first place, ie your (a) proposal in the second source.

Why then talk about “controlling for SES”? The correlation SES/educational ability very much exists, but why would one want control for SES? The goal is, allegedly, to test scholastic aptitude, so that we can identify students that are likely to succeed at school. If SES correlates with this ability, by controlling for this, all you’ll do is control out the effect.

> High incomes make people test well which lead to jobs with high incomes.

No, there is little evidence that high incomes makes people test well. Here, and in the entire thread, you are committing what is called a “sociologist’s fallacy”. This is from Meehl, P. E. “Nuisance variables and the ex post facto design.”:

> While every sophomore learns that a statistical correlation does not inform us as to the nature of the causality at work (although, except for sampling errors, it does presumably show some kind of causal relation latent to the covariation observed), there has arisen a widespread misconception that we can somehow, in advance, sort nuisance-variables into a class which occurs only at the input side. This is, of course, almost never the case. The usual tendency, found widely among sociologists and quite frequently among psychologists (particularly among those of strong environmentalist persuasion) is to assume sub silentio that there is a set of demographic-type variables, such as social class, domicile, education, and the like, that always operate as nuisance variables to obscure true relationships, and that function primarily as exclusively on the input side from the standpoint of causal analysis. This automatic assumption is often quite unjustified. Example: We study the relationship between some biological or social input variable, such as ethnic or religious background, upon a psychological output variable, such as IQ or achievement. We find that Protestants differ from Catholics or that Whites differ from Blacks. But we find further that the ethnic or religious groups differ in socio-economic class. We conclude, as an immediate inference and almost as a matter of course, that we have to ‘control’ for the socio-economic class variable, in order to find out what is the ‘true’ relationship between the ethnic or religious variable and the psychological output variable. But of course no such immediate inference is defensible, since on certain alternative hypotheses, such as a heavily genetic view of the determiners of social class, the result of such a ‘control’ is to bring about a spurious reduction of unknown magnitude in what is actually a valid difference

This was published over 50 years ago, but nevertheless, this mistake is done over and over in sociological research and discussions.

> Articles that defend the accuracy of the system tend to implicitly defend its legitimacy, which I think is a much more important question (…)

I actually have no idea what you mean by “legitimacy” here.

[garbagetime]

Your comment would be equally or more effective without this sentence:

>You seem to be completely unfamiliar with the research in the field.

[Dylan16807]

I disagree. When a comment is sufficiently wrong, it's useful to underscore that.

[garbagetime]

If a user wishes to underscore that a comment he is replying to is wrong, then he should do just that. There is no need for the user to attack the person making the comment.

[Dylan16807]

Calling someone "completely unfamiliar" is not much of an attack. I don't think it's any worse than saying the comment is "completely wrong", but it gets the point across slightly better.

[xyzzyz]

I was being charitable here. The less charitable alternative is that the person I was responding to was familiar with the literature, but nevertheless engaged in willful deception.

[gotoeleven]

Since acknowledging your evidence will inhibit my ability to blame the white supremacist cis-hetereonormative ableist patriarchy for my problems I will instead respond with an anecdote about how I know a lesbian indian in a wheel chair that bombed the SATs and is now president of the united states.

[Onanymous]

well, maybe, just maybe, that's because the socio-economic status correlates positively with cognitive abilities and the kids somehow inherit the traits of their parents?

[wingely]

Meh. Dumb rich kids end up with positive college and career outcomes regardless of their test scores. Let’s not suggest that wealth correlates to ‘better traits’.

[buscoquadnary]

That's funny I grew up as one of six children and my mother was a school teacher. Guess I have to go back to my HS and tell them they need to lower my scores because I wasn't rich enough.

[BoorishBears]

You should instead chide them for not emphasizing the fact statistics measures trends, not absolutes.

[Nowado]

Methodology classes didn't excel either.