[alex_smart]

>When a measure becomes a target, it ceases to be a good measure.

The success of JEE in selecting excellent students year after year is evidence to the contrary.

When the measure is set high enough, it actually becomes an excellent motivating factor in itself. Do you think the people participating in the various Olympiads (IMO, IPhO etc) not benefit from the experience? How about the Olympics?

[ItsMonkk]

While I am not Indian and the only thing that I know of this is what I have read here and from having watched 3 Idiots(2009), all metricization is the same. You can not test intelligence, you can only test knowledge. But everyone wants to know intelligence.

The examples you cite are finding the farthest distance in their field, and the only people capable of going that distance are people who move fast AND work hard. It only works when the participants are geeks for exactly the thing that is being tested.

As the people who study for a long time keep increasing their studying, the scores naturally improves, and therefore they need to make the test harder to keep the same pass rate. This only works so long as the juice is worth the squeeze.

We see this with the interview circuit. Top engineers, instead of studying up on CICO choose to interview elsewhere where real problems are asked. They have turned down working on esoteric problems that no one faces in the real world, even though if they were to face it would easily be able to solve it - likely by themselves from first principles.

We care about intelligence. The only way to learn someones intelligence is by seeing them work. It's by seeing them understand a problem new to society. Tests can not measure intelligence. Only personal accountability will work, but personal accountability doesn't scale.

The key is living in a society that doesn't scale.

[alex_smart]

>You can not test intelligence, you can only test knowledge.

You absolutely can test for a combination of intelligence and knowledge.

>But everyone wants to know intelligence.

Seldom do people actually care about intelligence in isolation. What use is intelligence without the discipline and ability to apply yourself towards a goal for a considerable length of time?

>As the people who study for a long time keep increasing their studying, the scores naturally improve

That simply does not work. Even with the coaching industry trying to lure parents into sending kids to them since Kindergarten, year after year it turns out that the students who do best in the JEE have only had two years of dedicated preparation. Because that is all the time that is needed for a talented student to prepare for the material that is being tested.

> and therefore they need to make the test harder to keep the same pass rate

If anything, the JEE has gotten easier with time.

Honestly, your entire argument seems to be based on dogma and not evidence. Selecting for the top 1% students by their aptitude for math/engineering, within an acceptable error margin, is not really as difficult a problem as you are making it out to be.

[ItsMonkk]

I know more about the South Korean system.

They grade to a curve. When the students get better, the test gets harder. Their students spend 16 hours a day studying for their university exams, and start at the age of 12. 46% of students are depressed. Suicide is the leading cause of death in both 10-19 and 20-29 population, mostly due to the stress.

The goal of all students is to pass the test so that they can make it into Seoul University so that Samsung will hire them. The bottleneck is the test. Life afterwards is easy.

Rigorous testing does not give you the best 1% of the population, it filters the people willing to go through the system down to the top 1%. Some of them will be of the .1% best, and others will be those who worked harder as they did not have to help their little sister get home, or their parents clean the restaurant after school.

You can not know your false positives. You can not know your false negatives. Every Einstein born to a poor family who does not get to study is wasted potential to humanity. Every Einstein whose passion is in Machine Learning and spends all of his time on that topic will fail this test.

[alex_smart]

>The goal of all students is to pass the test so that they can make it into Seoul University so that Samsung will hire them. The bottleneck is the test. Life afterwards is easy.

That sounds quite similar to the JEE tbh. Won't you say that a significant cause of the stress is the cultural expectation that all parents have that their should be able to get into Seoul University and eventually a job at Samsung?

In India there is a good reason for this mindset, because we do not have many good universities and base salaries are very low and of course there is not much of a social security net. I wonder what causes a similar mindset in a developed country like South Korea.

> Every Einstein whose passion is in Machine Learning and spends all of his time on that topic will fail this test.

Any person who does not have a basic understanding of his background and social situation is not an Einstein.

Regardless, that is a very strange example indeed. I really don't understand what kind of a person has the background and social circle that enables him to do top quality research in a highly specialized field of engineering (that also requires considerable funds to run any sort of experiments) at the age of 17 but also doesn't have the connections that will help him get into a decent university.

[valarauko]

While I largely agree with your argument, I'm curious if you have a solution in mind that could work in the Indian context.

[ItsMonkk]

Use the start-up model whenever possible.

1. Take as many bets as you can.

2. Pay a great deal of attention to those bets. Guide them.

3. As soon as you know the bet failed, let it fail.

Entrance exams are an anti-pattern. They commit to much to early. This means they have to be correct, so they get more strict. This squeezes out exactly who they were hoping to find. Same problem with interviews. Same problem everywhere.

[valarauko]

I'm not sure I follow how this would work for students applying for college. In the specific example of the JEE, you have ~ 13K really prestigious seats that ~ 2.2 Million students are competing for, with about ~ 50K seats of decreasing desirability up for grabs. Similarly for the top medical colleges, ~ 340K applicants are competing for ~ 1200 seats. More than half of those seats are reserved for students from traditionally marginalized & oppressed communities burdened with severe economic & societal hardships. I do not understand how a startup model helps in this scenario; ie, college admissions where desirable seats are a severely scarce resource.

[valarauko]

>Even with the coaching industry trying to lure parents into sending kids to them since Kindergarten, year after year it turns out that the students who do best in the JEE have only had two years of dedicated preparation. Because that is all the time that is needed for a talented student to prepare for the material that is being tested.

Do we have any figures (anecdotal or otherwise) of how many students in the top rankings have had coaching?

I am of the opinion that the JEE (and every other competitive exam: AIIMS, etc) should be much much harder, and designed with a specific eye to defeat coaching. I'd go so far as to draw an analogy with crypto algorithms designed to be ASIC - resistant.

[alex_smart]

>Do we have any figures (anecdotal or otherwise) of how many students in the top rankings have had coaching?

I would say 90%. Probably even higher. That's also because pretty much anyone who is serious about taking the exam gets some coaching. However, I know at least a few people that cracked JEE without any coaching. Even a guy who got rank 1 without coaching (Piyush Srivastava).

> I am of the opinion that the JEE (and every other competitive exam: AIIMS, etc) should be much much harder, and designed with a specific eye to defeat coaching

I am not sure how making the exam harder would defeat coaching. The value of the hard problems is that it helps you distinguish better between the very top of the top students. At least at my time there were always a number of such problems thrown in for that purpose.

I also don't quite understand what you hope to achieve here. The whole "coaching centers teach you tricks to solve problems" idea is way exaggerated. (Honest to God, I have a terrible memory and I actually derived half the formulae I needed during the exam itself.) There are many other ways in which coaching centers add value. The most important of which being that you surround yourself with and compete against other talented, motivated students.

[valarauko]

The goal isn't for the exam to be hard for the sake of being hard: it's to increase the discriminative power of the exam, increasing the gap between the top student and the bottom student to be selected. The fact that 13 students this year have the top score tells me the test cannot meaningfully discriminate between them, and that is likely true of raw score as well - probably hundreds or thousands of students share the same raw score, with the need to break ties. We're talking about a total point score spread of 375 to pick ~ 50K students, and that's not even including the requirements for minimum qualifying scores. If 13 students are able to get the same raw score, and still can't be meaningfully separated based on the established tie breaking criteria (presumably even more students got the same raw score but could be assigned a lower percentile based on tie breaking), then the issue is likely many magnitudes larger at lower raw scores where the distribution is much more crowded. I'm advocating a system where perhaps 10 points separates even the students at the very top. Of course, this system has its own problems, but that's besides the point: no system is perfect, and I think its still better than the current system.

Sure, the very top students are likely to be both talented & hard working, and would perhaps have still gotten on the list without coaching (though perhaps not high enough to have their pick). It's in the minutiae of small differences in raw scores where the benefits of coaching likely help. If coaching gives me an additional 10 points, that could mean the difference between getting a seat vs not for a bright student.

I think you're suggesting that joining a coaching centre is a strong signal of seriousness, rather than a material benefit from the actual classes. Possible, but as others in this subthread have pointed out, access to coaching is something out of reach for many students, irregardless of their seriousness. To pose the question another way: how many of the 50K people who got a seat would not if coaching (in its current form) did not exist? Perhaps the top 100 names would be redistributed within the top 5000, but they'd still have a seat. There's likely a lot more students in the long tail whose fortunes were determined by access to coaching. My concern is less with the top 100 and more with students at the precarious end.

[alex_smart]

Raw scores! (Btw what are the un-raw scores called? cooked?!)

13 students getting top scores!!

My God, it has been a while since I wrote the exam, has the situation changed that much since then? At my time, there was a 25 marks difference between the top two ranks.

Is there any chance you are talking about JEE Mains and not the JEE Advanced? JEE Mains is supposed to be the easier test that serves as a "prelims" to the more difficult Advanced exam. For Mains itself, it is not all that important to distinguish between the top students.

>I think you're suggesting that joining a coaching centre is a strong signal of seriousness, rather than a material benefit from the actual classes

I am also suggesting that there are other benefits from joining coaching that are probably more important than the much hyped "tricks" for solving problems. For one, joining a coaching class means that you are part of a community of other students with similar goals and aptitude.

Otherwise, I agree with your overall argument (although not in its entirety). Yes, at the lower end of the ladder, small difference in scores can result in big jumps in ranks and yes, coaching would definitely have a significant impact on the prospects of these students.

The point I do not agree with is lack of access to coaching. Even middle class families of Bihar, the poorest state of India, find the resources to send their kid to Kota for a couple of years. So, it is not that inaccessible. Plus there is apparently free material available on youtube now, created by the IITs itself.

I would also argue that people much below this threshold have bigger worries in life. We as a society definitely need to ask ourselves why it is not possible to have a dignified, livable wage for the majority of people in our country when many other countries seem to have solved this problem. Solving such hard socio-economic problems is a ridiculous expectation from a measly entrance exam.

[random314]

Several of the candidates qualifying for the exam have appeared multiple times for the exam. Very often the students have been studying for more than 2 years.