[Wowfunhappy]

I am very close to obtaining an elementary school teaching degree (46 of 49 credits completed), and as of this year I am a full time teacher at a private school (which doesn't have to care whether I officially have a license). My masters program is considered among the best in the state.

Unfortunately, I don't have many good things to say about my masters program. The majority of my classes have been interesting but useless in a real classroom. Teaching is just one of those things that you largely learn by doing.

Teaching does take a lot of skill and practice—I am surrounded at work by more experienced colleagues, and watching them always leaves me impressed—but I don't think it's something you can learn from a textbook.

Similarly, the licensure exams are just awful, at least in New York. I will leave you with a real practice test question from the official preparation materials. This is for the content knowledge test on "Science and Technology".

----------

A construction company is evaluating proposals for the creation of a new playground. They are using the following scale to assess the relevant criteria:

    +--------------+------------------------+
    | Scale number | Scale score assessment |
    +--------------+------------------------+
    | 1            | Far below standards    |
    | 2            | Below Standard         |
    | 3            | Meeting standard       |
    | 4            | Exceeding standard     |
    +--------------+------------------------+

Use the chart below to answer the question that follows:

    +----------------+-----------+-----------+-----------+-----------+
    | Criteria       | Company 1 | Company 2 | Company 3 | Company 4 |
    +----------------+-----------+-----------+-----------+-----------+
    | Safety         | 3         | 4         | 3         | 1         |
    | Quality        | 3         | 2         | 3         | 4         |
    | Creativity     | 4         | 3         | 3         | 4         |
    | Sustainability | 3         | 2         | 2         | 4         |
    | Utility        | 3         | 3         | 4         | 4         |
    +----------------+-----------+-----------+-----------+-----------+

According to the evaluation detailed in the chart, which company should be awarded the project?

----------

Ready for the answer? Take a moment to think about it before looking...

The answer key says it's company four, because they have "the highest overall score. We are not told any information about categories being weighted and therefore we cannot pay special attention to the low safety score."

[hn_throwaway_99]

This answer made me unreasonably angry. Apparently to become a teacher in New York, you're not allowed to use your brain. "We are not told any information about categories being weighted and therefore we cannot pay special attention to the low safety score." means exactly the following when you're talking about a playground: "Congratulations, all the kids love this creative, high quality playground! Unfortunately they're all at the hospital."

I agree with another commenter, you should name and shame, this is beyond stupid and deserves to be called out as the idiocy it is.

[Wowfunhappy]

> I agree with another commenter, you should name and shame, this is beyond stupid and deserves to be called out as the idiocy it is.

You do you want me to name, exactly?

The tests are made by Pearson Education. The website for the certification tests is https://www.nystce.nesinc.com/.

[hn_throwaway_99]

Yep, that was exactly what I meant. Thanks a lot for posting!

[gopher_space]

All of the federal "can you walk and chew gum" hiring tests are booby-trapped like this. The highest tier of scores go to the people who follow instructions literally and without regard to context, and who never apply their own moral judgment.

In the above example the last company is clearly sacrificing lives to get work done.

[dwattttt]

> "Congratulations, all the kids love this creative, high quality playground! Unfortunately they're all at the hospital."

The company even optimised the number of children who can experience it! A child in the hospital is one who got to use the playground, _and_ one who isn't hogging it, stopping others from experiencing it.

[ang_cire]

> "Congratulations, all the kids love this creative, high quality playground! Unfortunately they're all at the hospital."

I absolutely love this! xD

[throwaway421967]

But you were given "information about categories", Anything under 3 should be an auto-reject because it does not meet the standard.

"standard" usually implies that it's an outside rule from a regulatory body/certification agency that you need to conform to.

It's such a weird question to ask in the first place.

[s1artibartfast]

I dont think standard usually means codified. It is also frequently used to mean typical, or within expectation.

It this example, outside law or regulation on standard creativity and utility seem pretty unlikely.

There is also a pretty big difference between standard (singular) and standards (plural), where the latter is more likely to imply a set of minimum requirements.

[ang_cire]

The very first line item (Scale number 1) uses "standards" plural: "Far below standards"

[s1artibartfast]

weird

[s1artibartfast]

The correct answer is obviously "insufficient information"

Maybe company 4 is the best, but I don't see why no information on weights implies equal weights.

Edit:

After thinking about this phenomenon for a while, I think there is an argument for testing implied or unstated prompts. It is frustrating to have to read minds, and quess what expectations are in different contexts. However, building a mental models of other people is an important skill.

I dont know that I would want to hire someone entirely incapable of it, who routinely required complete and explicit instruction.

That said, I dont think this kind of cognitive test is what they were going for

[amluto]

> We are not told any information about categories being weighted and therefore we cannot pay special attention to the low safety score.

This is extra hilarious because, if you don’t know the weighting (or even that each score is linear), then adding the scores in the first place is an invalid operation.

[yencabulator]

Regardless of the weighing, company 1 is the only one meeting all standards, and thus should be the only one qualified.

[corny]

Company 4 is a reasonable answer. What's the point in having a scoring system if you're going to use personal judgement to determine the winner? Of course company 4 would be a bad fit. But the problem is the method that was originally chosen to evaluate the companies. The key is "according to the evaluation detailed in the chart". In reality, the company should look at these results and realize that they need a new evaluation method.

[KAKAN]

Is that a choice question? Or does it require a subjective answer? If it's the latter, then the question is reasonable to some degree, I guess?

“Company 4 might be the best in terms of naive computer-calculated average, but anything that doesn't meet a particular standard generally should not be allowed by law to be used/built. So Company 1 is the only choice, and perhaps the best, subjectively.”

I think this is a good question for discussion, because a child might answer “Company 4” at first just by looking at it and averaging it as anyone without any context would, but then you could say something more about it. But, I don't think it was intended to be analysed or answered this way. Either way, dumb question to ask for a qualification exam IMO.

[Wowfunhappy]

It's multiple choice, I posted screenshots of the real thing downthread: https://news.ycombinator.com/item?id=39073974

[indymike]

> We are not told any information about categories being weighted and therefore we cannot pay special attention to the low safety score."

And this is why we have debacles like the 737max.

[dilippkumar]

This is extremely meme worthy material. Would you mind sharing the source?

[Wowfunhappy]

The source is copyrighted / costs money. Also, I did purposefully choose the worst question as an example.

Practice tests can be purchased from this site https://www.nystce.nesinc.com but I can't find the page for this specific test (and don't really want to spend time looking). You're looking for the practice test for CST 245. This is the Arts & Sciences section of the Multi-Subject test for Teachers of Childhood, Grade 1 - Grade 6.

[Wowfunhappy]

Self-replying to add on, if you just want to see the original table (as opposed to as my ASCII version), here are screenshots of the question. I just can't share the full test since it's paid material. (Sharing one question for the purpose of commentary and criticism is obviously fair use.)

Question: https://i.postimg.cc/wvn7tRf8/Screen-Shot-2024-01-20-at-7-00...

Answer Key: https://i.postimg.cc/90TzyXK4/Screen-Shot-2024-01-20-at-7-02...

[SyrupThinker]

Seems about right for "Science and Technology", a variant of this question also shows up in German IHK (chamber of commerce) exams for software developer certifications, with a similar expected answer.

Although you do sometimes get weights… the important part is explaining that you evaluate some weighted sum and take the best result.

How much impact would this question have on the final result for the licensure exam?

IIRC during my own IHK examination this was worth around 5pp, which is almost enough to drop you an entire grade.

[Wowfunhappy]

> How much impact would this question have on the final result for the licensure exam?

Fun fact: I have no idea!

Basically all I know is that (1) tests are scored on a scale from 400 – 600, (2) the minimum passing score is 520, and (3) the test is not graded on a curve. But I don't know why they even bother sharing these numbers, because they don't divulge any information on how they're calculated.

https://www.nystce.nesinc.com/content/docs/NYSTCE_ISR_Back_M...

> Although you do sometimes get weights… the important part is explaining that you evaluate some weighted sum and take the best result.

To be clear, this is a multiple choice question. You need to select (A) Company 1, (B) Company 2, (C) Company 3, or (D) Company 4.

[dan-robertson]

What’s the point of a question like this? Somewhere someone will have written down something they want teachers to be able to do and that will have been translated into a question like this. Was the original requirement bad or the translation into the question? What kind of system would lead to a better question?

[Wowfunhappy]

I believe the standard is just "elementary school teachers should be able to read and interpret a table of data".

[jdietrich]

>What’s the point of a question like this?

To filter out people with the capacity and inclination to engage in critical thinking, because those people will not last as teachers.

https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-t...

[selimthegrim]

There is such a thing as testing for the conjunction fallacy.

[Wowfunhappy]

Sorry, I'm not sure I follow. I know what the conjunction fallacy is but I don't see how it applies here.

[selimthegrim]

Assuming a higher safety score means a “better” company isn’t an example (loosely speaking)?

[red_admiral]

That seems to match some real-world examples. "plug doors" anyone?

[rvba]

Are those MBA questions? Because that would explain a lot!