Hacker News new | ask | show | jobs
by balnaphone 1046 days ago
Am I misreading this?

> How tall was the president when JFK was born?

I infer that "JFK" is being referred to as "the president", regardless of what stage of life he's at. Therefore the correct answer would be the length of his body as a newborn, with nothing to do with who was the president at the moment of his birth.

For example, given the headline: "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!", would you expect an essay from Herbert Hoover while he was in office, or an essay from JFK when he was in grade 9?

5 comments

A lot of exposés of LLM weaknesses involve abusing the ambiguities of English with questions that are impossible for humans to give a satisfactory answer to, because they can be read in multiple ways.

Part of my bullishness on LLMs is that a lot of the criticism is so bad, and if they weren't the real deal there would be more low hanging fruit to attack them on. Cryptocurrency was having real problems from the first moment people realized how much data they would have to download to initialize a node.

> A lot of exposés of LLM weaknesses involve abusing the ambiguities of English with questions that are impossible for humans to give a satisfactory answer to

There's a general unstated assumption that a truly intelligent entity wouldn't make mistakes, which is sort of contradicted by, you know, the existence of humanity. I think making (certain kinds of) mistakes is a _sign_ of intelligence, rather than a sign that it's not.

Yes, you're misreading it. "the president" is not a label that applies solely to JFK, or one that applies at all to JFK in the context "when JFK was born".

If someone gave me the question "who was the president when JFK was born?", the answer is clearly not JFK, so it isn't correct to infer that JFK is the president whose height, party affiliation or essay writing skill is being asked about "when JFK was born".

If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked. If you flip it to "was JFK strong at writing essays when the president was a high school freshman" then yes, I'd consider both to refer to JFK because "the president"can't be identified as anyone else, but I'd also consider it to be bad writing by someone cargo-culting the idea of elegant variation...

> If someone gave me the headline "was the president strong at writing essays when JFK was a high school freshman?" I would assume that either they were trying to trick me or that they were a non-native English speaker or computer program that didn't understand how English syntax or the concept of presidency worked.

I don't think it's as complicated as all that. Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people. If they had meant to refer to the same person, they would have used a pronoun. I'd probably guess they were either talking about the President at the time at which JFK was a freshman (Hoover, I think) or the current President at the time the question was being asked.

On the other hand, if the question was "Was President Kennedy strong at writing essays when JFK was a high school freshman?" then yeah, I'd agree with your take. In this case, it's clear that the writer is intentionally referring to the same person but using two different proper nouns, which is a very odd phrasing.

> Most English speakers wouldn't refer to a single person by two distinct proper nouns in a sentence like this, so I'd assume they were referring to two separate people.

I agree with this (though elegant and inelegant variation is a thing). But nor would any English speaker be likely to ask about the skill of an anonymous holder of a position with reference to an ambiguous date range associated with a time in which a later holder of that position would be extensively practicing that skill, which is why I classed it the sort of sentence with no natural meaning you'd only conceive in order to be deliberately ambiguous (or because you don't understand how to write English properly).

The original sentence isn't ideally constructed, but at least the president when JFK was born is a natural and easily identified sentence object, and it would be odd to ignore that in favour of an alternative interpretation that also involved interpreting "how tall" as being more likely to refer to the [probably-not-recorded-for-posterity] length of a newborn infant (who wouldn't be idiomatically referred to as "tall" or the president in the context of his birth) than the height of a person who was president in the relevant time period

> "who was the president when JFK was born?", the answer is clearly not JFK

Yes, you can deduce that, since the question is nonsensical (trivial) if "the president" refers to JFK in that context.

> If you flip it to "was JFK strong at writing essays when the president was a high school freshman"

This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.

> This is a perfect illustration of why the original sentence can be read as referring to JFK, since in speech it is common to have ill-defined references that are later clarified, and there's no formal rule that enforces that all terms must be defined before use.

References that are later clarified is exactly how we arrive at "the president when JFK was born" as defining Woodrow Wilson as the object of the sentence (and it makes much more sense to ask about how tall President Wilson was in the context of 1917 than to wonder how "tall" baby "president" JFK was in 1917).

Whereas "when the president was a high school freshman" can't be used to identify anybody else [unless there is a time period in which a high school freshman was the president!], even if the sentence hadn't already made it clear that JFK was the object of the sentence. But it's still an example of bad writing.

Even when there's no ambiguity, we don't do sentences like "The SpaceX founder was about 18 inches tall when Elon Musk was born" which is basically how you're implying the president question - which has a much more reasonable alternative interpretation - could be answered...

It is possible to write a truly ambiguous sentence in English; nobody is arguing otherwise. But whilst the original sentence might make you think a bit, it's not naturally interpreted as being equally/more likely to be a question about the not-tall not-president in the context of the time period instead of the person with height and presidency in the context of the time period, unless you're being obtuse or lack fluency in the language.

I wrote:

> "Was the president strong at writing essays when JFK was a high school freshman? Read this essay to find out!"

You write:

> "when the president was a high school freshman"

These are very different circumstances, of course.

If the original sentence had "reigning" before the word "president", then I would absolutely agree with the given interpretation.

I've noticed that Americans call ex-presidents "president", and ex-governors "governor", so that merely adds to the confusion. English is not my native language, and I'm not American, so perhaps I fall into the "lack fluency" category. Conferring with other native speakers nearby, every American immediately reads it as intended, whereas Brits and Canadians do not.

I conclude that the original sentence is poorly written, and that I misread it.

The fact JFK is being referred to as JFK in the same sentence as 'the president' is being referred to - rather than 'how tall was the president when he was born' or 'how tall was JFK when he was born' indicates they refer to different people. Therefore, 'the president' would most likely refer to whomever was the US president when JFK was born, though if the article referred to the president of some other organization before referring to JFK itt might not be the case.
As an non native english speaker i read:

> How tall was the president when JFK was born?

as, How tall was the "reigning" president, (which in the year 1917, Kennedys birth year, was Thomas Woodrow Wilson) on the day that JFK was born."

Let's talk about Biden, then. How tall was the president when he was born?
Ha! Perfect example!
It is ambiguous. Very ambiguous. You could reasonably ask: president of what?