[0xab]

> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image.

Yeah, that's exactly what our paper said 5 years ago!

They didn't even cite us :(

"Measuring Social Biases in Grounded Vision and Language Embeddings" https://arxiv.org/pdf/2002.08911

[hkmaxpro]

I think social biases (e.g. angry black women stereotype) in your paper is different from cognitive biases about facts (e.g. number of legs, whether lines are parallel) that OP is about.

Social biases are subjective. Facts are not.

[rcxdude]

As far as the model's concerned, there's not much difference. Social biases will tend to show up objectively in the training data because the training data is influenced by those biases (the same thing happens with humans, which how these biases can proliferate and persist).

[vokhanhan25]

I see a clear difference. One is objective (only one correct answer), one is subjective (multiple plausible answers)

[EvgeniyZh]

Well you send a vaguely worded email like "I think you may find our work relevant" and everyone knows what that means and adds the citation

[anguyen8]

Hello 0xab,

Sorry that we missed your work. There are a lot of works in this area both textual and visual, especially social biases.

We wish to mention all but the space is limited so one can often discuss the most relevant ones. We'll consider discussing yours in our next revision.

Genuine question: Would you categorize the type of bias in our work "social"?

[3abiton]

It's easier to succeed if you ignore the issues, andthe users are not aware of it.the rate of evolution of "AI" recently is so fast, no one is stopping to do actual benchmarks and analysis of allyhe new models.

[moralestapia]

That's weird, you're at MIT. You're in the circle of people that's allowed to succeed.

I wouldn't think much about it, as it was probably a genuine mistake.

[JackYoustra]

What does allowed to succeed mean?

[moralestapia]

Your work usually has 1,000x the exposure and external validation compared to doing it outside those environments, where it would just get discarded and ignored.

Not a complain, though. It's a requirement for our world to be the way it is.

[_345]

Is there truth to this? Do you have any sources to link to on this

[moralestapia]

Sure dude, here's the link to the UN Resolution about which researchers deserve attention and which others do not, signed by all countries around the world [1].

*sigh*

It's pretty obvious, if you publish something at Harvard, MIT, et. al. you even get a dedicated PR team to make your research stand out.

If you publish that on your own, or on some small research university in Namibia, no one will notice.

I might be lying, though, 'cause there's no "proof".

1: https://tinyurl.com/3uf7r5r7

[ramblerman]

What do you genuinely think they built upon from your paper?

If anything, the presentation of their results in such an accessible format next to the paper should be commended.