Hacker News new | ask | show | jobs
by john_horton 1249 days ago
oh hey - it's my paper! If anyone is interested in exploring these ideas, feel free to get in touch (@johnjhorton, https://www.john-joseph-horton.com/). FWIW - I think it would be really neat to build a Python library w/ some tools for constructing & running experiments of different kinds. I think the paper only scratches surface of what's possible (esp. once GPT4-ish has an API).
2 comments
doctoboggan 1249 days ago
This is really cool. I had a similar thought that GPT3 could be used to simulate political polling. A few weeks ago I tried telling GPT3 that it was part of a specific demographic (age, gender, race, income, political leaning, etc) and then asked it how it would respond to certain political questions (I tried gun control, immigration, abortion and some other issues). GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be.

My thinking was that this could be used as a quick polling test to see how the real population may respond to certain new ideas.

More work would need to be done to calibrate it, as without specific demographic details the answers tended to be liberal leaning. But its an interesting idea which could be used to create instant focus tests on any number of topics.

link
capitol_ 1249 days ago
> GPT3 was able to change its answers in believable ways depending on what demographic I instructed it to be.

Doesn't this just mean that your own preconceptions about those demographics matches the language models preconceptions? How would we know that is matches reality when presented with novel ideas/concepts that we want to get feedback on?

link
doctoboggan 1249 days ago
That's part of what I mean by it needing to be calibrated. Initially some polling could be done with real people and the GPT agents. Whatever calibration factors are needed to make those two line up could then be used when asking the GPT agents novel questions.
link
visarga 1249 days ago
> I had a similar thought that GPT3 could be used to simulate political polling.

There's a good paper for that task. See my other post: https://news.ycombinator.com/item?id=34385489

link
anigbrowl 1249 days ago
You might like to include this experiment in future research: https://www.science.org/doi/10.1126/sciadv.1600451
link
john_horton 1249 days ago
that's a great suggestion - thanks!
link