| I built GuessGPT, a web game where you try to guess ChatGPT's answers to subjective questions like "What is the best movie ever?" and "What is the most delicious food in the world?"(no signup required) I was curious about how ChatGPT's "opinions" are formed and wanted to create a fun way to explore its internal knowledge. This project aims to give us a glimpse into how ChatGPT's responses are shaped. What's different: Unlike a trivia game, there are no right or wrong answers. It's about understanding how ChatGPT "thinks". The game gets more interesting as you start to notice patterns in its responses. Also, you will be able to compare your answers to others' soon, so we can get a sense of how ChatGpt's answers compare to the community. Try it out here: https://guessgpt.vercel.app Feedback Request: I'm looking for feedback to make GuessGPT even better. Feel free to share your thoughts and suggestions, especially on: What other questions should I add? What features would make the game more engaging? Did any of ChatGPT's answers surprise you? Technical Details: I obtained both the initial questions and the answers from the GPT-4o model via its API with temp=0 settings. The code is open source, updated link is on the game's website. Here is the prompt to get these answers: """You are a helpful assistant. Provide only a one entity answer. Do not offer explanations or context. The answer must be one entity if possible only one word. Do not use "unclear," "debatable, "subjective" or similar words. Give a definitive, one entity response.""" |
I feel like the “if possible only one word” may make things a bit weird due to no fuzzy matching. I put “Sherlock Holmes” for one thing, and the expected answer was “Sherlock”. Also I misspelled the name of a movie.
Maybe adding some sort of estimation of whether the user’s answer is probably referring to the same thing, could be good? Like, maybe give half a point if the user text has a lot of overlap with the target text, compared to the length of the texts, assuming the text lengths aren’t too different?