|
|
|
|
|
|
by cal85
646 days ago
|
|
|
Good points. But I feel like even with the cities article it could still ‘cheat’ by recognising what the data is supposed to be and filling in the blanks. Does it even need to be real though? What about generating a fake article to use as a test so it can’t possibly recognise the contents? You could even get GPT to generate it, just give it the ‘Largest cities’ HTML and tell it to output identical HTML but with all the names and statistics changed randomly. |
|
|
This isn't a good idea, if you want a fair test. See https://gwern.net/doc/reinforcement-learning/safe/2023-krako..., specifically https://arxiv.org/abs/1712.02950.