[amluto]

Why would one expect ChatGPT to know the answer to this question? ChatGPT “knows” what it was trained on. The training data is unlikely to include a definitive answer to your question. And ChatGPT is not currently smart enough to do the kind of analysis that would determine the answer, nor is it likely to be able to do the kind of queries that would be needed to figure it out.

[gtirloni]

The training data could include internal docs that describe how it ignores or not the robots.txt file.

[amluto]

If I were involved at OpenAI, I would not include the internal wiki, Slack archives, Dropbox folders, etc in the training data. While it would be highly entertaining, it would not be a good idea.

[iKevinShah]

I agree on that - that private data (in a best case scenario) should not and would not be included in the training but there would be some parts of internal documents which would be public (lets say public website) - It is expected that chatGPT would know at least those ..