[superb-owl]

I had great plans to feed Erowid's database through Claude to get a better classification of drug phenomenology. Sadly they have explicitly disallowed the feeding of Erowid reports into LLMs: https://erowid.org/experiences/email_warning.php

I appreciate the stand they're taking, but the potential for greater understanding and harm reduction seems to outweigh any potential downsides of putting public webpages into an LLM.

[poincaredisk]

Commendable stance from you. It's sad that big tech won't care and will index their content anyway. I wish terms of use like this were enforcable.

[astrange]

They are, via CFAA. It depends what their robots.txt is set to or the AI version of that.

Anyway, the influence of random web text on AI is overrated. They're going to filter out pages that don't contribute, and bad words/topics/personal info will get it removed.

[ziddoap]

Might be worth reaching out to them, assuming you meet their criteria of "researcher".

>The Erowids state on their website that researchers cannot “mine” data from their site but that they’re open to discussing projects with researchers, provided they’re properly credited and cited.