[tptacek]

Running a crawler and poking around API artifacts manually simply aren't the same thing under the law, even though on the wire what's happening is the same. As long as your crawler isn't programmed to go looking for SQL injection vulnerabilities or whatever, there's no case to be made that you had any intent to gain unauthorized access. That's what matters here: your intent.