[c3pa]

> Our algorithms picked up these red flags and more, and assessed Sino-Forest as high risk when we ran our models on the company’s historical filings.

When you're backtesting your models, how do you distinguish between novel fraud that the industry is _now_ aware of vs. fraud that was visible but ignored -- if your model has learned _from_ Sino-forest, how do you know it would have caught Sino-Forest at the time?

> For instance, this sentence sounds like it could be indicative of terrible things going on behind the scenes but is in fact, just boilerplate disclosure: “We face risks and uncertainties related to litigation, regulatory actions and government investigations and inquiries.” You can see how ML models easily get confused.

Humans too, you're describing at least one comment in every HN thread about an SEC filing :D

[kbennatti]

Great q. Sino-Forest is an out-of-sample test so our models didn't technically "learn" from it. That said, very valid comment. Historical testing only goes so far. Assessing whether our algorithms work in deployment has been cool. Check out some of our live, in deployment examples here - https://bedrock.substack.com/p/bedrock-ai-vs-activist-shorts

[naturalauction]

>https://bedrock.substack.com/p/bedrock-ai-vs-activist-shorts

How often are companies rated with a risk factor this high. As in does a risk factor in the 80s mean that fraud is extremely likely or is it just notifying humans that this filing might be worth reading over with a fine-toothed comb.

[piesauce]

Less than 10 percent of companies have a risk score above 80. Our historical testing shows that around 1/3rd of companies with scores above 80 turn out to be fraudulent. (It is hard to test this so this might be an overestimate)

[frijole]

Just about every company mentioned in the article increased in value near and/or after the time of those reports. Pretty interesting.

Great work!

[kbennatti]

> Humans too, you're describing at least one comment in every HN thread about an SEC filing :D

LOL. Yup!