[supermatt]

> When we ran the full dataset through the deep scan, it caught every single confirmed phishing site with zero false negatives. The tradeoff is that it flagged all 9 of the legitimate sites in our dataset as suspicious

Huh? Does this mean it just flagged everything as suspicious?

[john_strinlai]

indeed... it seems like it just says everything is phishing... which they go on to say is desirable?

"The tradeoff is that it flagged all 9 of the legitimate sites in our dataset as suspicious, which is worth it when you're actively investigating a link you don't trust."

so, you dont really need the scanning product at all. if you just assume every website is a phishing website, you will have the same performance as the scanner!

[jdup7]

Yeah probably could have done better at describing the methodology. The dataset is just the confirmed (manually by a human) phishing urls. We only included the FPs to show that the tooling isn't perfect there were many TNs that we did not include. Going forward we could definitely frame these results better.

[badgersnake]

lol, return false;