I know that PBKDF1 is deprecated, but using SHA1 is an upgrade to PBKDF1 which was once the standard. I'm not sure if "extremely unsafe" is the right level of worry. You can still do iterations to make the hashing slower, if for some reason you don't do the right thing and use the methods you listed.
Is there a good way to 2 way encrypt something in something like rails where the app needs to use the decrypted token. If you have app logic doing decryption then someone can just look at the source to figure out how to do this. Is the only good solution to use a compiled language?
Compiled programs are as easy to read the source for (at least the relevant bits) as interpreted ones. Still, if you encrypt it and someone gains access only to the database, they wouldn't be able to use the tokens.
Hashes like SHA1 and MD5 are fast. This makes them great for things like verifying file contents... You can hash a lot of data and get your answer very quickly.
This is exactly why they are not particularly well suited to passwords. A brute-force attack (since you're salting your passwords I'm sure!) against them isn't a huge undertaking. Just with the equipment in the computer I'm on right now, I'm looking at generating about 11.5b MD5 hashes per second, or 3.1b SHA1 hashes per second.
At eight characters a-zA-Z0-9, I'm looking at about 5 and a half minutes to brute force every combination with MD5. Under a day for SHA1.
Hashes like bcrypt and scrypt, on the other hand, are designed to be slow. Their complexity factors actually provide means to slow the hashes down even further as hardware becomes faster. Instead of 11.5b/second, you can increase the complexity until you're only able to generate one hash per second... Now it takes you over three and a half centuries to hash what might take one second with MD5.
Even ignoring the possibilities of MD5/SHA1 being 'broken', they're simply too fast to be considered for hashing passwords.
But it appears that almost all user passwords (99.8%) appear in the top 10,000 list [1]. So even a brute-force attack on a slow hash like bcrypt is pretty cheap in the vast majority of cases. So switching from md5 to bcrypt doesn't improve your security much.
According to that one guy with that one list. I acknowledge that the top N passwords are X% of all user passwords. I sincerely question the 99.8% figure. The problem with doing studies like this is while we have some fairly big password dumps we still do not have the universe. Furthermore, there are some number of un-cracked passwords in the dumps we have. Further complicating the situation are password policies which may reject common passwords. It has been many years since we learned that "password" is the most common password and is commonly disallowed.
Therefore 1) it is not useless to not increase your storage security even if Y% of your users use bad passwords as you are protecting 100%-Y% of you users. 2) Y% is probably not 99.8% for you, and if you are worried about it you can take steps to mitigate the problem.
ps. He is ignoring punctuation which is an important detail for actually doing the cracking.
pps. I appreciate the sentiment (users choose shitty passwords) but not the conclusion (so don't bother storing them well). The proper conclusion is use scrypt/bcrypt and increase the work factor. You can take reasonable steps to protect your users and you should.
Yes the figure of 99.8% does seem a little high. After a bit more research it seems Mr Burnett himself can see 'a few flaws' with that figure [1].
Just to clarify: my original point wasn't that you should continue using md5. Rather, it was that bcrypt doesn't improve your security much. Given the problems with the 99.8% figure, it would be better to say, "the extra security that bcrypt provides might be less than you expect".