Hacker News new | ask | show | jobs
by jvehent 3415 days ago
Too many comments here recommend to clean up the commit and just hide the mistake under the rug. This is wrong.

If you leak a password to any public location, there is only one reasonable course of action: CHANGE IT!

Don't even bother rewriting the commit. Focus on changing that password right away, and while you're at it, figure out a better way to manage your secrets outside of your source code in the future. Mistakes happen, but they shouldn't be repeated.

8 comments

the solution is to store the password and any other sensitive information in a text file that you read when your program starts up. And don't forget to add that file's name to .gitignore so git will ignore it. As simple as that. :)

If you leaked the password in the git repository, change it as @jvehent just commented.

I think its as important to make it "hard to do the wrong thing" as "easy to do the right thing". In this case having to explicitly exclude a file containing passwords from being deployed would fail that rule of thumb.

The Azure Key Vault is a good solution that so far seems easy to work with (I've only just started using it though) and it can make the storage of secrets easier to secure but you still have the issue of storing the credentials to the key vault, so its difficult to fully get rid of the problem although you can definitely isolate access to your passwords that way and then have a more limited number of credentials to secure.

Somehow I doubt Azure Key Vault is easier than a gitignore line and a text file
Probably not, but it's a lot easier to accidentally commit (or otherwise expose) a text file.
Its not easier for sure, but it is more reliable and harder to mess up by making a single character mistake.
.gitignore is easy but the pw is still naked. Probably a good practice to hash it.
Probably not as the result would be useless for authenticating against another service.
As you say, the Azure Key Vault helps making things more secure (by allowing to control, log and revoke keys usage), but it does not help at all with the problem of API keys in the source code - it's just another set of keys that you need in your config.
The way we do it for azure app services is to store most keys in the "application settings" section of the portal so most deployed connection strings, api keys, oauth2 credentials etc only exist in config files for local development and for everything else they are defined in the portal. You can lock down access to the portal and app services to only people who will be managing them (in a larger shop than us, dev ops) so developers wouldn't even have the keys to the kingdom. There are probably better ways to do this, but it has gotten us away from keys in config files.
Sometimes! (Full disclosure: MS employee, big-time Azure user in a different group)

An AAD Application can act on behalf of a logged-in user (using OAuth2 or openid) with the correct delegated permissions. This means you can grant key/secret/certificate CRUD privileges to an AAD user or group, and then use OAuth to obtain a token granting access to the KeyVault resource. All activity is performed by the client (read: application) on behalf of the user (read: human) against the resource (read: key) without having to store any secrets at all.

I use the keyvault pretty extensively, and have really grown to like it.

It can be slightly more complex than that if you use a package manager to publish your work. I once accidentally leaked a password due to having both a .gitignore and an .npmignore, and forgetting to include my .env file in the latter. Fortunately, I realized what had happened almost immediately and was able to change the password. Now I tend to `tar tf` everything before publishing.
Because of issues like this I tend to set up my deploy scripts to first `git clone` into a temporary directory, then do the rest of the work from there.
I believe that it's better to not have that kind of sensitive files be dotfiles.
Shamless plug: SecureStore, our .NET secrets manager: https://neosmart.net/blog/2017/securestore-a-net-secrets-man...

I'm drafting a writeup and will post it to HN when that's ready. Other secrets managers I've seen posted to HN seem far too overcomplicated, at least for our company's needs. This is a step up from reading secrets from a plain text file, but not so complicated that you need a separate docker image running a service dealing out passwords to your webapps or similar.

EDIT: rationale for use, as requested:

Using this approach, you can better manage your secrets since you can actually commit the passwords file in your code base, and the API lends itself to easily switching between dev and production ids/secrets. You can track revisions to the secrets file, rolling back your commits rolls back the secrets as well. You can also include the deployment of secrets in your deployment script - typically, the encryption key for your vault is only generated and distributed to the production servers once, while secrets may be added, changed, and removed continuously during the development and product lifecycles.

Using a simple secrets manager like NeoSmart's SecureStore lets you embrace the benefits of deployment automation, revision control, and more, without sacrificing safety and security in the name of productivity or ease-of-use.

EDIT2: what the heck, just took 10 minutes off to write it up and publish it: https://news.ycombinator.com/item?id=13654005

What hashing algorithms are you using? Are you updating that hashing algorithms as cipher-suites are being broken? Are we as a user required to set a random seed? Just because it is encrypted doesn't mean it is impossible to decrypt. Especially if default settings in the hash are being used and someone uses a poor password found in another breach. Its a great idea, but I would still worry about security issues around publishing a password hash.
High level APIs are used for all crypto. The user does not have to generate their own seed. You can use your own key file or stretch a secret phrase once to create one with the library. Passwords are otherwise not hashed, only encrypted.
I think you need to add a sentence giving one good reason to use anything other than a plain text file.
Thanks, good idea.

I just did, though I may have gone overboard as it is more of a paragraph than a sentence. We developed SecureStore out of necessity, believe me, KISS all the way.

The fundamental trade-off for any new tool is that devs have finite capacity to learn new tools. That's why we like to learn a few tools really well and then reuse the heck out of them. However, we are open to warning-stories - that if you do it the naive way, at some point you'll get bitten by X, Y and Z. And even then we might not acre until we actually get bitten!
Agreed that you absolutely cannot store sensitive passwords in your source code repo.

Your proposed solution, however, has its own share of problems for some deployment scenarios. Where do you get this file from? Assuming you are meant to place it by hand each time you deploy your application... what about autoscaling? What if you want unattended deployment of apps?

There are lots of ways to handle those scenarios.

Deciding where to store your secrets is extra easy if you're in the cloud. In AWS you can use KMS to store it if it's 4kb or less. A cli command or API call can decrypt it for you. If it's larger, you can use a tool such as credstash which lets KMS manage the keys.

If you're in an environment that's using Chef, it can handle them. Ansible has a solution as well.

Or you can use something like Hashicorp Vault, though it requires setting up servers for that purpose.

Once you've decided on one of these tools, it's no problem to script the retrieval of a secret into your deployment mechanism. It will work fine for autoscaling or any other unattended deployment.

Oh, yes, I didn't mean there wasn't a solution. We use Hashicorp Vault, for example. I simply meant that "store passwords in a file" (as mentioned in the post I replied to) is too simplistic to cover all scenarios.
Files are good because virtually anything can read and write them, so they are very portable. I used to work with a really great security guy who created a good system for handling secrets. He had us write them to files, but only to a tmpfs mount, so that they were written to memory and not disk. We also didn't write them to regular files, we used named pipes. This way the application on initialization would read a secret from the pipe and would block waiting for it to be available if it hadn't been written yet. There was a separate process in place to handle the retrieval of the secret and the writing of it to the pipe. As soon as it was written, the application would finish reading and continue its initialization. This ensured things happened in the correct order, and also made the file to be one-time-use.
Kms doesn't have a size limit if used right. You should use kms to store a key and store the data on s3 encrypted.
Sure, which is why I said to use e.g. credstash in such cases. It stores the secrets in DynamoDB while using KMS to handle the keys. I guess you are talking about using S3 server side encryption, which is another approach.
No I'm literally talking about taking the row key and using it as the lookup from kms for the crypto key. Then you take the plaintext, the crypto key, encrypt the plain text and store it whereever. It's 1 additional aws api call over storing the stuff unencrypted, and about 5 lines of code in java that can be turned into a 1 line library call. Not sure why you need credstash.
+1 Upvote for credstash
I'm curious of cases for when people are running into size limitations for storing secrets... what type of secrets are > 4kb? I could imagine some example but I'm wondering about real world examples...
Every file or message you want to send encrypted through AWS or store permanently in S3. I often use crypto as signing as well, since it mostly comes for free code wise at my job.
You totally can store sensitive passwords in your source code repo, just so long as they're encrypted. Here's how we do it: https://neosmart.net/blog/2017/securestore-a-net-secrets-man...

SecureStore is designed to be repo-friendly, it purposely avoids needless IV/payload regeneration, is based in plain text, and preserves element order to avoid driving source code managers crazy.

Or environmental variables loaded at provisioning though this just moves the target. But these options do not themselves have the "in motion" characteristics that provide the best reliability, scalability and security.

If your needs are more sophisticated this means keyservers where hardware security module (HSM) might by part of the equation.

Agreed that is good practice. Oftentimes though when you're prototyping something you aren't really thinking that much about the structure of the project, and then you end up accidentally committing passwords. But yeah, would be a good practice to have this deeply ingrained and just do it automatically for every similar situation.
Completely agree. There are forks, mirrors and crawlers on GitHub, even you rewrite the commit and force push to GitHub server, the original commit data still exists in the forks and mirrors, and in fact anyone can even view the original commit in your own repo if they know its commit hash.
Very much this.

The first thing you should be doing is making the password useless by changing it. Doing anything else is entirely irresponsible. Sure, remove the file in question after that... but you can't treat the old password as anything other than public knowledge at that point.

I've only ever leaked a webhook, realised minutes later, and then changed the webhook URL on the backend. It's not hard to do, and doing anything else is simply really crappy security through obscurity while hoping for the best.
Why would a webhook URL be a secret? Wouldn't it be more like internal API if anything?

I would assume that the parameters sent to the webhook, an auth token or something of the sort would take care of the security bit. Obscuring the URL seems like security-by-obscurity no?

For purposes of security, there's no difference between example.com/api/my-webhook?auth-token=[some-uuid] and example.com/api/my-webhook/[some-uuid].
Not if you treat the secret URL like a password. Plus, not all webhook callers allow you to authenticate them without supplying them a special URL.
But what if your codebase is used in thousands of places that you don't control? You can't always change it.

The real lesson is - don't put passwords in your code.

Don't use passwords/secrets/credentials that you can't rotate. If you've created a product in such a way that you can't rotate secrets, you have a large security issue that you should fix ASAP.

It's like someone responding to the suggestion to "use strong/unique passwords" with "but what if I don't have any authentication?"

Using the same password in thounsands of places isn't good either. Use unique random passwords.
Blackbox is one good way to store secrets: https://github.com/StackExchange/blackbox
Reminds me of the Bitcoinica fiasco.

Stuff happens, especially under pressure. But yeah, if that happens to you, there is no more reasonable course of action than changing it right away.

100% agree with this. Not worth the hassle to do anything else.