How does one make sure that their code isn't used in a closed source project though? What's to stop them from using it if no outsider will see their code?
You can't, if the code is hosted separately from the tool.
The solution is to provide the tool and the hosting together. For example, GitHub or Bitbucket could acquire this service and add it to their offerings, and it would fit right in with their existing business model.
I think you're getting even further away from open source.
If this tool could connect to a remote repo (like the GitHub repo) and operate on the changelog, then you'd be fine. No one would be able to use it privately without publicly exposing their repo.
Perhaps then you could charge a fee to get an ssh key to add to you authorized_keys file, which would allow for private use.
The solution is to provide the tool and the hosting together. For example, GitHub or Bitbucket could acquire this service and add it to their offerings, and it would fit right in with their existing business model.