[mikeash]

"Strongly consider taking a backup of your current working directory and .git to avoid any possibility of losing data as a result of the use or misuse of these instructions."

WTF?

What is the point of a version control system if you have to take backups of it to avoid losing data when performing certain operations?

I use git, I like git, but certain aspects of it are fundamentally broken.

[gemma]

No, that advice from the article is fundamentally broken. Outside of the garbage collection system (which runs by default after what, 30 days? 90?), Git doesn't delete committed content. Any commit you "lose" through rebasing, amending, resetting, etc. can always be recovered. It's a little more complicated than renaming a directory, sure, but it's important, and it's not something a Git tutorial should ignore.

Git IS safe, and ANYTHING involving changes to history can be undone without resorting to backups. Data loss can occur when you're mucking about with uncommitted changes, but that's a risk in most other version control systems as well.

[prezjordan]

Surprised to see no one in the comments has mentioned the reflog [0]. It's really very easy.

[0]: http://jscal.es/2013/08/05/seriously-the-reflog-isnt-that-sc...

[crystaln]

I'm not 100% sure this is true, however it is also a fundamental flaw of git. There should be a way to remove commits permanently in order to remove mistakenly checked in large files or private content.

It's also definitely not true with uncommitted changes, including gitignored files.

[gemma]

I still don't see the "fundamental flaw". Non-reachable commits are automatically deleted by the garbage collection system, which can be also be run manually. Accidental commits with large files or private content can be "modified" (technically copied and rewritten, since individual commits are immutable) with rebase, amend, filter-branch, etc. Those operations make the original commits unreachable, so garbage collection takes care of deleting them.

And like I already said above, data loss can occur when you're working with uncommitted changes, just like in most other version control systems. If the content is not under version control (in this case, not in a git commit), it's not safe.

Honestly, you guys should go watch Linus Torvalds' presentation at Google about Git. The entire point, the massive problem he was trying to solve, was preservation and verification of data integrity.

[lomnakkus]

git filter-branch will let you remove content permanently and irrecoverably if you really need to.

Regarding uncommitted changes: This is in the same category as forgetting to do your backup before starting to mess around, IMO. I would encourage anyone to simply get used to committing extremely often and just using a quick interactive rebase before pushing.

[mtdewcmu]

At the last job where I used git, I'd work in a separate branch, and I started using `git merge --squash` to merge into the main branch to keep the history from getting too difficult to follow. When git merges a bunch of different histories into one, it becomes almost impossible to make sense of if people make lots of small commits. I shy away from `git rebase`, because it seems dangerous.

[lomnakkus]

Never fear! "git rebase" isn't nearly as dangerous as many have been led to believe... unless you start rebasing things you've already published/pushed elsewhere. In that case you need to be very proactive about notifying everyone who could possibly have checked out your branch, etc. Otherwise: it certainly takes a little getting used to, but I find a little one-on-one "mini-mentoring" others with the first few rebases helps them immensely, so if you have someone who can help you in person it might be a lot easier to get comfortable with the process that way.

[crazygringo]

I understand your puzzlement, I found this confusing too at first. But then I realized it makes sense -- one of git's strengths is that you can rewrite the history. The "point" of a version control system, at least with git, is not backup which retains all history, but rather versioning which retains the history you want to retain.

Obviously, if you choose not to edit the history, then you never need to back up in this sense, and you're free to do that. But then you can't ever go back and change things (like remove accidentally committed passwords, etc.)

But if you choose to rewrite the history, and mess up, then you'll be glad you had a backup. And (in response to other comments), even if there are ways of still retrieving/fixing data, it's often easier to just restore from your backup, especially when you're trying out git commands for the first time, and you're not entirely sure if they'll work exactly how you expect. None of us are git experts from the beginning, and I've resorted to git backups numerous times when trying out a command for the first time, and then discovering it wasn't the right way.

[simcop2387]

An easy way to do that, is the way I tend to do it; Create a new branch based off the one you're rewriting history in, and that will actually keep all of that for you even after you rewrite it all. Makes it really easy to restore later with git reset if you need it.

[Myrmornis]

Yes, this is how beginners should be taught to "back up" in git.

[simcop2387]

In the interest of making this easier:

    git branch branchname-backup
    #do dangerous stuff ...
    # whoops I just broke the branch really bad I'll start over
    git reset branchname-backup

[zimbatm]

Actually `git reflog` contains the HEAD history. Even after a rebase it's possible to checkout to an old commit (unless git has garbage-collected).

[_ikke_]

Git is quite safe, and most operations that involve doing things to history can be undone. Unsafe operations happen when the working tree and uncomitted changes are involved.

Also, sometimes it's easier for a user to roll back to an older back up than to untangle the mess they have created.

Third, git itself is not a backup. When your repository gets corrupted, you're out-of-luck when you don't have backups for those files. So it's still good to take backups of your repositories.

[mikeash]

First, your use of the word "most" is inherently incompatible with the phrase "quite safe".

Second, why would a version control system make it so difficult to roll back to an old version that it's easier to restore from backup? This is insane.

Third, I'm well aware of this, and of course you should be making backups of your git repositories (and everything else). But those backups should be there to protect against hardware failure and other external data-loss events, not protect against git itself.

[pyre]

You're discounting the idea that someone might want to destructively rewrite their history. Here's an example: What if you want to retain history, but remove a password that was hardcoded into a source file?

The simple options are:

- Remove the hard-coded password, and create a new repository with the current state of the code as a starting point.

- Start a new repository with the current code state, but keep the old repository around under lock-and-key, then perform 'complex' patch operations to move changes between the two repositories (e.g. roll back to a previous version of a file before the cut-off).

- Go back through your history, and manually create a new repository from each patch, but removing the password when you get to that commit.

If git always preserves all history, no matter what, then these are your only options.

While operations like `git-filter-branch` sound scary, they don't delete the commit objects from your .git folder. If you created a new branch called (e.g.) master-old because running `git-filter-branch` on your repository, then you can always 'rollback' to master-old if you end up in failure. Or slightly more complex, you could use the reference listing in the reflog to 'rollback' the changes.

[mtdewcmu]

It might be a better idea to just change the password.

[pyre]

There is a point past which you can be too pedantic and be confused with a troll. I think that you may be straddling that line.

Next time, rather than just assume that the poster isn't smart enough to realize that a compromised password should be changed, maybe you could take in the fact that it's probably just an example of data that you might want to extract from your history if it's automatically there. I can think of numerous scenarios where someone might want to remove a password from the history even if it's not compromised (e.g. want to publish a private repo).

[mtdewcmu]

I meant to raise the question of whether it's worth trying to expunge something sensitive from git. You'd have to track down all the clones of that repository. Even if you thought it was gone, it would be prudent to change the password anyway.

[mikeash]

I'm not discounting it, I simply don't agree with how git implements it.

IMO the correct option is to create a new repository that has the same history as the old repository minus the offending commit (or possibly with an edited version of that commit that leaves out the offending string).

Because it creates a new repository, there's no risk of data loss in your old repository. Once you're confident that the operation succeeded, you can swap them.

I haven't had to do this for a long time, but as I recall, this is basically how svn does it. It works fine.

The problem with git is that it makes this far too easy and it works by editing existing repositories rather than creating new ones. So instead of once-in-a-blue-moon repository hacking to get rid of that password you accidentally committed, you get people rewriting history because they think the real history isn't "clean". I know a lot of people who routinely edit their local history before pushing changes to a shared repository because they don't want other people to see their true "dirty" history. This is insane.

Finally, I'm confused about something, so maybe you could clear this up for me. I keep seeing assurances that 1) git does not actually destroy any data, and you can always recover if you screw up and 2) editing history is sometimes a vital necessity for cases like when you commit passwords. You yourself made these assurances in this comment. However, 1 and 2 are obviously mutually exclusive. If you can always recover then you can't actually scrub the repository of accidentally committed passwords and the like. Which one is actually true?

[rajivm]

Re: 1 and 2

1) This is almost true. Anything that is committed to Git is recoverable. When you "re-write" history, Git is creating a new set of commits in the history, an "alternate history path." It does not destroy the original commits, but there is no named reference to them (unless you created a branch/tag pointing to this line of commits).

2) In this case, if you want to actually destroy these unreferenced commits, you must run "git gc". This IS a destructive command. It will remove any unreferenced commits from the repository. (gc = garbage collect). If you never garbage collect, you will always have access to anything that was ever committed. It just might be hard to find since the only reference is the ref-log (if it was recent) or the commit hash.

[mikeash]

Since garbage collection does happen automatically after a while, it seems that the "doesn't destroy data" bit isn't completely true. But I understand that it's a fairly rare case where you're going to screw something up and then not bother to get it back until after garbage collection cleans it up.

Thanks for clarifying that.

[sofal]

I know a lot of people who routinely edit their local history before pushing changes to a shared repository because they don't want other people to see their true "dirty" history. This is insane.

This is no more insane than editing a source code file before you save it to the file system. Git is used as a development tool as well as version control, and developers are therefore encouraged to commit often, even if the code does not actually compile yet. There is no more need to fill the published history with all of these WIP commits than there is for me to know about every goddamn keystroke you made while you were dicking around with that config file.

[mtdewcmu]

Is the history stored as a text file somewhere that you can just edit? I sometimes wish git were a bit more transparent and less of a black box.

[phaemon]

You don't actually know how git implements it, so how can you disagree with it?

There is no such thing as "an edited version" of a commit. A commit is identified by a SHA1 hash of its index of contents. If you change one bit you get a new commit.

You're a C programmer, right? If someone gave you a specification for writing a program to implement git, without telling your what it was, you'd tell them it would take 2 weeks. And that's because you'd reckon it would take 2 hours to knock out a rough version and a couple of days to clean it up.

Seriously, it's that simple. Just go learn how it works.

[mikeash]

I understand how it works. Of course there's such thing as "an edited version" of a commit: it's a new commit that you create by taking an existing one and altering it. If you want to argue about terminology, please be my guest, but that's all your dispute is.

[nknighthb]

1) Why are you taking a be-careful-don't-blame-me passage from a random article written by some guy as gospel?

2) All version control systems are vulnerable to data loss if you mess around with them in unusual ways. Would you say svn was fundamentally broken if somebody told you to take a backup before you screwed around with the repo?

[mikeash]

1) It's the sort of thing I've heard many times from many people over the years.

2) The difference is that svn does not build this functionality into the main command line tool, and there is no culture of doing terrible things with svnadmin to edit svn repositories the way there is of doing terrible things with git to rewrite git history.

[nknighthb]

I can't agree that UI quibbles and your perception of the "culture" (a perception I don't share at all) are "fundamental flaws" in git.

[perlgeek]

On piece of data that is easy to lose, with any version control system I've worked with so far, is uncommitted data. And that's also the only data I've lost with git so far, after using it for several years. (And yes, it was my own stupidity, saying 'git checkout .' and only noticing later that there was something I wanted to keep).

The advise to take a backup doesn't hurt, and might be helpful if restoring the original state is more effort than doing it with git operations.

[oneandoneis2]

you don't need to make backups - git won't lose your data. I lost all hope that the article might be worth reading at that line.

[mateuszf]

Yep, that's true. Restoring data is as simple as checking latest changes using "git reflog"

[mtdewcmu]

I agree that git is both a great advance and seems fundamentally broken at the same time. One of git's advances is that it treats commits as snapshots of the entire tree rather than diffs[1]. A snapshot might as well be a tarball of the whole directory, except that git uses references to previous snapshots to store it efficiently. So in this aspect, git is like a backup tool plus compression. It's not quite a useful tool just for making compressed backups of source code, though, because data is buried in opaque internal files in the .git directory and can't be untangled from the commit history. You can't get at your data without going through git's tools, which means you might need to make your own backups in case git goes insane, and you can't use the backup functionality without creating indelible history.

I'm thinking that the repository could be moved out of the working directory and placed in its own file that's not invisible. If the repo was reified into a visible file, then repos would be portable and you could ftp them. The backup functionality could be separated from the history-tracking functionality, so you could make backups freely without adding noise to the commit history. A backup would basically be a tarball that you could append to a repo file, taking advantage of previous entries for compression. Commits, however they were implemented, could reference snapshots, but they needn't be 1:1.

[1] http://git-scm.com/book/ch1-3.html

[eru]

> I'm thinking that the repository could be moved out of the working directory and placed in its own file that's not invisible.

Symlinks are your friends.

> then repos would be portable and you could ftp them.

tar might come in handy.

> The backup functionality could be separated from the history-tracking functionality, so you could make backups freely without adding noise to the commit history. A backup would basically be a tarball that you could append to a repo file, taking advantage of previous entries for compression.

You can already do this. You can have commits without ancestors or descendants in your repository, and they will still benefit from delta compression.

[mtdewcmu]

The idea is to make it less monolithic. I'm digging more into the inner workings of git now. The functionality might already be in there, just not obvious.

[mcv]

Backups are of course always a good idea, but you don't need them specifically to work with git. Git is its own backup system. If you think you might do something potentially harmful, do it in a new branch. If something goes wrong, you can always throw it away.

If something has already gone wrong, and you didn't do it in a separate branch, you can still go back to a previous situation.

Rewriting history in any serious sense (beyond a local reset or rebase for stuff that hasn't been pushed to anyone else yet) is always a bad idea. History is history for a good reason.

Of course any existing commit can always be reverted; that's not rewriting history. A revert is simply a new commit.

[Estragon]

It makes it fast and easy to back out if you screw anything up. Even if the data is still there, it can be complex to pull it back out and configure it the way it was when you started (as the commands in this tutorial demonstrate.) So a fast, easy snapshot before executing complex commands is a smart move.

[zimbatm]

git is safe but you have to know all the fancy commands like `git reflog`. I remember being puzzled by a merge conflict when I started learning git. I didn't know what it was and `git reset` or `git revert` weren't doing what I expected. All I wanted was to go back to the previous state. In the end it was easier to clone the repo and start over again.

[mtdewcmu]

You probably wanted `git merge --abort`. It's not very clear what the various states are that git can be in. There seems to be a 'fixing merge conflict' state, and it's hard to find documentation that warns you about this state and what your options are once you're in it.