I wouldn't say I want to abandon anything git is doing as much as evolve it. Objects need to be able to contain syntax tree nodes, and patches need to be able to target changes to particular locations in a syntax tree instead of just by line/col.
Right, but for what purpose? I don't see much gain, and now you're left trying to fit a square peg into a round hole. The git CLI would be technically working, but not practically useful. Same with an IDE: if you checked the files out you could technically open them but not easily change your program.
You could also think of it this way: if I abuse git's protocols to do my thing, I pretty much give up my chance to become a standard in the future. If people are going to adopt a new standard they want it to be clean: you don't want to be trying to sell a greenfield technology that's loaded up with legacy tech debt. Making a data format that renders broken in every existing tool that touches it would be showing a lot of bad faith up front.