If you prefer to ship fragile binary file formats for no reason other than finding it too onerous to define something as trivial as an eight-byte file header, that’s silly, but nobody is going to stop you.
If you’re going to use text-based serialization formats as your justification for the decision, however, I’d suggest you look into all the fun bugs, security issues, and weird edges cases that arise from parsers having to make a best guess at character encoding and file format when all you have to work with is the file extension, maybe a byte order mark, and heuristics over the file contents.
The magic means it’s possible to identify the file type. Maybe you’ll add a tool later that operates on multiple types of files.
The version means you can evolve the contents in non-backwards-compatible ways, while maintaining the ability to read/parse the old version.
It’s a pain in the ass to add a magic or version number later; there’s a reason why nearly every file format on the planet has both.