Definitely be sure to check the length though. Imagine a mistaken client trying to send HTTP, but of course the first four bytes "HTTP" when interpreted as a 32-bit integer, whichever endian, is an absurdly large buffer.
I mean, at this point you're effectively defining a new network protocol (or, you will be shortly once you implement ways to work around all the other issues you're going to run into). I'd go all-in from the start and start every packet with a magic string/byte sequence of your own, a length, and probably a version code just to make it extensible.
Or see if there's an existing protocol you can abuse for what you want. If it's transactional, you get a pretty big ecosystem of battle-tested clients/servers/proxies/etc if you use HTTP.
A 16-bit length (64k max message size) is usually sufficient, or even 24-bit (16M max) if you really feel the need, but 32 bits is far more than should be needed for parsing messages in memory; it would be fine for a streaming application, however (in which case a 64-bit length wouldn't be a bad idea either.)
Or see if there's an existing protocol you can abuse for what you want. If it's transactional, you get a pretty big ecosystem of battle-tested clients/servers/proxies/etc if you use HTTP.