I somehow thought early versions of the PDF spec were published as a .ps version for that very reason, but my duck-fu is failing me finding any such link. It may require wayback-fu and that's beyond my level-of-effort :-)
Actually a lot of the PDF format is plain text, but can contain binary streams. You can open a PDF in a text editor and see the header, and skip to the end and see the xref index and some other parts. The binary sections are enclosed in plain text start and end markers, but you probably won't be able to read much of the actual content this way since it will be compressed or encrypted.
It must have been hard guessing at the spec until you could read it properly.