[ebeip90]

I strongly recommend checking out 010 Editor [1].

It's very powerful, lets you display in multiple different formats (not just 1/2/4/8 bytes, but interlaced formats and byte-arrays) and has the most amazing templating / scripting engine I've seen for this type of tool.

The only caveat is that it isn't free, but if this is something you do for a living (as I do) it's an indispensable tool for exploring file formats and other binary data sources.

[1]: https://www.sweetscape.com/010editor/

[mockery]

Yes! The 010 Editor's templating language is a nearly unique (in my experience) hybrid between C-like declarations and imperative flow that's exactly what I want for parsing arbitrary binary files.

The template executes like a script (with conditionals and looping) and a line like "int32 myNum;" is actually just syntactic sugar for "read 4 bytes from the current file pointer and label it myNum." Their example probably does a better job explaining it than I can:

  struct FILE {
      struct HEADER {
   char    type[4];
          int     version;
          ushort  numRecords;
      } header <bgcolor=cLtGray>;
  
      struct RECORD {
          int     employeeId;
          char    name[40];
          float salary;
          if( file.header.version > 1 )
              int      numChildren;
          if( file.header.version > 2 )
              time_t   birthDate;
      } record[ file.header.numRecords ];
  
  } file;

( From: https://www.sweetscape.com/010editor/templates.html )

[mkroman]

Yep, same!

010 Editor is one of the few commercial tools that I just have to have a license for - I don't use it to make a living, but I've participated in some CTFs and challenges, and just yesterday I noticed that a WAV file had metadata which I didn't know was a thing, so I just opened it in 010 Editor with the WAV template and I could see how the metadata was stored, and all other fields!

[jahewson]

If you’re on the Mac, Synalyze It! is somewhat similar.

https://www.synalysis.net/

[radicality]

I used this for viewing mysql binary logs recently to understand the format. I was upset when I realized the $10 version was severely nerfed. With lack of scripting I couldn't write even a basic grammar to parse out the binlog file. At that point I was too annoyed to shell out another $40 for the pro version.

If you are considering this, get the Pro version. I would recommend to the owner to just make the non-pro version free.

[TomVDB]

What's the lowest level of granularity? Byte level?

Because I'd love templating feature like that works at the bit level.

IOW: you can specify some a bunch of fields to be, say, 3 bits each, and that when all the bits of a byte are used, it simply spills over to the next byte.

[oauea]

https://www.sweetscape.com/010editor/manual/Bitfields.htm

[kop316]

Serious question, is it really worth it for $130?

[mkroman]

The next time you want to dwell into a file format, download it and do the 30 day trial - maybe there's already an existing template for the file you're looking for - if not, try making your own - it's super easy! Then you can decide if it's worth it. If such a situation never arises, then no, it's not worth it in your case.

I used it a lot when datamining in World of Warcraft - I would find reverse engineered documentation from previous versions of the game, write it into a 010 editor template struct, run it on the DBC file and then I'd usually get a result where the header and the first few fields would be fine, but the rest would be completely wrong. All I did was add a field in the template before the first obviously wrong field, with different, random sizes, to add random padding until the field started looking correct again.

It's a pretty specific case, but it almost always worked!

[_bz2r]

Okteta is part of the KDE project, GPL, and has similar templating functionality

[rootw0rm]

it's also my hex editor of choice, has been for years