[buzzm]

:-) You visited the site so props for that. But I find it odd you call the soup of braces, colons, and quotes a complete mess. That's JSON. As in industry standard everyone-uses-it JSON. If you're not familiar with the space I can see why PGN has more human readable appeal.

[SideQuark]

I've written some of the high perf JSON code used in industry, so I am well aware of it. A main reason companies (and me, and many others) work so hard on making high speed readers is to get them withing a fraction of the speed of reading many native formats.

For example, converting a 32 bit float to text for JSON, then parsing it back in later, is orders of magnitude slower than simply using the binary one. And in many languages it's not bit exact, so it creates a giant mess. I've worked on getting many projects that try to roundtrip floats correctly since the JSON layer was breaking projects (hint: C++ has precisely one, corner and rarely used, format/parse that promises to roundtrip, and this is not even guaranteed across compilers or systems or versions - only one compiler version. To do better you import your own Grisu or successor and implement tests to catch system errors.... or switch to hex format, which is much nicer). So yes, I have done tons of JSON, but I get called mostly to fix/remove/mitigate JSON since groups and people know I am the guy that fixes idiot-added JSON where it breaks things.

Next, JSON is also an order of magnitude bigger than many other formats. There's good reason AI models are not JSON, video is not JSON, etc. JSON slows down major pipelines because of the size increases and slowness of formatting and parsing compared to simply using a smarter format. Junior engineers love JSON because they don't understand the pros/cons and can jam it everywhere. Many projects I've been called in to speed up were slow due to "JSON all the things!".

Next, I've also written high performance chess engines (and all sorts of custom searching engines to do stuff like find proof games, do various things related to retrograde analysis, and construct richer data for endgame tables for my own analysis, some published...), with all the bells and whistles and bit tricks and used things like Z3 to invent better magic bitboard constants, which also improved state of the art there. So I also know that space.

Naming your format BGN for "Better Game" Notation (?) to replace PGN, when your format breaks pretty much every thing PGN was designed to be used for, and is widely used for, is simply ludicrous. It's not a better game notation - it's an unusable game notation. You want it to be a searchable, queryable format, of which many already exist, yet again yours is going to be orders of magnitude slower since you chose JSON, replacing at most bytes of chess move info with variable length (thus ensuring you cannot random access information but must parse the 1000+ characters before move 50 to see move 50...) 25+ to ~40 bytes each. Your format is simply a terrible data format, solving nothing.

And most big engines and certainly all big chess database software support all the major formats.

Did you look at either the motivation for PGN or for existing chess query supporting database format before you made this? Or in the intervening 5 years?

So I don't think you have improved anything over the previous technology. There's a reason you have zero downloads over 5 years.

Here's a hint: if you're going to make something claiming to be a better X, then you should reproduce what X was designed for, and then, and only then, add features.

> As in industry standard everyone-uses-it JSON

That's PGN. As in industry standard everyone-uses-it PGN. If you're not familiar with the space I can see why JSON has more human unreadable appeal.

Good luck getting to 1 download.

[buzzm]

UGN -- unusable game notation. That has a ring to it.

UGN was designed with SPARK and AWS S3 pbzip2-aware threading JSON readers in mind. For an analysis of a year's worth of lichess games -- over a billion, maybe more -- I believe UGN is a good solution. Try bucketing clock time vs. blunders or castling analysis or opening vs. ELO on 1 billion PGNs. Not simple search; aggregation. In the end, UGN is an information architecture; JSON is used as a convenient implementation/representation but it happens work well with SPARK and S3. And you can reliably use jq or even grep on the same UGN files. I didn't feel the need to create yet another binary format.

Anyhoo.... We're up to 60 downloads!

[SideQuark]

> Try bucketing clock time vs. blunders or castling analysis or opening vs. ELO on 1 billion PGNs

No one uses PGN for that. Strawmen are easy to defeat.

Heck, I can crush your example by simply using a proper binary format, fixed length records, and reading flat files directly into memory, all for about 1 hour of coding to transcode your format into something designed for analysis.

Better yet, compare your examples to any of the current best chess DB formats. They all support all your query examples a massively more. And are properly designed, tested and improved over decades of actual use, by professionals and researchers alike.

People have been doing billion+ chess game analyses for well over a decade. Chess.com added 1 billion games in Feb alone two years ago , they just passed 100 billion games total in Jan, they regularly run all sorts of analyses on all the games, and provide datasets and services to researchers for similar work.

It's hard to improve on what you do not understand. If you spent some time understanding the space you may be able to improve on it. Until then a lone wolf, unaware of what is in the space, will simply de-invent a worse wheel.