Hacker News new | ask | show | jobs
by Drdrdrq 2752 days ago
It is. But it's a problem of JSON itself, not just jq.
2 comments

TIL: JSON has no specified number implementation: http://www.ecma-international.org/publications/files/ECMA-ST...

>JSON is agnostic about the semantics of numbers ... JSON instead offers only the representation of numbers that humans use: a sequence of digits.

So... anything is valid, per the spec.

JSON != JavaScript

> echo 1152921504606846976 | python -c 'import sys, json; print(json.load(sys.stdin))'

1152921504606846976

Python's json package != JSON

JSON: https://tools.ietf.org/html/rfc8259#page-8

The link says that it's up to the implementation, which means it's valid for Python's JSON implementation to support larger numbers.

It's less "interoperable" but not strictly invalid, by my read.

What the GP means is that JSON doesn't require an implementation to decode JSON integers as arbitrary-precision integers, to be "conformant JSON."

Therefore, you can't assume that if you pass some JSON through an arbitrary pipeline of JSON-manipulating tools, written in various languages, that your integer values will be passed through losslessly.

Therefore, you just shouldn't use JSON integer values when you know that the values can potentially be large. This is why e.g. Ethereum's JSON-RPC API uses hex-escaped strings (e.g. "0x0") for representing its "quantity" type.

It looks like JSON doesn't specifically define how numeric numbers should be stored. It just recommends expecting precision up to the double precision limits.

Still interesting to know it's not just a jq quirk.

Just because python has bigints and its stdlib json module supports bigints in json doesn't mean that is an interoperable thing to do.
The problem is made worse on the receiving end (the browser). I've ran into this issue when serialization libraries in Java send a 64-bit long value as a sequence of digits, then things over ~50 bits get silently truncated, you find out about it, then switch to quoted strings.
That's Python not adhering to the JSON standard as defined in the RFC.