Hacker News new | ask | show | jobs
by braindeadly 497 days ago
From the webpage:

"Go WASM is slow at parsing large amounts of JSON, which led to dramatic architecture changes and the creation of a “smart backend” for incremental data loading over WebSockets, using Go's rarely-used encoding/gob format."

From https://pkg.go.dev/encoding/gob

"This package is not designed to be hardened against adversarial inputs, and is outside the scope of https://go.dev/security/policy. In particular, the Decoder does only basic sanity checking on decoded input sizes, and its limits are not configurable. Care should be taken when decoding gob data from untrusted sources, which may consume significant resources."

How do they sanitize the gob data?
3 comments
adtac 497 days ago
>from untrusted sources

if you can't trust your own backend, step one is reevaluating your life choices

link
xlii 497 days ago
..or a career.

After years of experience I question everything ;-)

link
omcnoe 497 days ago
It's not completely clear from the link but it seems like they are only using encoding/gob for backend -> client transmission over the websocket. In which case, it's your own backend, it is a trusted source. But the risk is that if there is any data going the other direction over the websocket in encoding/gob format. Probably best to just avoid completely.

Go JSON parsing isn't just slow in WASM, it's slow in native code also. I've never dug into why though, surely there are some big cost savings if this were to be optimized?

link
broken_broken_ 497 days ago
At work the Go backend was also using gob encoding in one case to store some values in the database. After some profiling, I noticed it was the top allocation part of the whole app. It was also showing on the CPU performance profiles. Finally some very light “fuzzing” (just a case were the wrong value was written to the database, and reading it back crashed the app) made me decide to rip it out and use JSON instead. Instantly, this section of the code disappeared from all profiles. I honestly do not understand why this format exists in the Go standard library and I would never use it. There are plenty other formats available…and a decoder should never crash in the face of invalid data!
link