[marcus_holmes]

Go maps have a defined type (like map[string]string), so you can only put values of that type in them. A JSON object with (e.g) numbers in it will fail if you try and parse that into a map of strings.

As others have said, the issue with Go parsing JSON is that Go doesn't handle unstructured data at all well, and most other languages consider JSON to be unstructured data. Go expects the JSON to be strongly typed and rigidly defined, mirroring a struct in the Go code that it can use as a receiver for the values.

There are techniques for handling this, but they're not obvious and usually learned by painful experience. This is not all Go's fault - there are too many endpoints out there that return wildly variable JSON depending on context.

[claytongulick]

I feel like good JSON handling is sort of table stakes for any language for me these days.

The pain of dealing with JSON in Go is one of the primary reasons I stick mostly with nodejs for my api servers.

[lelanthran]

> The pain of dealing with JSON in Go is one of the primary reasons I stick mostly with nodejs for my api servers.

Unless you're dealing with JSON input that has missing fields, or unexpected fields, there is no pain. Go can natively turn a JSON payload into a struct as long as the payload's fields recursively match the struct's fields!

If, in any language, you're consuming or generating JSON that doesn't match a specific predetermined structure, you're yolo'ing it and all bets are off. Go makes this particualr footgun hard to do, while JS, Python, etc makes it the default.

Default footguns are a bad idea, not a good idea.

[marcus_holmes]

This.

In $other_language you'll parse the JSON fine, but then smack into problems when the field you're expecting to be there isn't, or is in the wrong format, or the wrong type, etc.

In Go, as always, this is up front and explicit. You hit that problem when you parse the JSON, not later when you try to use the resulting data.

[simiones]

Go's JSON decoder only cares if the fields that match have the expected JSON type (as in, list, object, floating point number, integer, or string). Anything else is ignored, and you'll just get bizarre data when you work with it later.

For example, this will parse just fine [0]:

  type myvalue struct {
    First int `json:"first"`
  }

  type myobj struct {
    List []myvalue `json:"list"`
  }
  js := "{\"list\": [{\"second\": \"cde\"}]}"
  var obj myobj
  err := json.Unmarshal([]byte(js), &obj)
  if err != nil {
    return fmt.Errorf("Error unmarshalling: %+v", err)
  }
  fmt.Printf("The expected value was %+v", obj) //prints {List:[{First:0}]}

This is arguably worse than what you'd get in Python if you tried to access the key "first".

[0] https://go.dev/play/p/m0J2wVyMRkd

[marcus_holmes]

It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

One of the techniques for dealing with JSON in Go is to not try to parse the entire JSON in one go, but to parse it using smaller structs that only partially match the JSON. e.g. if you endpoint returns either an int or a string, depending on the result, a single struct won't match. But two structs, one with an int and one with a string - that will parse the value and then you can work out which one it was.

[lelanthran]

> It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

To me it looks like a footgun: if the parsing failed then an error should have been signalled. In this case, there is no error and you silently get the wrong value.

[simiones]

> It totally makes sense from a Go perspective: You created a struct, tried (but failed) to populate it with some json data, and ended up with a value initialised to its zero-value. This is fine :)

I do agree that there are good reasons on why this behaves the way it does, but I don't think the reason you cite is good. The implementation detail of generating a 0 value is not a good reason for why you'd implement JSON decoding like this.

Instead, the reason this is not a completely inane choice is that it is sometimes useful to simply not include keys that are meant to have a default value. This is a common practice in web APIs, to avoid excessive verbosity; and it is explicitly encoded in standards like OpenAPI (where you can specify whether a field of an object is required or not).

On the implementation side, I can then get away with always decoding to a single struct, I don't have to define specific structs for each field or combination of fields.

Ideally, this would have been an optional feature, where you could specify in the struct definition whether a fields is required or not (e.g. something like `json:"fieldName;required"` or `json:"fieldName;optional"`). Parsing would fail if any required field was not present in the JSON. However, this would have been more work on the Go team, and they generally prefer to implement something that works and be done with it, rather than working for all important cases.

Separately, ignoring extra fields in the JSON that don't match any fields in the struct is pretty useful for maintaining backwards compatibility. Adding extra fields should not generally break backwards compatibility.

> One of the techniques for dealing with JSON in Go is to not try to parse the entire JSON in one go, but to parse it using smaller structs that only partially match the JSON. e.g. if you endpoint returns either an int or a string, depending on the result, a single struct won't match. But two structs, one with an int and one with a string - that will parse the value and then you can work out which one it was.

I have no idea what you mean here. json.Unmarshal() is an all-or-nothing operation. Are you saying it's common practice to use json.Decoder instead?

[lenkite]

Umm, you can unmarshall into a map[string]any, you know ?

    dataMap := make(map[string]any)
    err = json.Unmarshal(bytes, &dataMap)

[simiones]

You can, but then it's a lot of work to actually traverse that map, especially if you want error handling. Here is how it looks like for a pretty basic JSON string: https://go.dev/play/p/xkspENB80JZ. It's ~50 lines of code to access a single key in a three-layer-deep JSON.

[lenkite]

Its more like 30 lines of code without the prints. However, one generally should code generic functions for this. The k8s apimachinery module has helper functions which is useful for this sort of stuff. Ex: `NestedFieldNoCopy` and its wrapper functions.

https://github.com/kubernetes/apimachinery/blob/95b78024e3fe...

Ideally, such `nested` map helper functions should be part of the stdlib.

[simiones]

Sure, in production you'd definitely want something like that, but the context was an interview exercise, I don't think you should go coding generic wrappers in that context.