[kortex]

0 as a default for "no int" is tolerable, 0.0 as a default for "no float" is an absolute nightmare in any domain remotely related to math, machine learning, or data science.

We dealt with a bug that for weeks was silently corrupting the results of trials pitting the performance of various algos against each other. Because a valid response was "no reply/opt out", combined with a bug in processing the "opt out" enum, also combined with a bug in score aggregation, functions were treated like they replied "0.0" instead of "confidence = None".

It really should have defaulted NaN for missing floats.

[hot_gril]

What about floats made this a problem? We're treating 0.0 specially in some places.

[jakjak123]

I think your anecdote is rather weak with regards to the way protobuf works, but to entertain, why would a confidence of 0.0 be so different from None? 0.0 sounds very close to None for most numerical purposes if you ask me.

Wait, are you using Python?

[sebastos]

message LatLon {

    double lat = 1; // Geodetic latitude, decimal degrees

    double lon = 2; // Geodetic longitude, decimal degrees

}

"Hmm, lat = 0, I guess they didn't fill out the message. I'll thrown an exception and handle it as an api error"

[Later, somewhere near the equator]

"?!?!??!"

------------------

"Ok, we learned our lesson from what happened to our customers in Sao Tome and Principe: 0.0 is a perfectly valid latitude to have. No more testing for 0, we'll just trust the value we parse.

[Later, in Norway, when a bug causes a client to skip filling out the LatLon message]

"Why is it flying to Africa?!?!"

------------------

Ok, after the backlash from our equatorial customers and the disaster in Norway, we've learned our lesson. We will now use a new message that lets us handle 0's, but checks that they really meant it:

message LatLonEnforced {

    optional double lat = 1;

    optional double lon = 2;

}

[At some third party dev's desk] "Oh, latitude is optional - I'll just supply longitude"

[...]

"It's throwing exceptions? But your schema says it's optional!"

------------------

Ok, it took some blood sweat and tears but we finally got this message definition licked:

message LatLonEnforced {

    optional double lat = 1; // REQUIRED. Geodetic latitude, decimal degrees

    optional double lon = 2; // REQUIRED. Geodetic longitude, decimal degrees

}

[Later, in an MR from a new hire] "Correct LatLon doc strings to reflect optional semantics"

[hot_gril]

If both lat and lon are required, you don't need to throw an exception for lat=0. If you want lat=null lon=0.0 to mean something like "latitude is unknown but longitude is known to be 0.0," yeah you need optional or wrapped primitives.

Edit: If a client doesn't fill out the LatLng message, that's different from lat and/or lon being null or 0. The whole LatLng message itself will be null. Proto3 always supported that too. But it's usually overkill to check the message being null, unless you added it later and need to specially deal with clients predating that change. If the client just has a bug preventing it from filling out LatLng, that's the client's problem.

The confusing part here is that even if the LatLng message is null, LatLng.lat will return 0 in some proto libraries, depending on the language. You have to specifically check if the message is null if you care. But there are enough cases where you have tons of nested protos and the 0-default behavior is actually way more convenient.

[sebastos]

Yeah - I think what I'm getting at though is that you want to guard against situations where somebody accidentally doesn't set one of the fields, and yet 0 is a valid value to set on that field. You could accidentally forget to fill in latitude, and that would be bad news.

Def +1 the confusingness of how you have to explicitly check for has_latlon() to make sure you're not just getting default values of a non-existent LatLon message. The asymmetry between primitive and message-type fields in having explicit presence checking is also part of my beef. It's weird to have to teach junior devs that you can do has_* checks, but only for messages.

[hot_gril]

It's safer in a way to guard against these situations, but it seems like they don't intend you to do that often because there are more downsides outweighing it. Proto2 had the "required" feature that had its own issues. Our team trusts to some degree that the user actually read the API spec, and so far it's been ok.

I can imagine message nullness being clearer in languages with optional unwrapping like JS. Like foo.bar.baz gives an error if bar is null, and if you want default-0 you use foo.bar?.baz. Idk if that's what happens though.

[jakjak123]

In the case for Lat/Lon, I guess that 0.0 could have a meaning, though it is very unlikely someone is exactly at lat/lon 0.0. An alternative is to translate to the XY coordinate system, though that is not a perfect solution either.

If you really feel like expressing that LatLon as possibly null, it should rather be:

message User {

  optional LatLon position = 1;

}

[xarope]

working with gRPC allowed me to understand how go(lang)'s use of things like sql.nullstring works (pseudo'ish code}:

  astring = sql.nullstring{isNull=true, value=""}

  if astring.isNull {don't do anything}
  else {process astring.value}

So similarly, gRPC has a method called HasField, so:

  if packet.HasField(astring) {then process packet.astring}

Is it wordy? Yes. Is it elegant? Sadly, No. But does it work? Yes.

[kortex]

Yes it was python but that has nothing to do with it. Same would happen in go, rust, R, or matlab.

Correct answers: 1.0, 0.0, 1.0

Confidence from algo: 1.0, 0.0, n/a

Confidence on the wire: 1.0, 0.0, 0.0

Score after bug: 66%

Score as it ought to be scored: 100%

It was enough to make several algorithms which were very selective in the data they would attempt to analyze (think jpg vs png images) went from "kinda crap" in the rankings to "really good"

[jakjak123]

Well, only in python is that N/A value also a float. In protobuf, go or Java for that matter, that data model must somehow be changed to communicate the difference.

If you had use 3 float values in Go or Java you would have had the same problem.