Hacker News new | ask | show | jobs
by manacit 3589 days ago
Yeah! I think a few responses have covered this below, but I'll give you our spin (and why it's painful, compared to what people have offered up).

Most of the projects that we're integrating gRPC into are existing codebases that have their own build tools that are (mostly) in isolation. JSON schemas have been agreed on beforehand, and there are separate client implementations in different languages that basically exists as independent units.

By adding protobufs to this process, the "JSON schemas that have been agreed on" become protobuf definitions - which is _fantastic_ for development teams, because they have a single spec to work from, and there is no ambiguity (or, significantly less).

The challenge comes when we are trying to generate gRPC clients in Go, Ruby and Python for the same profobuf file - in order to do it in an automated fashion without a 'monorepo', we need to create a build system that pulls this protobuf from a central place and generates the client, which doesn't exist right now.

It's not a huge challenge to ensure services can communicate at all - as you said, protobufs have thought of this and have an extensive amount of decoupling built in. When we're working on adding new features however, we need to have a place to keep the "gold master" of protobufs, and grab it for all of our projects to build+deploy at once, which is where the above becomes challenging.

Not an un-solvable problem, and different languages have different tooling for this. We've settled on placing the proto definitions in the "server" side (most of our interactions are fairly well modeled by client/server), and then updating the clients as-necessary, as we can deploy server changes without needing to update the clients immediately.
1 comments
GauntletWizard 3589 days ago
There's no need to pull the proto file on every build. Proto also has a set of rules for how to maintain wire-compatibility across versions[1]. Following those rules and distributing the definition only when you need new fields should be sufficient.

That said; If you've got a set of shared proto definitions, you should probably either go to a monorepo, or share the shared bits with a git submodule. Doesn't prevent you from needing to follow those conventions, but does make it far easier to debug when things changed.

[1] https://developers.google.com/protocol-buffers/docs/proto3#u...

link