I was curious and dug into the Go client code. You linked to the definition of KV – the easiest way to create one is with NewKV [1], which internally creates a RetryKV [2] wrapper around the Client you give it.
RetryKV implements the KV methods by delegating to the underlying client. But before it delegates an immutable request (e.g., range), it sets the request retry policy to repeatable [3].
Retries are implemented with a gRPC interceptor, which checks the retry policy when deciding whether a request should be retried [4].
The Jepsen writeup says a client can retry a request when “the client can prove the first request could never execute, or that the request is idempotent”. In my (cold) read of the code, the Go client stays within those bounds.
For non-idempotent requests, the Go client only retries when it knows the request was never sent in the first place [5]. For idempotent requests, any response with gRPC status unavailable will be retried [6].
Unlike jetcd, the Go client’s retry behavior is safe.
The watch is a simple processing loop that receives and sends on a bi-directional GRPC channel. Leases have a similar loop for keep-alive messages, everything else is quite literally delegated.
I get that it's difficult to translate this 1:1 into Rust without channels and select primitives, but saying it's complex is wild. Try the server-side code for leases ;)
RetryKV implements the KV methods by delegating to the underlying client. But before it delegates an immutable request (e.g., range), it sets the request retry policy to repeatable [3].
Retries are implemented with a gRPC interceptor, which checks the retry policy when deciding whether a request should be retried [4].
The Jepsen writeup says a client can retry a request when “the client can prove the first request could never execute, or that the request is idempotent”. In my (cold) read of the code, the Go client stays within those bounds.
For non-idempotent requests, the Go client only retries when it knows the request was never sent in the first place [5]. For idempotent requests, any response with gRPC status unavailable will be retried [6].
Unlike jetcd, the Go client’s retry behavior is safe.
[1] https://github.com/etcd-io/etcd/blob/main/client/v3/kv.go#L9... [2] https://github.com/etcd-io/etcd/blob/main/client/v3/retry.go... [3] https://github.com/etcd-io/etcd/blob/main/client/v3/retry_in... [4] https://github.com/etcd-io/etcd/blob/main/client/v3/retry_in... [5] https://github.com/etcd-io/etcd/blob/main/client/v3/retry.go... [6] https://github.com/etcd-io/etcd/blob/main/client/v3/retry.go...