Hacker News new | ask | show | jobs
by sauljp 699 days ago
Hi! it validates if the message was (or not) generated for schema registry, it checks if the message has the schema registry magic bytes (this bytes has the schema id). so, it deserializes messages with or without schema registry.

This is the drawback: you have to generate the descriptor first.

1) download the schema from schema registry: http :8081/schemas/ids/<my-id>/schema

2) generate the descriptor with the schema: protoc --include_imports --descriptor_set_out=my-descriptor.desc --proto_path=. my-schema.proto

3) use kaskade: kaskade consumer -b my-kafka:9092 -x auto.offset.reset=earliest -k string -v protobuf -t my-protobuf-topic -p descriptor=my-descriptor.desc -p value=mypackage.MyMessage
2 comments
ddoolin 698 days ago
Perfect! I had tried this last night before your comment (I saw these instructions in your README) but I was trying to include `-s url=` and `-s basic.auth...` which appears to force you into using `-k avro` or `-v avro`. But, once I removed the Schema Registry config options, I finally got the TUI showing up and was able to read messages! Thank you very much. Nearly all of my topics are Pb-encoded so I'll have to throw something together to get all the descriptors and perhaps abstract a bit. I love seeing the different uses of Textual coming out and this is no different; this is a big improvement from kcat IMO.
link
sauljp 698 days ago
Hey ddoolin, It's so nice to read this, yes! kaskade ignores the schema registry metadata for json and protobuf messages generated for schema registry, you do not need to provide url (only in case of using avro).

The only catch (in the protobuf case) is to generate the descriptors with protoc, but the good thing is generally we all have them, so maybe is not a big problem.

link
perezd 697 days ago
It'd be cool if you supported the Buf Schema Registry's Reflection API: https://buf.build/docs/bsr/reflection/overview

Then, the descriptors could be delivered on demand!

link
sauljp 697 days ago
Working on it
link
sauljp 699 days ago
about the first step: if you already have the .proto file you do not need to download it from schema registry
link