[anirudhrx]

Congrats on the launch! This is really cool. Would love to see OTel integration in the future. I'm curious if this might eventually work with request-context based routing, i.e. being able to use the propagated metadata between layers to dynamically test different versions of the stack, replay requests / route to specific underlying implementation versions at different levels of the stack.

[marcklingen]

Thx. Currently looking deeply into how we can make it as dynamic as possible to help people experiment with configurations and test on production samples. What do you exactly refer to with levels/layer in stack? See multiple meanings that would all make sense to me and would love to learn more

[anirudhrx]

Sorry if I'm not using the right terminology here. By layers, I'm talking about the different places from where you're emitting spans - like in your example, the UI, prompt-embedding, vector-store and context-encoding. What I was curious about was two-fold - one whether there are use-cases that make sense to have experimentation of the kind that allows a user to swap out a particular step in that stack - and second about making use of context propagation like baggage enables in OTel.

[marcklingen]

Got it, appreciate the clarification.

(1) yes, that's what many of our users are doing while swapping might refer to changing model parameters, prompts, embedding stores, embedding parameters, or splitting a "layer" into multiple (parallel) steps. Alternatively when experimenting on a higher "layer", we often see wildly different implementations that are experimented with over time. We added version and release tagging which to build dashboards on the impact of these experiments on quality/latency/cost (currently in alpha, we will release it soon).

(2) I see how metadata propagation could be much more elegant via OTel. Currently the metadata fields on traces/span and filtering based on them kind of does the job but I see how it could be much easier to ingest and use. Thank you!