|
Thanks for the thoughtful reply. What you describe sounds awesome, and in fact some of these concepts are present in our stack but not fleshed out to this level, so maybe we could get there someday. The challenge is making it practical. Currently, Fanout supports HTTP in addition to WebSockets. We consider the system to be generalized, in the sense that the user can build whatever they want on top of the available primitives, and we can always add more primitives. Most people are implementing "web" protocols (REST, SSE, GraphQL, etc), so what's available today gets us pretty far. You're right of course that we don't support arbitrary TCP protocols. I suppose the reason we don't support bare TCP is Fanout is optimized for implementing pub/sub-style interfaces with minimal compute. In that context, it is preferable to work with coarse-grained input in order to avoid per-session processing. However, with Fastly Compute now in arms reach, I think we could embrace per-session processing, in which case implementing arbitrary TCP protocols could become practical. Your idea of supplying custom L7 parsers to the routing layer is clever, and is not too far off from some things already on our mind. For example, our routing layer supports inspecting requests using custom rules supplied by the compute layer, though this is not yet exposed to Fastly users. Relatedly, the routing layer has an internal component that parses TCP streams into messages (by known protocol) and ships them over an IPC without caring about their content. So it's not too much of a jump to imagine how we could get to user-supplied L7 parsing. |