Right. But also from a streaming PoV in the frontend: how does that look? Have a bunch of products from X site (not ranked at all), have a bunch of products from Y site all come in sequentially. Been thinking of how to display them.
Right now I get all products from all sites then rank and classify and categorise them.
As you get the results send them out (using async methods). As they arrive auto-sort them according to default user preference which is probably sort-by-price. Users will notice the scrollbar has increased in size or some other visual cue.
.NET looks like : Dict<WebSocket, List<resultObj>> webSocketConnections