[mrsofty]

we had a big julia push this month after 2 years of just messing around. It's better than APL to read ( so is Sanskrit) but we hit a SCREECHING halt when we realized that it wasn't going to happen that we could our streaming data with Pluto notebooks on the web. Pluto Notebooks are wonderful and can handle streaming data just not on a hosted web page with multiple people using it. We tried to use Stipple.jl ( part of GENIE.jl) and that kept freezing ( we suspect because of pacing issues so 1 sec plus should be fine). The point of all of this is that we have found julia to be GREAT to build the back end stuff but not for manipulation of streaming data on the web. We can easily fix this with ZMQ and send the data to Python but julia was supposed to be a 1 language solution. We're trying to dodge the web side of things with Humane so maybe we'll be happier bunnies in 2024

[sundarurfriend]

> when we realized that it wasn't going to happen that we could our streaming data with Pluto notebooks on the web

It sounds like the issue is probably unrelated to using Pluto, and likely more to do with the streaming libraries used and memory management - but that's just a guess based on the minimal info here. When you say it couldn't handle streaming data, what issues did you have? By "streaming data with Pluto notebooks on the web" do you mean PlutoSliderServer or something else?

FWIW, Fons and co are very responsive to user issues (for eg. on the Zulip pluto channel [1]), so if you haven't tried that already, I'd recommend that. Similarly with Stipple, I believe they're trying to build a company out of it, so they'll probably be very receptive to business use cases and making them work.

[1] https://julialang.zulipchat.com/#narrow/stream/243342-pluto....

[mrsofty]

Fons IS the person that informed me that we couldn't deploy our app using Pluto PlutoHooks PlutoSlider. The issue is that we need to have people able to go to a web page and have their own view of the notebook.

On the Stipple front it's a pacing issue, we think. The developers have been WONDERFULLY helpful and improved our code tremendously. GENIE is a great solution and I am sure that they will be successful. I believe they WILL produce a MWE of this task and we'll certainly look to see if we can make it work. Right now we can't as some of our lab equipment and financial systems generate sub 0.5 sec data stream.

[jcheng]

Can I ask what you would "easily" do after you "send the data to Python"? What Python framework would you use to easily build interactive real-time streaming data apps?

I'm asking because I work on Shiny, a reactive web framework for Python (and R) that aims to solve this problem well, and I'm having trouble figuring out how Python people have been doing this sort of thing. It's straightforward with a lower-level web framework like Flask/FastAPI but then you lose the nice reactive properties of something like Stipple.jl (and Shiny).

[mrsofty]

Sorry, missed this as we're in meetings this week. So to answer your question.

We use zeroMQ to move data around so to get it into a python script is very easy for us. We "were" going to create a dash/plotly app to consume the data stream and create a trading augmentation tool as we find dash a pleasure to work with. We also find that it's very well supported using videos so not even we ( non web developers) can mess it up. We adhere to the Tufte approach to graphical representation of complex data sets and python allows us to take a minimalist approach to doing that.

We are also discussing the implementation of async components in a python specific hosting company that seems to address the same market as GENIE. That would give us the GUI approach should we chose to use it. We have experienced some problems with streaming data into Stipple but I would expect that they will provide a MWE that we can modify. I have high hopes for the GENIE team, they seem like excellent people.

All that said we are progressing out Carbon discussions with various people. In Chicago we are very lucky that we have a deep pool of people skilled at kernel latency avoidance, this helps us consider what we can expect from Carbon and if there are any advantages to what we want to achieve.

I hope this answers your question.