[_giorgio_]

All I see, usually, is some AI YOLO algorithm applied to an offline video.

This is the first time that I've seen a "complete" setup. Any info to learn more on applying YOLO and similar models to real time streams (whatever the format)?

[yeldarb]

We’ve got an open source pipeline as part of inference[1] that handles the nuances (multithreading, batching, syncing, reconnecting) of running multiple real time streams (pass in an array of RTSP urls) for CV models like YOLO: https://blog.roboflow.com/vision-models-multiple-streams/

[1] https://github.com/roboflow/inference

[llm_trw]

Just stream it one frame at a time to the model and eat the latency: https://www.youtube.com/watch?v=IHbJcOex6dk if you need more hand holding.

There's a reason why there's a whole family of models from tiny to huge.

[_giorgio_]

Thanks for the link, but what happens when you have a video stream, be it a usb webcam, or an RTSP stream, and the hardware can't keep up?

I'm on windows.

Ideally I'd like the frames to be dropped, so the inference is done on the last received frame? Is this a standard behaviour?

[yeldarb]

If you do it naively your video frames will buffer waiting to be consumed causing a memory leak and eventual crash (or quick crash if you’re running on a device with constrained resources).

You really need to have a thread consuming the frames and feeding them to a worker that can run on its own clock.

[_giorgio_]

Sorry for the newbie question

Under windows, say that I have an RTSP stream (or something similar)

Would you use a single python script with which one of this multithreading solutions?

1 import concurrent.futures

2 import multiprocessing

3 import threading

[llm_trw]

That's not how loop devices work on Linux.

[hug]

This repository seems to be exactly what you are asking for. It's YOLO analysis of video frames passed in through Real Time Streaming Protocol.

[_giorgio_]

Yes, probably it's only one reasonably sized, let's say that with a lot of patience you can study it! I'll search for some online resources too.

I thought that this topic yolo object recognition would have much more following, instead there are really only a few projects.

https://github.com/search?q=yolo+rtsp&type=repositories&s=fo...