|
It sounds like you are talking about real-time data capture, analysis, visualization, encoding, and archiving. If so, what you describe is not just multimedia. It is a real-world hybrid of multiple specialities, and definitely becomes more difficult because you have to address all these aspects at once to meet your original goal. If you had started with pre-recorded videos, you could imagine having pursued the challenges incrementally. Process a video and output some frame-by-frame analysis metadata. Process the analysis results and visualize them in the context of the source video imagery. Re-encode the results to a good storage format. Explore options for quality, correctness, performance, and resource requirements. But, if you need to do this on live streams and run for indefinite periods of time, you suddenly need to be much more aware of computational delays, working memory sizes, buffering of input and output data, etc. You need it to work consistently and predictably while handling the full workload and any potential variability in other elements of the system. How much can the incoming video stream vary in bandwidth? How much can decoding, analysis, visualization, and encoding speeds vary with changes in the video content? How reliably can the input and output IO paths support the bandwidth? And, as you mentioned, it becomes challenging to then squeeze this into a slower system than it was originally designed and tested on. Ultimately, you have to design for failure: think about how you want it to perform if it cannot satisfy the full objective in the time and resources available. Drop frames? Degrade stream quality? Drop entire low-priority streams...? |