Hacker News new | ask | show | jobs
by polygamous_bat 928 days ago
You can also use software to detect “cuts” in the video, which can be used to improve the frame-extraction over just getting six evenly spaced frames from the video.
2 comments

This is a task called "video summarization". See https://paperswithcode.com/task/video-summarization . I guess the whole project is something like summarizing from video + subtitles + text to pictures + text.
Not the post author but I tried this with ffmpeg and failed. Do you (does anyone) want to share some pointers?
I used something like this a few years ago in a project sort of similar to this one. There's a bunch of parsing and processing to do with that, and the "0.3" value is ... fiddly, but it worked pretty well:

    ffprobe -show_frames -of compact=p=0 -f lavfi "movie=THE_VIDEO_FILE,select=gt(scene\,0.3)" -pretty`
I played with that too before.

`ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)'" -vsync vfr frame-%2d.jpg`

(from the repo pforret/filmpace)

For this project, I want to find an A.I. solution for finding the most 'interesting' frames. Not even sure how to measure interestingness yet, might be the presence of text, the presence of a human ...

PySceneDetect (https://www.scenedetect.com/) might be useful.