[underyx]

Wish this could filter to videos that actually are music videos instead of just audio track + album cover. Perhaps it would be possible to make an educated guess by looking at the video file size. As it stands, less than 10% of my tracks actually end up displaying a proper video.

[immannino]

So I’ve actually been trying to brainstorm this for awhile. It’s really tough because I can’t programmatically validate the videos. Right now I was thinking of some kind of caching to keep a fire base of the correct music videos, and then having a pop up for the user to give feedback on whether or not the video I displayed for them is a music video or just music+album cover, and then have them be able to link the correct music video or submit a ticket and I could manually update that video in a database. Not 100% sure yet but I’m working on it

[rajington]

I had a similar problem previously and considered automating it by comparing frames. It's not easy and would have obvious false positives/negatives, but YouTube does help by publishing key frames and it could at least be used to help rank results.

Take this example music video:

https://www.youtube.com/watch?v=6vopR3ys8Kw

Frames:

https://img.youtube.com/vi/6vopR3ys8Kw/0.jpg

https://img.youtube.com/vi/6vopR3ys8Kw/1.jpg

https://img.youtube.com/vi/6vopR3ys8Kw/2.jpg

https://img.youtube.com/vi/6vopR3ys8Kw/3.jpg

And this lyrics video version (more interesting because it's SOMEWHAT changing):

https://www.youtube.com/watch?v=FoAqHxm5dpo

Frames:

https://img.youtube.com/vi/FoAqHxm5dpo/0.jpg

https://img.youtube.com/vi/FoAqHxm5dpo/1.jpg

https://img.youtube.com/vi/FoAqHxm5dpo/2.jpg

https://img.youtube.com/vi/FoAqHxm5dpo/3.jpg

Simply finding the differences between frames would give the first video a higher score than the second.

[spdustin]

I commented on your parent post before reading this...

Nice catch on the thumbnails that YouTube already captures. A histogram comparison between the second and third auto-generated thumbnails from the lyrics video was mostly equivalent when I ran one. That would be a good sign that it's not the actual music video.

[spdustin]

What about snapping a few frames—maybe 33%, 50% and 66% of the way into the video—and doing a simple face detection?

Perhaps, on top of that, a histogram comparison of the same frames, which should capture if the video is pretty much lyrics on a static background.

[underyx]

1. It's not like all (or even most) music videos have faces in them.

2. I'm pretty sure it's not trivial to download just three specific frames of a video. And downloading so many videos would probably be expensive/bannable.