[kellenmace]

Yeah, YouTube doesn't provide a way to search all of YouTube based on the spoken words in videos.

I could update VideoMentions Search to allow users to select multiple channels, and then perform the search across all of those (maybe importing all the channels they're subscribed to could be handy... ). One way or another though, it would still require selecting specific channels to search within. That limitation notwithstanding, I still think it's a useful tool, though!

Thanks for checking it out!

[bredren]

I get why you wouldn’t be able to index all of YouTube, that’s a big ask.

However, I don’t use YouTube enough to mess w channels much. I’m usually searching on a particular topic.

For example, “drop ceiling panel replacement.”

Perhaps you could help users limit the channel scope by making an intelligent channel selection by keyword.

So I would put in “home improvement,” and you could choose some appropriate channels to search for my search terms.

[kellenmace]

Hey @bredren! Yeah, I agree that this makes for a nice user experience. This is how the paid VideoMentions.com service works- users can search for channels by name or keywords, without the need to paste in URLs. It looks like this: https://cloudup.com/cUgKqErcx8G

That auto-complete lookup requires spending a finite number of API calls, though, which is why it’s restricted to customers and not available on this freely accessible VideoMentions Search page.

Thanks for checking it out, and for your feedback!

[filoeleven]

Your pricing model is really solid. I can see how useful this is to people or companies with a brand.

I don’t have one of those, but I can also easily imagine wanting to find “that one video that I watched a while ago” in some topical “power tools” channel where they talked about “parallel battery” tech, without recalling the channel name. If I really wanted to find that again in order to buy the tool, it’s easily worth $5. And then I have a whole month to chase down other vaguely-recalled references!

Well done. I hope this post gets the attention it deserves. You have solved a problem that many people have, and have griped about. Companies and people who maintain brands need to know about it. And your free tier (which I assume is funded by the paid options) has already helped me find something I’d been after for a while. I want this to stick around.

[bredren]

Ah! Okay. Makes sense. Thanks for the reply, maybe I’ll try out the pro version.

[joshvm]

They don't offer direct search, but isn't this what "key moments" is in search results? Try eg how to change a lightbulb.

I believe SeekToAction works even if the uploader didn't put chapters in. This was a relatively recent update, to make it fully automatic. So it's presumably doing some audio/video analysis to figure it out. All you need to do is tell Google how to seek your video (so it also works with non youtube videos too).

https://developers.google.com/search/blog/2021/07/new-way-ke...

[kellenmace]

Oh, cool! I hadn't heard of the Key Moments/SeekToAction feature before. I'll have to dig in and explore that a bit. Thanks for the tip!

[mistermann]

Could you do a search based on their watch history if they have it enabled?

[kellenmace]

This is a cool idea! It wouldn't apply to people who want to search all the videos on a given channel for specific keywords (including those they haven't watched). I can see it being useful for folks trying to locate a specific video they remember watching in the past, though.

One consideration is that getting the user's watch history would likely require calls to the YouTube API. So that means I would have to make this a paid service in order to offset the code of those API requests. The beautiful thing about the current iteration is that it doesn't rely on YouTube's API at all. By scraping YouTube pages and leveraging a few NPM packages, I'm currently able to offer free and unrestricted access to it.

If enough people request that ability though, I'll consider incorporating it.

Thanks for checking out my project and for the great idea!

[victor141516]

Sadly YouTube API doesn't allow you to retrieve user's history, so you need to scrape that aswell.

I created a exactly that project: https://github.com/victor141516/YourArch

[mistermann]

If you're ambitious, I think a compelling rap genius style social media site could be built based on people commenting on specific segments of YouTube talks (and podcasts, whatever). Rap genius (now genius.com) seemed to be doing something really novel, but then....something happened. But I think the idea is rich with opportunity.

This project is doing a variation of it:

https://news.ycombinator.com/item?id=31527544

My vision of it would require some pretty strong UI skills, or maybe nowadays it's not so hard to accomplish impressive things.

[hanniabu]

That'd certainly be useful

[hanniabu]

Do you cache the channel, search term, and results for faster more efficient responses later?

[kellenmace]

Hey @hanniabu! I do client-side in-memory caching of videos data, only. No server-side caching. In fact, there is no database involved at all- the client-side app calls serverless function API endpoints to fetch the YouTube channel and video data it needs. Here are the tricks I'm using to make it fast:

- As soon as the "Channel URL" field loses focus, I start fetching the most recent 30 videos on that channel in the background. This way, by the time the user enters the keyword and date range, I've already fetched some (maybe even all!) of the data ahead of time, which means less wait time for them.

- Once a specific video's data (title, description, transcript, etc.) has been fetched once, it is saved in memory. All other searches the user performs from that point on will pull the video data from the in-memory cache, if it's there. Otherwise, it will fall back to fetching the video data over the network. This in-memory caching makes subsequent searches within the same date range (or a shorter date range) take <1 second.

- Network requests to fetch video data are processed concurrently rather than one at a time. So the browser fires off as many as it can in parallel to get them all resolved as quickly as possible.

- As soon as any matches are found, the UI updates to show the user. This way, the user can start scrolling through the matches and reviewing them while the search is still in progress– they don't have to wait until it finishes to start interacting with the matches.

Thanks for checking out my project!

[alex_smart]

>I do client-side in-memory caching of videos data, only.

This works only if the user does not navigate from and back to your website or refresh the page, but if they do, you make the same api calls all over again. You should set HTTP Cache-Control headers in your response from the server, so that the browser knows that it can serve that data from its cache and does not need to make those requests again. You would then probably not need the client-side in-memory cache at all.

[kellenmace]

I could do this, but it would introduce other cache invalidation issues. Take this sequence of events, for example:

* The user performs a search, reviews the matches, then closes the browser tab.

* The title or description for one or more of the videos on that YouTube channel are updated.

* 45 minutes later, the user remembers that they wanted to search for something else on that same YouTube channel. They return to VideoMentions Search to perform another search.

If I had sent cache-control headers in the first response telling the browser to cache the results of those HTTP requests, they would not be refetched for subsequent requests and the user would potentially be shown incorrect matches.

By fetching the data fresh whenever the user opens VideoMentions Search in a new browser tab, I guarantee that they're getting correct matches based on the titles, descriptions and spoken words that the videos have at the time the search is performed.

So as it's implemented currently, I'm choosing to take a performance hit in exchange for improved accuracy. And I'm comfortable with that tradeoff since searches seem decently speedy to me when I use the tool.

Thanks for your thoughts, and the alternative client-side caching idea!