| > And how do you build the full index initially without recursively walking the filesystem? Otherwise you're not going to match Everything's performance on initial index creation. I wasn't planning to; it's a once-off cost - the user experience while using any software isn't degraded by the installation time, is it? > And regarding the second crucial question: How do you know that a file you saw the last time your app or daemon was running, hasn't been modified in the meantime? It's a daemon. if it isn't running while the user is using their desktop system, then it's not working or the user has turned it off. In any case, if a component of the software is not running, then the software is not running. I mean, seriously, even during regular updates, daemons still run. Even during distro upgrades daemons are still running. The rare cases where files are removed/changed/moved while daemons are turned off are fractions of fractions of a percentage. My desktop system currently has 2.5m files. There are maybe a dozen files which will be modified during a maintenance-mode bootup, which has happened exactly zero times in the last decade. For a Linux desktop file finding utility, monitoring all file writes, moves and deletes pretty much puts you ahead of any game in town right now, right? > You keep saying that, but you're also not giving an answer to how you're going to solve the two major and pretty much only issues. Perfect is the enemy of good. Issue 1 - Initial index creation: I will create the index during the s/ware installation process and never create it again unless it is missing. To speed the creation during installation, I will use the mlocate.db file if it is found. Issue 2 - Files that are changed/moved/removed when daemon is turned off: I don't really care, mostly. Those files a) have such a small probability of both existing and being of interest to the desktop user that lottery jackpots have a higher chance of happening to the user, and b) After an MVP, if the userbase requests those files, I'll either hardcode their locations and always check only for those dozens of files that can possibly be changed when daemons are turned off, or allow the user to specify via configuration, the pathname patterns to always check. I believe that this is enough to satisfy my original claim[1] of " "similar in performance and query capabilities""[2]. [1] https://news.ycombinator.com/item?id=38686022
[2] I don't recall making any claim along the lines of "walking the filesystem tree is never used". |
This whole topic started with you claiming that you can even beat Everything in that regard, which is why I even got involved in that discussion.
Remember, your response to:
> A huge part of Everything's speed comes from reading the master file table that other people mentioned, so you would need a way to quickly read file table entries on linux.
Was
> Not a problem. And no, I'm not talking about inotify either, and I'll additionally index the contents of (text) files as well with a negligible additional performance hit. It can be done as fast as, or faster than, `Everything`.
And btw. indexing content will obviously only put you even further behind. The cost is not negligible.
> It's a daemon. if it isn't running while the user is using their desktop system, then it's not working or the user has turned it off.
> My desktop system currently has 2.5m files. There are maybe a dozen files which will be modified during a maintenance-mode bootup, which has happened exactly zero times in the last decade.
I have many users, myself included, who use things like shared filesystems, which get modified by multiple systems. And like I already said, modern Linux systems also perform all of their updates in such a maintainance mode. So your app will give thousands of false positives or miss thousands of files completely on those systems.
Sigh... So you're also not going to solve the second issue. I mean I clearly asked you these questions multiple times and I tried to make it clear, that this is where the problem is, to save you and me time, and still you kept it a secret up until now that you're not even attempting to fix those problems.
So I'll have to take back my claim: Under these circumstances I can't guarantee that you'll make a lot of donations, because your app won't do anything special compared to others.
> For a Linux desktop file finding utility, monitoring all file writes, moves and deletes pretty much puts you ahead of any game in town right now, right?
Well kind off, but it's not particularly difficult to solve that issue. The dev versions of FSeaerch already can do that.
> Issue 1 - Initial index creation: I will create the index during the s/ware installation process and never create it again unless it is missing. To speed the creation during installation, I will use the mlocate.db file if it is found.
So you're doing exactly what everyone else is doing. You can also ingore the mlocate index, because it doesn't contain enough information (size, date modified, ... aren't indexed by it so you'd need to stat all of those files anyway).
> Issue 2 - Files that are changed/moved/removed when daemon is turned off: I don't really care, mostly. Those files a) have such a small probability of both existing and being of interest to the desktop user that lottery jackpots have a higher chance of happening to the user
Like I already said, you're ignoring the hard and important problem. That's fine, but you suggested otherwise and now you're again doing nothing out of the ordinary.
> I believe that this is enough to satisfy my original claim[1] of " "similar in performance and query capabilities""[2].
Well, it depends, you're not going to beat Everthing in the areas me and others care and in an attempt to get anywhere near that, you're trading accuracy for speed (what makes Everything special is that it's both fast and accurate/reliable). That's fine, but this is nothing new or special, so I'm not really interested in that.