[ianai]

Why does a package manager need to track their users at all?

If you want usage statistics for packages just track how often individual packages are downloaded on the server side. A maintainer has no need to know who's installing what.

[woodruffw]

> If you want usage statistics for packages just track how often individual packages are downloaded on the server side. A maintainer has no need to know who's installing what.

To be clear: Homebrew has no idea which users are installing what. We only store counters for package install, failure, etc. events, and everything that's stored is visible on the Homebrew website[1].

Homebrew's architecture doesn't really have a "server side" in the way your suggestion requires: the formulae and bottle components rely heavily on public services like GitHub Packages and GitHub Pages, which don't offer those kinds of analytics.

FD: Member of Homebrew.

[1]: https://formulae.brew.sh/analytics/

[chongli]

What kind of decisions do you make based on the analytics? Do you drop unpopular packages? To me, one of the advantages of a package manager is having a huge database of long-tail packages that are just one command away. If you kept only the popular packages you might as well just have an installer that installs them all together in bulk.

[woodruffw]

> Do you drop unpopular packages?

Yes: Homebrew deprecates and/or disables packages if we see evidence that they're unmaintained and not actually supported on the platforms we support, or only used by a tiny fraction of users while also requiring disproportionate maintainer time (e.g. due to complex or flaky builds).

The goal is to balance conflicting user interests: 99% of users want maintainer effort focused on the top 100 (or 500, or 1000) packages, and many of those packages also require significant maintainer effort (e.g. making sure that they don't cause transitive breakages).

[chongli]

So if a package has a small community of users and is well-maintained upstream but not popular overall you'll keep it around?

[woodruffw]

Generally speaking, yes: we don't remove things that aren't disproportionately burdensome. You can see that in the yearly analytics[1]: there are packages that only get a few dozen downloads a year, and we don't remove them because they don't cause any issues.

[1]: https://formulae.brew.sh/analytics/install/365d/

[chongli]

That's good to hear! I think small, well-maintained projects with loyal users can add tremendous value to a package ecosystem due to the long tail effect. I believe these long tail effects are a big part of the reason behind the staying power of tools like emacs, vim, and LaTeX.

[pimterry]

> Why does a package manager need to track their users at all?

According to https://docs.brew.sh/Analytics they use it to measure how often formulas fail to install, to get overall metrics on which OS versions are used, and to correlate those (i.e. to tell on which OS versions specific packages fail to install correctly).

> A maintainer has no need to know who's installing what

Aside from the IP, they don't know who's installing what, and in the new model announced in this post they now don't store IPs or any other user token at all, so it should be purely anonymous aggregate metrics.

[kelthan]

I agree that they don't need to know the "who", but it is perfectly understandable that they want to know "what" is being installed. And as part of the "what", they would want to know on which platform, and whether the install succeeded or failed, and probably a few other metrics about the install to ensure that things are working correctly and identify gaps that should be filled.

Based on what I read on the site, that looks like exactly what they are doing, and they are explicitly NOT storing information that would identify "who".

[woodruffw]

Correct: there's no identifiable information being stored, either before or with these changes.

[dmix]

They can already gleen a lot of this since they run the hosted formula db anyway. A 90-day analytics capture isn't a big deal IMO.

[Patrol8394]

Aside from the IP? IP nowadays is all you need…

[hardware2win]

I think nowadays IP is less and less relevant when majority of people sit on dynamic IPs

[lvh]

They've been quite clear about what they store and it's not IPs.

[ianai]

All stuff that should be in a trouble ticket from a whiney user. Which we know this type of user would be.

Edit-Also, this is for Mac OS. Chose a few standard OSes to support and test them. If a system update will fix the issue then it shouldn't be fixed at the package manager level.

[lamontcg]

Much rather deal with anonymized telemetry blowing up than tickets from whiney ass users.

[ianai]

Does anybody put a paywall in front of submitting tickets? Seems like homebrew could if anybody. Ie $10 to submit and might need more for a larger problem.

[lamontcg]

> Why does a package manager need to track their users at all?

Do any of you actually work in this industry shipping software products to end users? Without telemetry the problem there is literally one of trying to read the mind of your end users to figure out what they're doing, hoping that your internal CI manages to reflect the configuration in their environment.

[Shish2k]

I think HN has a very varied audience - some work in the industry, others want A/B testing to be made illegal on the grounds that it is non-consensual mind-control experimentation :P

[bathtub365]

The groups of people who work in the industry and those who believe A/B testing is psychological experimentation aren’t disjoint.

[jeltz]

I am in both groups. I work in the industry and I am so tired of colleagues wanting to grab or data they can get their grubby hands on and then barely use it at all for anything useful. So many companies collect data just in case.

[hungryforcodes]

Users report issues to GitHub? It's not like Brew users aren't sophisticated in that sense.

In addition to being INCREDIABLY slow, now I have to worry about what it might spy on. If I have a problem I'm more than happy to go to GitHub (or which ever site it's hosted on), and report it.

[closewith]

I imagine many of us work shipping software to end users and also respect their right to privacy, and only track their actions with informed consent.

[account42]

This industry has managed to ship software products without telemetry just fine - mass-collecting usage data from end users is only a relatively recent trend.

[pdimitar]

Any actual arguments?

I don't see why something that's little more than a file server needs telemetry.

[woodruffw]

Homebrew is a package manager with thousands of packages, not a file server. We maintain those packages, and knowing when they break (or can be deprecated due to lack of use) is critical to the project's sustenance.

[pdimitar]

Okay, fair enough. But the breakage can't be detected without telemetry then, I take it?

If so, that's... not ideal for sure.

[woodruffw]

Homebrew can detect a lot of things during normal maintenance work: there are extensive tests and checks during compilation and bottle building, for example.

However, we can't catch everything: Homebrew has millions of users, and those users have all kinds of different setups. We can't predict every possible host and software interaction; basic failure analytics help bridge the gap there.

[pdimitar]

Okay, I'm sold, and I'm sorry for being blunt.

Where can I learn more? Can you point me at the right place in the source?

I'll not be banning Homebrew telemetry.

[lamontcg]

And with packages that compile.

Which the software that I used to be employed maintaining has actually broken homebrew compiles when they've been installed at the same time (which I think I made better but I never got the PM who actually owned the product to spend the resources to properly fix).

A good example of how the configuration in the end user environment can affect package installation.

[therealdrag0]

Have you looked at the analytics yet? Or are you only speaking from ideological priors?

The most valuable one I’d guess is package install error rates. Seems pretty useful to me.

[shepherdjerred]

> something that's little more than a file server

You're doing an awful disservice to Homebrew.

[pdimitar]

I am, yes, and sorry about it.

I don't like telemetry at all and I believe we have to find other ways to do QA. Hence my strong reaction.

[brookst]

Maybe they want to include the most common packages in their unit tests, or understand usage patterns so they can prioritize development?

It’s very hard to write and maintain good software without knowing how it’s used. No package manager needs to know how you specifically use it, but aggregate data and the ability to identify scenarios it does not handle well are both very important for SW lifecycle.

[ianai]

It's 2023. Hard drive space shouldn't be an issue. Test installing the full software suite, make it work, and you know the lesser installs will all work.

[orf]

Do you think “hard drive space” is the constraining factor when building and testing over 6.5k third party packages?

Do you really not see any advantage to maintainers having visibility into what packages people actually use?

[Larrikin]

How much would you be willing to pay so that Homebrew can maintain a large amount of hardware covering nearly all configurations?

[justinclift]

Just checked their OpenCollective, and they seem to have about US$100k there:

https://opencollective.com/homebrew#category-BUDGET

They seem to be receiving about US$2k/month via Patreon too:

https://www.patreon.com/homebrew

I think their Patreon was around the same when I looked ~12 months ago.

[shepherdjerred]

$2k/mo is a terribly low figure for a piece of software that nearly every dev on macOS uses.

[justinclift]

That US$100k seems to have arrived over the last year, and their overheads - from quickly looking over their OpenCollective expenses tab - seem to be mostly buying hardware for CI things, and a monthly GSuite bill.

It's also pretty likely they get a lot of their resources for free / sponsored too.

So, US$2k/mo might be fine (no idea).

[shepherdjerred]

> Test installing the full software suite, make it work

Are you paying for the compute?

[cassianoleal]

Assuming their claims of anonymity are true, they won't be tracking users at all.

I imagine they can get much richer metrics through this as opposed to only tracking downloads on the server side.

I'm not saying I like it. In fact, I plan to keep it disabled. I'm just saying it's a bit naïve to think client-side analytics are the same as server-side download tracking.

[ianai]

Richer how, exactly? I fundamentally don't 'get' what richness they actually need.

If anything they need money, of course, and to know their software works for their users. Prior to release have a test system install the full base, test those packages work, and you know anything less will work too.

[cassianoleal]

I haven't looked at what they're actually collecting, but here's a few things that come to mind:

- Time to install packages - Versions of things - Has the compilation (when required) failed? What dependency versions are installed? - CPU architecture - OS version ...

There's a lot more that can be sent from the client that's not available on the server side.

> I fundamentally don't 'get' what richness they actually need.

That's fine. Perhaps you could ask them instead of ranting about what you don't know or don't 'get' in a public forum?

> to know their software works for their users

Sounds like you're not very far from understanding why they want better telemetry.

> Prior to release (...) and you know anything less will work too.

Things break in unexpected ways. OSs are complex systems and there's a lot of interactions between components. Homebrew's user base is enormous and very diverse. There's 2 different architectures, many OS versions, lots of environment variables that might be set differently in each user's systems, different versions of libraries, ... I could go on but I think you get the picture.

Edit: s/collected/collecting/

[grugagag]

Im okay with sending data to the server if things break and be asked each time to okay it. Is it safe to assume that if no errors or exceptions are encountered nothing should be sent back home?

[cassianoleal]

How can they monitor failure rate if they only track failures though?

[grugagag]

Why does it have to be a rate? Failures counts and failure types are good enough to extrapolate whatever else you need

[mo_42]

This discussion on GitHub reveals the mindset of the Homebrew people: https://github.com/Homebrew/brew/pull/6745

[kspacewalk2]

The Homebrew folks think this is a non-problem. You may agree or disagree, but the pull request is certainly a non-solution to this maybe-problem. Not just because it gained zero traction and did not get merged anywhere, but also because it's just an obscure band-aid. Either opt-out anonymous telemetry is a good idea, or it's a problem. If to you it's a problem, advocate for its removal in its entirety.

So even if I have an issue with telemetry, good on the Homebrew maintainers for ignoring this MR.

[pseudalopex]

The status quo is different variables and even different mechanisms for each program. Many badly documented. Only out of context is DO_NOT_TRACK obscure.

I think you know developers who reject informed consent will never adopt an informed consent model. The proposal was the best users could hope for realistically. Did you never compromise?

[cratermoon]

"Without telemetry, developers rely on bug reports and surveys to find out when their software isn’t working or how it is being used. Both of these techniques are too limited in their effectiveness."

https://research.swtch.com/telemetry-intro