[loup-vaillant]

This hints at a more general problem: how does one gather a high-quality repository of knowledge on any given subject? How do you get enough stuff? How do you keep the noise down?

I believe some subjects make the problem harder than others. Programming for instance is full of hard to check claims. Even established techniques are hard to assess. Say you need to parse stuff. Will you go recursive descent? LALR? Earley? PEG? Might depend on what you want to parse, which environment you're working in, how much time you may invest… Or say you write a compiler. Will you use OCaml/F#/Haskell for the ease of handling recursive data structures? Or do you want C/C++ because of the speed, and you know tricks to avoid recursive data structures anyway?

One tempting solution is to start a secret society dedicated to hoard knowledge on the chosen subject. It would be hard to get in, but once there you'd only get quality stuff. (Or you might have gotten into a self-delusional sect…) The idea is, maybe if knowledge was visibly scarce and hard to obtain, instead of merely buried under a mountain of noise, we would treat it with the respect it deserves.

[mch82]

For most of human history groups of people have operated to restrict the flow of knowledge for the benefit of the controlling group. Technological advancement has exploded since governments began to incentivize release of information through the concept of patents.

Some examples...

See "guilds" https://en.m.wikipedia.org/wiki/Guild

See "companies" with "trade secrets".

See Vatican Library (no longer fully secret) https://en.m.wikipedia.org/wiki/Vatican_Library

See "classified information" https://en.m.wikipedia.org/wiki/Classified_information

[klenwell]

Had something like this debate with manager at last job. Came to a head in my annual review.

Manager: "As senior developer, you lack sufficient knowledge of our most important application."

Me: "Wait. I am the one who wrote most the documentation that is in our knowledge base for that application. I am the one who set up the knowledge base."

Manager: "That is not knowledge. Knowledge is what is in your head."

Me: blinks incredulously

I departed the company shortly after this exchange.

But I think you're right. It was certainly the manager's idea that if knowledge was visibly scarce and hard to obtain, his position would be treated (and generally was, among his superiors) with the respect he felt it deserved. But I'd have to assert that this secret society dedicated to hoarding knowledge was a self-delusional sect.

That's generally been my experience.

[brazzledazzle]

That's so bizarre. Was he trying to create some kind of official narrative on paper that he had the knowledge and you didn't? Or was he trying to tell you to document less and hide information?

[klenwell]

> Was he trying to create some kind of official narrative on paper that he had the knowledge and you didn't?

Yes, this. We had actually worked together amicably for several years, he as senior, I as mid-level. Then we both got bumped up. As part of my new responsibilities as senior, I tried to surface issues and confront them transparently. I was mindful not to point fingers or show anyone up but rather identify them as shortcomings in our practices or policies.

For cultural reasons, I think he felt threatened by this and assumed it was his responsibility to hide issues and save his own face. It fell apart pretty quickly. It sucked but I've used it as a career lesson.

[eponeponepon]

Self-evidently it is possible to gather a high quality repository of knowledge; it's been done many times in history.

I suspect that the real solution is an absence of secrecy - indefinite hoarding of information is no help to anyone except those who want their egos stroked by the number of "things" they have, and knowledge that nobody can access is barely knowledge at all.

As far as noise goes, you can never hope to rub it out completely. There will always be transcription errors, deliberate mistruths and gaps - the most you can do is carefully curate the repository to make sure you catch and repair as much as possible.

Essentially, the closest thing we have is probably Wikipedia. I don't know if that's especially complimentary to us as a species.

[js8]

This was already tried with Encyclopedia Britannica and it doesn't quite work.

The truth is, if you want high quality, more eyes the better. (And the article mentions that magical art was greatly improved by general accessibility of things on the Internet.)

[DougN7]

Why didn't the encyclopedia work (for its time)? Before the Internet it was the best, most information dense, source available. Not perfect, but better than any available alternatives.

[sweettea]

Heck, _during_ the age of the internet it's the best, most information dense source available, far outclassing Wikipedia on the most used topics.

[seanp2k2]

Curious about an example of this. As someone who spends many hours reading Wikipedia, the time and effort put into very niche subjects seems fantastic. I haven't opened a physical encyclopedia in probably 10 years.

[hackuser]

Britannica is online; you don't need to open a book. Go search it now.

[vacri]

Just trying now, it's slow, doesn't take me to the page by name ('Iraq' was my search), and was still trying to load from adservers as I type this. Slow to load, poorer layout that is harder to read (but not terrible), and split across multiple pages requiring another slow load. The 'demographics' section that I was looking for is needlessly split across pages. The page chugs as I scroll it up and down to read the one paragraph on-screen at any time (on a laptop with a decent CPU).

Is the information better? Maybe, maybe not. I don't know enough about the country to say. But it's a chore to read. Last updated in June 2013, so that's three-and-a-bit years without updates in a rapidly-changing part of the world.

My mother sold World Book encyclopedias when I was a kid, and we had a set. I grew up reading those cover-to-cover. There's something beautiful about having info at your fingertips that can be consumed at the speed of reading. Paper encyclopedias have it, and wikipedia has it, but online Britannica does not. Well, not from my excursion there just now.