Ask HN: How do people keep track of organizational knowledge?

[kadhirvelm]

With code coming in faster and faster, we've been losing track of context, what's out of date, the source of truth, etc. Also with individuals spending less and less time on any one problem, finding an expert to answer questions has become challenging. People don't have that single domain of expertise like we used to. There's too much surface area to cover.

Are we the only ones dealing with this? How's everyone else handling it?

[kjellsbells]

Two things.

1. If it's not written down, it does not exist. This needs to be a ruthlessly enforced rule. I know we all love those Mel the Programmer stories, but that's a story about fragility. Similarly if you are relying on SRE or dev heroics to solve field problems, you have a gun to your head and you need to solve it yesterday.

2. The TCO of knowledge is dominated by the maintenance cost, not the creation cost. I see this mistake made a lot. Anyone can create a confluence page, Slack comment, SharePoint site or a git comment. The org must make it someone's job to grow, weed, and maintain these gardens. It does not matter what system you use, but someone needs to look after it.

You get a bonus for building information "data structures" that make it easy for people to pull data out as your org changes, but this is probably something you should accept that you'll get wrong as your org grows. AI hides this problem because it makes search so easy, sometimes, but it will come and bite you hard one day, so be ready for it.

[AnimalMuppet]

If it's not written down in a place where other people can find it, it does not exist. So the place has to be either standardized or searchable, or preferably both.

[hash0]

In my line of work (IT Operations for a large international bank), we came up with a classification framework that tried to gauge information along two axes: Importance and Frequency of Use.

Importance is the more obvious one - there is critical info you absolutely need as a basis to understand the system, then there is also minor details. The difference is in an emergency, lack of the former will break your attempt to fix the system, lack of the latter will merely inconvenience you.

The Frequence of Use axis is the one that brought value to our operations when we introduced it. That's the question "How often will I need this info?". Regularly re-occuring, detailled procedures need to be accessible easily, so they go to a prominent Confluence page. But then there is information that you will hardly ever need, some arcanae about the system that only the old timers knew but that needed to be fixed in writing.

The most vexing bits of info are those that are highly important but are rarely ever needed. Do you plonk them into Confluence? Might not be the best idea because then you don't know how to quickly find it when need be. So we had an extra category for just that sort of thing which became our go-to info hub for all sorts of highly urgent but at first intractable issues with our system.

[andyjohnson0]

> How do people keep track of organizational knowledge?

In my experience:

1. Well meaning people try to build wikis/confluences/whatever that turn into backwaters of slowly rotting stuff

2. Beyond a certain extent that they can hold in their heads, most other people give up.

3. Some people hoard knowledge and use it for personal advantage.

I dont know what the solution is. Maybe an ai that reads every chat and email and commit - and builds some kind of browsable, queryable, executable knowledge trove.

[al_borland]

I’ve started hoarding knowledge, but not for personal advantage. There were two trends that led to it.

1. People would go in and “clean up” what they thought was a rotting Confluence. They were new-ish to the company and didn’t have the knowledge or experience to know what they were looking at. Rather than ask, they just deleted stuff. Some of those things they deleted were docs I wrote, several over 20 pages long that I referenced regularly… gone.

2. Constant re-orgs and tool changes meant it was becoming difficult to know where anything goes. Not long ago I had 4 bosses in 4 years. I got tired of starting over and shuffling things around.

If I keep what I need, no one else is going to delete it without consulting me, and I don’t have to try and migrate and re-organize it every time I get a new boss or management decides we’re using a new system.

I guess a third thing, which is less of a trend, is friction. The collaboration platforms are much higher friction than something like a local Obsidian vault.

I’ve probably written and shared more documentation than everyone else on my team combined, but at some point, I got tired of the uphill battle. I still do write and share stuff I think will benefit the team, but I no longer try and make everything public for the reasons listed. If someone asks for something I have, I will always send it over, and having my notes in markdown now makes that easy.

[kadhirvelm]

Do you think having an AI do that is 1) possible and 2) people will like that? Seems really invasive, no?

[andyjohnson0]

Its possible - teams chats where I work can have ai transcription enabled, for example.

As to whether people like it: I was thinking of a work context, where expectations of privacy are lower. I guess it could be an opt-in for group chats.

[al_borland]

Going into all chats is a horrible idea. Some chats between small groups are meant to stay that way.

[andyjohnson0]

I agree, not all chats. But it could be an opt-in for group chats.

[autoshiftops]

IMO, earlier we used to spend time on the issue, analysis(RCA), confluence documentation, however, now organizations are spending dollars on AI and expecting Individual Contributors to use AI to deliver the product in weeks rather than original timelines.

This behavior forcing every individual to skip the other necessary activities like RCA and documentation and straightly trying to deploy the product into production.

[Tadace]

We still make sure to maintain a healthy directory and "backlinks". And as soon as new updates are in, we have someone on the team dedicated to clarifying changes and updating the Wiki - Confluence pages, Notion, which are adequately linked to tickets and shared drives where necessary.

This way, new and updated context is not lost on the next user.

[lyfeninja]

Isn't that the purpose of wikis? Not saying they work all that great but I think that's the goal.

I do like the idea of having a live editable wiki as the knowledge base a LLM can reference.

[tenzo]

we've faced this same problem but thinking about it a little deeper reminded me that this issue is no different to pre-llm world. someone creates a spec, someone or a team build/implement the spec and after actioning any feedback from testing or from customers, the product evolves and is no longer aligned with the initial spec.

i think one way to solve this is having an someone or an agent update the initial spec with any new changes as and when they are made.

[al_borland]

I’ve seen at least one person where I work doing this. They have an agent doing massive updates on Confluence constantly. It seems like a mess. It checks the box for having documentation, if that’s all anyone cares about. But there is no way people are actually reading it, looking for changes is impossible, since the LLM re-writes the entire page every time, and even not knowing their project well, it was full of hallucinations and nonsense.

If the LLM is going to write the doc anyway, why not just do it on demand from the code if/when it’s needed, instead of burning tokens on documents that no one will ever read?

I also think a lot of the value in those historical docs are the reasons why certain decisions were made, as it can be important context if there is every a discussion on doing things differently. The LLM likely won’t have that information, or may just make up a bunch of reasons the business doesn’t actually care about.

[1337h4xx]

Just do what AI does to manage context: write docs, regression tests, content index, tools/skills, and CLAUDE.md

[elenaviter]

We capture organizational knowledge during each change in the product. Contributors can be agents and people and have different roles. Agents must follow our procedures, and one of the most important assignments is to journal their work in our knowledge base, in a feature session journal, so they are not losing the track somewhere in their harness.

Our "project knowledge" consists of artifacts of such types:

- product documentation (made by human). Intended product behavior and capabilities.

- technical documentation (maintained by human and agent who work together on feature). Implementation documentation.

- feature session journal (made by agent and human who work together on specific feature). Per feature session. Contains original problem, gaps, problems, misunderstandings and failed attempts.

- wiki and ontology articles for targeted knowledge retrieval. Built by Knowledge Keeper procedure under human supervision on demand. Consolidate the knowledge across subsystems.

- code and tests (mostly written by agents).

All these knowledge artifacts (except code and tests) are mostly markdown articles. They are stored in git - this allows versioned storage, authorship and diffs. All articles have front matter with summary, keywords, status (draft or reviewed and approved by human) and links to related articles, so we and agents know what is relevant and what the source of truth is. Wiki and ontology articles hold the semantic relationships for better retrieval.

Typical process of changes: As agent and human work on the feature, this is captured in the session journal. At any moment in this process, the engineer who works with the agent can ask the agent to hand off their work to other agents, integrate in the technical documentation or handoff to a Knowledge Keeper. Such handoff includes the message with documentation links and the link to the session journal, so one who continues starts with the context.

The Knowledge Keeper is a supervised procedure that keeps wiki and ontology aligned with the product documentation, technical documentation, known issues. Knowledge integration process is currently manually initiated by the handoff of the human who worked with agent on the feature. The Keeper prepares the integration plan which a human approves. Keeper builds the index on top of wiki and ontology after integrating the knowledge and tests the knowledge retrieval via MCP to check if integrated knowledge retrieves correctly and reports to the human maintainer. MCP allows the consolidated view of the product and its capabilities. Its answers point to the primary docs, therefore users do not need to find one person who remembers everything.

[toomuchtodo]

Confluence, Notion, Wikis, etc.

[kadhirvelm]

Do you all rely on people writing things down as they go? or do you have some automated system?

[toomuchtodo]

People. Time is carved out for both documentation and its review, like a PR review. Otherwise, how would you trust what is being documented? A human must remain in the loop if accuracy is a component of success criteria for organization documentation. I say this familiar with Karpathy's LLM Wiki pattern.

[al_borland]

On a past team of mine we used to bake documentation into our definition of done for user stories. If the doc wasn’t written/updated, then it wasn’t done. It was tough for some people at first, but after a while it became second nature for them. We also learned that it takes far longer to write that stuff than anyone estimates.

[midzer]

I organize myself in a some_project/ directory.

[richardprince4]

I use federated wiki for just about everything

[4lx87]

All I know is creating yet another place to store data is not going to solve your problem.

The problem of scattered data is to put data in one place.

[ipiyer]

Obsidian.