[sirgawain33]

I used to think I had this problem, too. I developed an elaborate categorizing and indexing scheme. I tried to apply it outside of my personal knowledge, creating a crawler/indexer for research and web sites in one my areas of interest. I thought "if only we organize things better we can change the world!"

I realized over time that the collection wasn't the hard part. It was the categorizing and simplifying. The author hits on it a few times:

"It is the extraction and organization of the information that takes time... I know full automation is not feasible, since the imposition of meaning onto the raw information is something that I must do, not the computer."

In my experience, the simplifying is really where you get all the gain.

What is simplifying?

It's distilling a complex research paper into a few key data points. It's naming files well so that you can search them with Spotlight. It's learning to write more clearly. etc.

That last one -- clear writing -- should have been obvious. Good writers manage to convey so much information in so little space. How do they do it?

The parallel to programming should be obvious. When you name things well, they become very easy to find and use.

We can expand this to a bigger point: if you really want to get smarter, you need to be learning to simplify because your brain can only hold so much at once.

It's like learning the law of gravity rather than cataloging every time an apple falls from a tree.

The following unintuitive conclusion arises: you should be looking to make your "Personal Knowledgebase" more difficult to grow because it forces you to go through the simplifying process sooner.

I stole this idea from someone on HN: anytime I find an idea or quote that I think it important, I clip it into a Word Document and print it out. I keep these in a binder. I have gone back to these notes so many more times than anything that I have in any digital form. More significantly, these bits of information have influenced my life more often and more deeply.

[walterbell]

This could be viewed as layered ontologies, moving from the largest dataset to the most abstracted. Our categories are necessarily personal, but they eventually overlap with generic ones. Presently we don't have good tools to associate personal categories with those of larger datasets, because categories span apps, platforms, orgs & license regimes.

  Classified data
  Corporate data
  Licensed data
  Books
  Open internet
  Personal notes/kb
  Human brain

[gavinpc]

Writing is the best and hardest way to think. "Drop by drop," as I've heard say.

I love the binder practice. I got a printer about a month ago, printed a bunch of files (notes, incomplete songs, etc), deleted them, then reduced the mess down to a few pages. That was a productive day.

I submit that we have a false equivalency—nay, a false superiority of digital files over paper. After all, that's what we're all about here, right?

As you say, the constraints of paper are a feature, not a bug. I would have made this comment shorter, but I didn't have time.

EDIT Here you go, from right now: https://news.ycombinator.com/item?id=8272394

[EC1]

I feel so opposite to this entire thread. I have everything on my computer neatly organized, indexed, and tagged properly. For quick input of ideas, I learned cursive and I learned to write it quickly and legibly, and I use this system: personally its flawless - http://bulletjournal.com/

I have a few filled up notebooks and anytime I need some information I look it up in the index, and flip to it. I even write down URL's and descriptions to videos I liked. If I'm watching a video, I write down the timestamp, url, and title, thats how I index media. Maybe if the contents are important like dialogue or lyrics to a song, I'll jot down an excerpt.

It's really cool looking back on old notebooks. Can't wait to virtually go through every day of my life on paper when I get older, show my kids too.

I have a master list document on my computer that has every entry in the table of contents categorized as well, so that makes finding stuff easier too.

[zz1]

THANK YOU! It really looks great!

[walterbell]

Coppola interview on his binder for Godfather: https://www.youtube.com/watch?v=awce_j2myQw

[jacquesm]

> anytime I find an idea or quote that I think it important, I clip it into a Word Document and print it out. I keep these in a binder. I have gone back to these notes so many more times than anything that I have in any digital form.

I do the same but I do keep it in digital form because I just hate paper. There is a browser plug in called 'scrapbook' that works very well for me, I've tied it to a hotkey (shift-ctrl-B) to capture whatever is highlit on the screen without further confirmation or other interaction. I periodically dump the scrapbook and distill it.

I like it because it keeps the data locally rather than on some cloud service.

[ThomPete]

I wholeheartedly agree.

We had these things called RedBooks at MetaDesign in SF. A huge library of notes, sketches, diagrams and final work etc from each project. There is something about the tangability of a physical book when you want to retrieve information thats just much more intuitive than having some complex structure on some hard disk somewhere. It's simply the wrong way to use computers.

The force of digitalizing information is the ability to retrieve contextual information from a fuzzy collection of knowledge.

[chipsy]

Exactly, I've posted similar remarks on HN: I keep little quotes and principles in my main "notes" file so that they reach my eyes frequently. I've also recently made the move to formalize my personal project data a little(Fossil repo, add tickets to build a todo list). I still do everything digitally, but it's increasingly with more of a process - a process that itself aims to be as small and transparent as possible.

[Houshalter]

Is it impossible a technological solution exists? There has been a lot of research on automatically categorizing documents, or reducing them to a few dimensions.

Even if it works well, I'm not sure how much utility this information would provide.

[creamyhorror]

I'm a bit late to the party, but I agree with your sentiment that distilling (or synthesizing) is key to a long-term personal knowledgebase. I don't use paper, but I often refer to my digital notes.

I use an open-source mindmapping program called Freeplane to record most of my knowledge and thoughts: concepts I've encountered, products I've evaluated, snippets and webpages I've enjoyed, plans I've made, pieces of my writing. I have mindmaps like "Personal", "Ideas", "Projects", "Coding", "Devops", and "Ruby on Rails". I learned about Freeplane from a friend a few years ago and it's been amazing. The keyboard shortcuts are very fluid (use them!) and the program is very fast since it's text-based.

How it looks (at top level): http://imgur.com/QRkJ9vh

I make sure I summarize the takeaway from each link or snippet I put into my mindmaps - what I want is information that my brain can directly use when I refer to it in the future. If I want more detail, I can refer to the source snippet or page.

See the reviews for Freeplane, they're almost all 5 stars: http://sourceforge.net/projects/freeplane/reviews

Quoting myself from a year ago:

========

Regarding software, I use Freeplane to record the majority of the thoughts and information I want to preserve, and I think it works well for me. It doesn't have the sheer freedom of pen on paper - which I still use for mapping out thoughts that I'm not quite sure about or have complex relationships - but it serves me well in 1) categorizing random scribbles and steps into meaningful subcategories, and 2) crystallizing the final, synthesized thoughts I have on a matter.

Plus I keep my map files in my Dropbox so they're synced to all my computers (although I haven't set up the software to view them on my phone). Scalable, easily reorganized and expanded, cross-connectable, very fast to input, synced and backed up, free - what's not to like about software?

I agree with you on the usefulness of mapping for testing. It's definitely has helped me a lot in problem-solving. I'll throw out a few nodes that I think I need to investigate, explore each one a bit, write more items to consider, and whittle down or branch out as necessary. So after a while I resolve all the branchy, bushy sub-issues and have a reasonable game plan. Sometimes I dive into code halfway, but switch back to the map to record where I am and add new issues that come up that I need to resolve.

I also record most of the coding methods I find while working on tasks, in general form. So my mindmaps are also a web of how-to notes or a gigantic cheatsheet that details how to achieve any effect that I've previously worked through: from comparatively minor ones like the syntax for Rails migrations or opening a new window in JS, to larger ones like how to set up a Rails+postgres+nginx stack on Ubuntu, recording every action taken and issue encountered along the way. Comparisons of tools and databases and frameworks, mysterious bugs that I've run across, Sublime Text shortcuts - they all go into the maps.

I'm not sure I need to record every thought that goes through my head like the author suggests, but I think there's a lot to be said for keeping a comprehensive, organized knowledgebase.

=========

edit: I should also mention that I use Anki for language learning. However, I don't find it necessary to use it to memorize other types of knowledge, because I don't actually need to memorize much knowledge - I only need to categorize and store it for easy retrieval.

For language learning, Anki also becomes less useful as you get better in a language - at a point, extensive reading becomes more important, and it acts as a "natural SRS". (This conclusion is from discussions on Chinese-Forums.com.)

[antman]

I have to add docear[1] which is Freeplane and can also sync highlights from pdfs.

[1] http://www.docear.org

[thallukrish]

simplification helps only in the way you start looking at things later. But then when you communicate to another person, he still needs all the data to come to the same level of simplification you achieved. Being able to organize the way your brain looks at it at some point in time and later reconfiguring as things evolve is what may be needed.