Hacker News new | ask | show | jobs
by good-idea 77 days ago
Two advantages I see to files in folders are the ability to browse and the ability to easily read - by both LLMs and humans.

But I also agree that this has limitations. If I were to challenge you and say that Obsidian solves this, or gets close to solving it, what gaps would you say were left unfulfilled?

I am because I'm working on something in this space and your comment touches on my three main ingredients: text files, SQLite, and graphs.
1 comments
zoogeny 75 days ago
Very good point and question. I am not intimately familiar with Obsidian because I haven't used it. My understanding is that it indexes directories of .md files to provide a graph-like interface over them, using the contents of the files themselves to store meta-data (e.g. in the form of links and YAML frontmatter).

Let's imagine a .zip file format that combines:

- A directory of .md and/or .xml files (there is more to structured text than markdown)

- A sqlite db in the root for some metadata

- A graph representation (RDF, triples, whatever) of the files

That is directionally what I am talking about, but with a standard spec so that someone receiving a project bundle knows how to parse it all.

You can still have all of the Obsidian stuff over the contents of the directory. But you can also store much more interesting meta-data in the db that may not be appropriate to put in any individual file (or in some kind of pure data .md file). It is something that can be sent over the network a bit more conveniently than a directory. And if you standardize the db schema and the graph you can have immediate interoperability between systems.

In fact, given advanced enough LLMs, you could just have an README.md or something in the root that sketches out the directory structure and files, the db schema for the metadata, and then just expect that the agent, knowing how to load and query a SQLite db, will be able to work with it. But I think there is value in having some loose standards around the db schema, some YAML frontmatter conventions, some well known graph representation, etc.

link