|
|
|
|
|
by sermakarevich
5 days ago
|
|
git or versioned path on cloud storage should work. Format is more important - I think hierarchical knowledge base is the best thing on the market atm. You keep main page with refs and short summaries to 5-10 topics. Model reads it and decides where to drop down next. You optimize the breadth and depth of topics for optimal performance. Dropping down is not a through away tokens - this helps model to understand wider context. I use it in quite a few repoes: -- https://github.com/sermakarevich/ai_knowledge_wiki Curated extraction of summaries from AI-related research papers, organized as a hierarchical wiki optimized for Obsidian and LLMs -- https://github.com/sermakarevich/chunker/tree/master/output/... Chunker processes a document into a hierarchy of self-sufficient chunks and multi-level summaries, producing a set of linked markdown files that an AI model (or a human) can explore through progressive disclosure -- starting from a high-level overview and drilling into details on demand, without ever loading the entire document. -- https://github.com/sermakarevich/kaggle_wiki A structured knowledge base of Kaggle competitions — solutions, notebooks, and indexes — built for fast lookup and reuse. |
|