[shaburn]

LlamaIndex, HuggingFace, LangChain, and Ludwig are functioning as partial proxy aggregators of what is working and working better than alternatives. Most people in this space have multiple models for virtually every step in their heads. You cannot and should not process them. Thus you need aggregation and filtration on some objective(s) and presumptions with a purpose or you're a boat on a turbulent sea.

One simple tool I use to evaluate is how easy is it to spin up a working demo, in isolation or context of other tools, to show the maturity of thought on the idea moving to application. If there is no benchmark on output(s) before you start that is more science/fantasizing than business. but maybe that's your goal, just don't delude yourself in what you're doing. I also look up project leads on Linkedin and such since these things require push and without a good promoter, usually don't pan out(see crypto for example).

Buried lead: I've built a few tools allowing me to parse the tree and graph structures of new projects in charts/schemas to get quick visuals for myself and the LLMs I use to code. Additionally, I have built tooling to test rapidly. For if everything is dynamic, optimizing measurement is quite valuable in and of itself.