[roykishony]

thanks! indeed currently we only provide the LLM with a short tldr created by Semantic Scholar for each paper. Reading the whole thing and extracting and connecting to specific findings and results will be amazing to do. Especially as it can start creating a network of logical links between statements in the vast scientific literature. txtai indeed looks extremely helpful for this.

[alchemist1e9]

Excellent! I’m glad my input was interesting.

txtai has some demos of automated semantic graph building that might be relevant.

I noticed you didn’t really use any existing agent frameworks, which I find very understandable as their value added can be questionable over DIY approaches. However txtai might fit better with your overall technology style and philosophy.

Has your team studied latest CoT, OPA, or research into Cognitive architectures?

[roykishony]

thanks. will certainly look deeper into txtai. our project is now open and you are more than welcome to give a hand if you can! yes you are right - it is built completely from scratch. Does have some similarities to other agent packages, but we have some unique aspects especially in terms of tracing information flow between many steps and thereby creating the idea of "data-chained" manuscripts (that you can click each result and go back all the way to the specific code lines). also, we have a special code-running environment that catches many different types of common improper uses of imported statistical packages.

[alchemist1e9]

“data-chained” will be very valuable, especially for the system to evaluate itself and verify the work it’s performed.

this is obviously just my initial impression on a distracted Sunday but I’m very encouraged by your project and I will absolutely be following it and looking at your source code.

The detractors don’t understand LLMs and probably haven’t used them in the way you have and I have. They don’t understand that with CoT and OPA that they can be used to reason and think themselves.

I’ve used them for full automated script writing, performing the job of a software developer. I’ve also used them to create study guides and practice tests, and then grade those tests. When one implements first hand automated systems with agent frameworks using the APIs it gives a deeper understanding of their power over the basic chat usage most are familiar with.

The people arguing that your system can’t do real science are silly, as if the tedious process and logical thinking is something so complex and human that the LLMs can’t do it when used within a cognitive framework, of course they can!

Anyway I’m very exited by your project. I hope this summer to spend at least a week dedicated to setting it up and exploring potential integrations with txtai for use on private knowledge bases in addition to your public Scholarly published papers.

[roykishony]

and yes we are implementing CoT and OPA - but surely there is ton of room for improvements!