[eliasgriffin]

Thank you for this awareness! I had to look up so many things you mentioned. I had some mind-explosions analyzing the-eye project. First of all, they were archiving 20k subreddits daily right up until Reddit changed its API. Maybe the-eye was the real reason!

Arxiv for sure. That was the first place I went, however no ability to download the entire database and scraping it would be a project on its own. It begs the question how can we as a civilization "own" our own data and not "rent" it through a website or paywall?

Which brings me to LexisNexus. That is a knowledge store we should all be able to access, crucial information about how the world works! It's only available for Governments and Law Firms...

In my hypothetical the Stackoverflow suggestion is on-point and would also be invaluable right in line with your AI strategem. We'd need some way to categorize, and order all that information to make it applicable would be the challenge I would think. Further, using your game theory of AI, it would need to also be in defense computer system ingestible maybe in real-time, so it would not be available/discoverable/decipherable/hackable up until the moment the data needed to be used. Some kind of OCR oracle for paper to digital "process" for AI defense.

I'm still following your breadcrumbs, I could say more!

[cookiengineer]

The point behind a cyber defense system that is incorruptible is that it needs to find ways to represent learning mechanisms in an unsupervised manner. The only thing that is really unsupervised (as of state today) is an evolutionary concept, that is why I stuck with ES/HyperNEAT in the past. NEAT as a concept allows to implement flexible adapters for all kinds of things, while also allowing to have a time aware strategy planning neural network (the CPPN) if it's e.g. based on an unfolding LSTM layer(s). The reason why NEAT fits so nicely for this use case is the predictability of results of given tasks, and the possibility to change the neural layer structures based on the tasks and agents, too. So in our case we built e.g. LSTM layers for data structures that need time awareness, while e.g. a bayesian layer makes sense to represent an index map that refers to linked knowledge tree branches.

The previous implementation also featured a custom kademlia DHT which was implemented in a shrinkable and fast-forwardable manner so that the generation of "already outdated solutions to identical problems" could be just skipped ahead directly to the leaves of the knowledge tree to save redundant computation time.

And the only swarm intelligence that is well researched enough to represent a delegated learning mechanism based on bayesian/statistical truths are bee swarms [1] :)

In case you are interested in a project like this, that's basically what we are building @ tholian.network

[1] Check out Honeybee Democracy by Thomas Seeley