[zopf]

I wonder to what degree an LLM could now produce frames/slots/values in the knowledge graph. With so much structure already existing in the Cyc knowledge graph, could those frames act as the crystal seed upon which an LLM could crystallize its latent knowledge about the world from the trillions of tokens it was trained upon?

[tkgally]

I had the same thought. Does anybody know if there have been attempts either to incorporate Cyc-like graphs into LLM training data or to extend such graphs with LLMs?

[radomir_cernoch]

From time to time, I read articles on the boundary between neural nets and knowledge graphs like a recent [1]. Sadly, no mention of Cyc.

I'd bet, judging mostly from my failed attempts at playing with OpenCyc around 2009, is that the Cyc has always been too closed and to complex to tinker with. That doesn't play nicely with academic work. When people finish their PhDs and start working for OpenAI, they simply don't have Cyc in their toolbox.

[1] https://www.sciencedirect.com/science/article/pii/S089360802...

[viksit]

oh i just commented elsewhere in the thread about our work in integrating frames and slots into LSTMs a few years ago! second this.

[thesz]

There are lattice-based RNNs applied as language models.

In fact, if you have a graph and a path-weighting model (RNN, TDCNN or Transformer), you can use beam search to evaluate paths through graphs.

[mike_hearn]

The problem is not one of KB size. The Cyc KB is huge. The problem is that the underlying inferencing algorithms don't scale whereas the transformer algorithm does.