[lmeyerov]

Think of it as an ensemble for blending your normal NN features (ex: RNN for time/clickstreams) with a model that can also leverage useful graph features (document citations, app logins, chemicals connecting, social graphs).

We think a lot about security/fraud and digital journeys, where NN + xgboost are popular in general, and graph is used seperately (or upstream) for looking at broader structure. GNNs help blend these models. For example, in analyzing malicious user accounts (ex: misinfo on twitter), we already get many time/nlp/etc scores for whatever events/entities we look at, and use the social network structure to ensure better propagation/blending, similar to why boosting and ensemble methods became popular to beginwith. Feel free to DM if interested, we are quite excited by this space and working on some things here.

[H8crilA]

So, if I understand you well, it would be something like this. Inputs:

1. {x0, x1, ...} - nodes in the graph, say users

2. Bunch of edges like {x_i, x_j}, say social connections

3. Some raw or processed features on nodes, say the text of posts, age of the account or some nlp-based scores for posts

4. Some raw or processed features on the edges (in particular maybe some coloring)

Before: people would train various classifiers/regressors directly on the nodes and/or edges, then maybe use the graph structure to propagate the scores.

After: But instead you could train whatever objective you have from raw features on the edges and nodes, with some extra message passing between nodes and edges. For example train (some of) that nlp-based classifier together with the graph part. And the benefit would be that, for example, you can extract some signals from the NLP part that would be more useful in determining the properties of neighbors, but not necessarily as useful in determining the properties of the current node/edge.

Question - what's the maximum range of such message passing? Sounds a bit like an RNN, where the unroll depth can be an issue. Though in practice most graphs have a low average path length, so maybe this is not a particularly big problem.

Although if you start unrolling graphs you'll very quickly load ~everything, so I guess the training must be completely reworked (flush data to distributed storage frequently then shuffle for the next step) or you cannot unroll further than maybe a few steps.

[lmeyerov]

Yes I think you are seeing it

Before: People might precompute graph scores ("pagerank", ...) and use as features for tabular NNs. Or use simpler and slow GNNs like GraphSAGE bc the domain fit was great (ex: Pinterest social recs)

After: heterogeneity and scale for graphs that fit in CPU RAM (1TB) w decent GPUs

Re:unrolling, yeah a bunch of papers there :) sampling, artificial jump edges, and adversarial techniques have been helping with aspects of generalization (far data, unbalanced data, ...)

[quibono]

I remember reading a bit about GNNs circa 2019. At that time it seemed to have mostly to do with point clouds (for LIDAR data and for 3-D modelling mostly) but I imagine things have changed lots on this front. Are there any interesting papers/resources you could recommend for one to get back up to speed?

[atomflunder]

From what I can tell, the field is indeed evolving very rapidly, but I have only worked on a specific application (knowledge graph completion), so I can't give an overview over all the current day applications. I can, however, recommend William Hamilton's excellent text book, which is available online [1].

[1] https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf

[lmeyerov]

For enterprise relevance in our world, the exciting things have been handling heterogeneity via things like RGCNs, and handling bigger scales via DGL (GPU tricks, sampling tricks, ...). Imagine fraud, hacks, and entity resolution from everything you've recorded on a user interacting with a system.

There are important cases like maps and chemistry that take more specialized techniques, but we focus on events/logs/etc. So less to say on the niche stuff, even if those niches cover big use cases like "how google maps works" or "how google auto-designs their TPUs"

For the logs/events/transactions/clicks/devices/users/accounts cases, happy to chat, but maybe not as useful elsewhere :)