[CuriouslyC]

You don't. generate a hierarchical clustering of the data, then collapse nodes into groups to get under a data set size threshold at any given view distance. That gives you full interaction and the ability to do mouseover info on groups, while being able to zoom in and interact with individual nodes if you want.

[infecto]

This is the way imo. Nobody is consuming 100b nodes in a chart.

[moshegramovsky]

This is mostly a data structure problem. I am certain this can be made interactive, but it will require some elbow grease.

If you want it to be interactive, you will need to figure out a few things:

1.) how to format the data so it can be streamed off disk. 2.) how to cull the offscreen bounding boxes quickly. 3.) how to cull tiny bounding boxes quickly.

The central problem is finding a way to group the nodes efficiently into chunks. A 2D approach is probably best. You would then have something that could be rendered efficiently.

Other than that, maybe a point cloud renderer? There might be one you can buy off the shelf, or something open source.

[CuriouslyC]

First step is to generate a graph distance matrix to use as features.

You can do the hierarchical clustering using HDBScan probably in reasonable time, it's a fast algorithm.

To have any sort of 2d display you need to project the nodes, which might require some form of PCA given the data set size. UMAP might also work.

From there, you can use an R* tree in conjunction with "cut-depth" cluster segmentation tied to zoom level with additional entity selection based on count and centrality. If you load it in postgres PostGIS can do this in one query.

All pretty straightforward stuff.

[aeonik]

It really depends on what the nodes represent, right?

A 1080p monitor has:

1,920 × 1,080 = 2,073,600 pixels

Each pixel can display 32-bit color, which equates to:

2^32 = 4,294,967,296 colors

So, while each pixel can display one of 4.3 billion colors, the monitor can display combinations of those colors across its 2,073,600 pixels. The total number of possible color combinations on the screen is astronomical.

The actual number of possible combinations is:

4,294,967,296^2,073,600

[abeppu]

Aside from tgv's correct point that this is implicitly a recipe for something that isn't useful as a visualization, I think even if we were able to distinguish 4B colors and make sense of each pixel -> color assignment ... the math isn't on your side. You responded to a statement about nobody consuming a graph of 100B nodes. Suppose we don't have any concept of edge weight, and an edge is either present or not, but edges are directed, then you have 100B^2 (i, j) pairs for potential edges, each of which is either present or not (i.e. 10^22 edges, each of which is a bit).

4,294,967,296^2,073,600 is very large but 2^(10^22) is much much larger

[tgv]

That way of looking at it doesn't make sense. When visualizing a graph, you want to see the connections between the nodes; coloring each node individually almost never makes sense; and the eye cannot distinguish 4B colors.

[hombre_fatal]

You end up bucketing those colors into differences the human eye can see, so you end up with a much smaller domain.

You do something similar with 100B data points since you're not literally looking at the relation between individual nodes when all 100B are on screen at once.

[justsomehnguy]

> Each pixel can display 32-bit color

It's only 24-bits of visible colours.