[coderdd]

Great to see this space moving! Any pointers on diff vs Kythe? Kythe has a mostly fixed schema, for one.

One of the pain points using Kythe is wiring up the indexer to the build system. Would Glean indexers be easier to wire up for the common cases?

Other is the index post-processing, which is not very scalable in the open source version (due to go-beam having rough Flunk support, for example).

Third, how does it link up references across compilation units? Is it heuristic, or relies on unique keys from indexers matching? Or across languages?

[simonmar]

Kythe has one schema, whereas with Glean each language has its own schema with arbitrary amounts of language-specific detail. You can get a language-agnostic view by defining an abstraction layer as a schema. Our current (work in progress) language-agnostic layer is called "codemarkup" https://github.com/facebookincubator/Glean/blob/main/glean/s...

For wiring up the indexer, there are various methods, it tends to depend very much on the language and the build system. For Flow for example, Glean output is just built into the typechecker, you just run it with some flags to spit out the Glean data. For C++, you need to get the compiler flags from the build system to pass to the Clang frontend. For Java the indexer is a compiler plugin; for Python it's built on libCST. Some indexers send their data directly to a Glean server, others generate files of JSON that get sent using a separate command-line tool.

References use different methods depending on the language. For Flow for example there is a fact for an import that matches up with a fact for the export in the other file. For C++ there are facts that connect declarations with definitions, and references with declarations.

[mrazomor]

In case using Kythe was an option, what was the rationale for not using it?

One major limitation of Kythe is handling different versions. For example, Kythe can produce a well connected index of Stackage, but a Hackage would have many holes (not all references would be found, as the unique reference name needs the library version). How Glean handles different library versions?

EDIT: the language agnostic view is already mentioned.

[Game_Ender]

Is there an example of using the C++ indexer? I saw hack and JS on your site but missed C++ (Python would also be amazing!).

[simonmar]

We want to open-source the C++ and Python indexers but they're not ready yet - we have to separate them from internal build-system-specific bits.