[sitta]

>> How do you ship a Java library that uses native code? Python/Node/Ruby worlds know, but OpenJDK ignores the question.

>It doesn't. jmod files and the jmod tool were added in JDK 9 precisely for that (and other things, too).

I have never seen a lib package native dependencies using jmod. It's not even clear to me how this would be done. Everyone I'm familiar with bundles binaries inside the jar. Can you point to an example? I would love to learn a better way of doing this.

[pron]

That's what JavaFX does (https://openjfx.io/openjfx-docs/#modular), but the problem is that Maven and Gradle don't support that well, which is why few libraries do that.

The idea is that an application is best deployed with a custom runtime created with jlink (well, all Java runtime these days are created by jlink and so every Java program uses jlink whether it knows it or not, but few use it directly because, well, Maven and Gradle don't support jlink well, and so few applications use jlink well), and jmod files are jlink's input from which it generates the image.

Anyway, build tools aside, a library can be distributed as a jmod file, consuming it with jlink places the native libraries in the right place, and that's about it.

[nvm0n2]

JavaFX doesn't use jmods, not really. Try following the tutorials for getting started and see for yourself: you end up using custom JavaFX build system plugins that download jar versions of the modules, not jmods. Also some alternative JDK vendors pre-ship JavaFX in their JDK spins to avoid people having to use jlink.

I've worked with jlink extensively. The problems are greater than just Maven and Gradle, which at any rate both have quite sophisticated plugins for working with it. There was just a major error at the requirements analysis phase in how this part of Java works. Problems:

1. You can't put jmods on the module path, even though it would be easily implemented. They can only be used as inputs to jlink. But this is incompatible with how the JVM world works (not just Maven and Gradle), as it is expected to be able to download libraries and place them on the class/module path in combination with a generic runtime in order to use them. Changing that would mean every project to get its own sub-JDK created at build time, which would explode build times and disk space usage (jlink is very slow). So nobody does it, and no build system supports it.

2. When native code gets involved they are platform specific, even though (again) this could have been easily avoided. Even if JVMs supported jmods just like jars, no build system properly supports non-portable jars because the whole point of a jar is that it's portable. It has to be hacked around with custom plugins, which is a very high effort and non-scalable approach.

3. jlink doesn't understand the classpath at all. It thinks every jar/jmod is a module, but in reality very few libraries are usable on the module path (some look usable until you try it and discover their module metadata is broken). So you can't actually produce a standalone app directory with a bundled JVM using jlink because every real app uses non-modularized JARs. You end up needing custom launchers and scripts that try to work out what modules are needed using jlink.

4. The goal of the module system was to increase reliability, but it actually made it lower because there are some common cases where the module system doesn't detect that some theoretically optional modules are actually needed, even in the case of a fully modularized app. For example apps that try to use jlink directly are prone to experiencing randomly broken HTTP connections and (on desktop) mysterious accessibility failures, caused by critical features being optional and loaded dynamically, so they have to be forced into the image using special jlink flags. This is a manhole sized trap to fall into and isn't documented anywhere, there are no warnings printed. You are just expected to ship broken apps, find out the hard way what went wrong and then fix it by hand.

At some point the JDK developers need to stop pointing the finger at build tool providers when it comes to Jigsaw adoption. It's not like Maven and Gradle are uniquely deviant. There are other build systems used in the JVM world and not one of them works the way the OpenJDK devs would like. They could contribute patches to change these things upstream, or develop their own build system, but have never done it so we end up with a world where there are ways to distribute libraries that work fine if you pretend people never moved beyond 'make' and checking jars into version control.

[pron]

> You can't put jmods on the module path, even though it would be easily implemented.

Yes, it could be easily implemented, although things would work nicely even without that. You make it sound as if there would have been proper build tool support if that were the case, but JARs can be easily put on the module path and still build tools don't properly support even that yet.

> When native code gets involved they are platform specific

Well, yeah. Native code is platform specific, and that's one of its main downsides and why most libraries don't (and shouldn't) use it. But when it is used, it's used for its upsides.

> At some point the JDK developers need to stop pointing the finger at build tool providers

All of the problems you mentioned could only be solved by build tools, but we're not pointing fingers in the sense that we blame build tools for things being bad. After all, modules have been very successful at allowing us to do things like virtual threads and FFM and remove things like SecurityManager. Build tools providers can have their own priorities, just as we do. But if you want to enjoy your own modules or 3rd party modules then you'll need good support by build tools.

> They could contribute patches to change these things upstream, or develop their own build system, but have never done it

Yet. There were more urgent things to work on, but maybe not for long.

[nvm0n2]

Gradle will put JARs on the module path if the app itself is a module. If JMODs were an extension to the JAR format rather than a new thing, and if there was no such thing as the module path (modules were determined by the presence of their descriptor), then nothing new would be needed in build systems and everything would just work.

[pron]

No, working with modules requires the ability to place some JARs on the classpath and some on the classpath arbitrarily. Any kind of automatic decision regarding what should be placed on the classpath and what on the module path misunderstands how modules are supposed to work. The -cp and -p flags don't and aren't supposed to differentiate between different kinds of JAR contents. If they did, you're absolutely right that the different flags wouldn't be needed -- the JDK could have looked inside the JAR and automatically say this is supposed to be a module or not; the reason they are needed is because that's not what the flags mean.

A module in Java is really a different mode of resolving classes and applying various encapsulation protections. If x.jar is a JAR file, `-cp x.jar` means "apply certain rules to the resolution and access protection of the classes in the JAR" while `-p x.jar` means "apply different rules for those same classes". In most situations both `-cp x.jar` and `-p x.jar` would work for the same x.jar, applying those different rules, regardless of whether x.jar declares module-info or not. The decision of which rules need to be applied belongs to the application, not the JAR itself; being a module or not is not something intrinsic to the JAR, it's a rule chosen by the application.

It's a little like choosing whether or not to run a program in a container. You can't look at the executable and say this should run in a container or not. The decision of whether to set up a container is up to the user when configuring their setup. module-info basically means: if the user sets up a container to run me, then these are hints to help configure the dockerfile. In an even stronger sense, a module is similar to a class loader configuration; some classes may work better with some classloader configurations than with others, but ultimately the decision on the setup is not something intrinsic to the classes but to the application that loads them, and the same goes for modules.

So having the build tool or the JDK guess which rules need to apply to which classes makes as much sense as having them guess the proper heap and GC configuration the application needs -- you can have an okayish default, but for serious work the app developer needs to say what they want. The JDK makes it very easy; build tools make it very hard.