[clra]

I don't follow Java too closely and the world "module" is so hopelessly generic that I had a hard time understanding what the article was talking about. It links to this document [1] though, which is a long read, but does an excellent job of describing of describing all the ins and outs of the module system.

(tl;dr Packages can now declare themselves as modules instead of simple namespaces, and they get to choose exactly what they export and what other modules they require. Projects can now resolve their types from within modules with a module path instead of on a per-type basis from the class path. It's a huge step forward for Java modularity and dependency management.)

I have a hard time being too impressed because more modern languages are doing this out of the gates and with far fewer hacks and more out-of-the-box tooling (e.g. Rust), but acknowledge just how monumental of an effort it was to design the new system and then build it into Java's core in a way that's mostly backwards-compatible. Java's going to be around for a long time, and this is a big long-term gain for the ecosystem.

I'd really love to see languages like Ruby tackle this next. It's sorely needed.

---

[1] http://openjdk.java.net/projects/jigsaw/spec/sotms/

[curun1r]

> ...because more modern languages are doing this out of the gates and with far fewer hacks and more out-of-the-box tooling (e.g. Rust)

Haha...I love Rust, but you must not have been following the recent saga over modules in the community. Two things became abundantly clear:

1) There are definite issues with the current way that Rust does modules and they create problems for newcomers that don't know all the intricacies.

2) There's very little consensus in the community about the best way to fix those problems.

Initially, a couple of different, somewhat major proposals were made that would largely overhaul the system. Over the course of a few iterations, those were whittled down to a few, much smaller changes that mostly keep the current system but remove some of the stumbling blocks. It's a credit to the Rust team that they've handled it in such an open manner, but it's also creating a bit of a "design by committee" feel that's probably going to create something that everyone can live with and very few will think is close to perfect.

[Rusky]

The problems the Rust module system has and the changes being made to it are purely syntactic. Rust has already solved, and isn't changing its solution to, the kinds of problems Java is solving with Jigsaw.

FWIW, I don't agree with the "design by committee" assessment on the new system, either. There was certainly a lot of committee-flavored input, but it was only used for brainstorming and gathering requirements. The actual RFC, especially at this point, is pretty coherent and put together by only one or two people.

[kibwen]

> it's also creating a bit of a "design by committee" feel that's probably going to create something that everyone can live with and very few will think is close to perfect.

I wish that the term "design by committee" didn't have the stopping power it has. Rust's current module system absolutely wasn't designed by committee, and if there's one complaint that I'd levy against it it'd be "overengineered", which is typically what people seem to tend to expect from systems designed by committees!

Personally, whether or not something is designed by a committee is orthogonal. What matters more is whether that something is well-integrated with the sibling systems that it is integrated with (read: unsurprising), and whether its design exhibits good taste (obviously, openly, wantonly subjective, but perhaps we can usefully say that a design can be judged to have good taste if it is so judged by people who think that its sibling systems also have good taste (so read: consistent)).

In truth, there probably is a positive correlation between things that are designed by committee and things that violate the two principles above; it's hard to get lots of people to commit to a consistent vision, especially if that involves serious tradeoffs. It's part of why languages with BDFLs tend to be considered at least coherent, if not elegant: it's easy for a committee of one to have a vision consistent with itself. But where this pejorative doesn't necessarily apply is when the committee is small, close-knit, and all share the same values.

When it comes to Rust, the committee in question is the language team, which is just seven people (in contrast to the dozens of commenters on this issue, who are there to provide perspective and arguments, not to cast votes). Of these, the team lead (Niko Matsakis) is someone that I personally trust to have excellent taste; I feel the same about the RFC author, Aaron Turon, who is also on the language team. And, for better or worse, teams choose their own new members, which gives good taste the chance to propagate; this runs the risk of stagnation (lessened, hopefully, by the RFC process), but also avoids the fractured nature of committees assembled from far afield.

For the record, the module system is exactly one of those things that I've long felt is subpar about Rust (though still better than headers, obviously--that's not a debate, it's a massacre). I haven't had the chance to play with the revised modules RFC yet, but we'll have a lot of experience with it before it potentially gets stabilized, and from what I've read it does look like it ultimately improves consistency and reduces surprises when judged against the rest of the language, which makes me optimistic. I'd love to have to find something new about Rust to whine about. :)

[curun1r]

For the record, I wasn't talking about the overall module system with my "design by committee" description, just the recent proposed changes. Whether there were only 7 votes or not, it definitely felt like there was an effort to appease as many of the people who commented on the issues as possible, and that wasn't a small number. Don't get me wrong...I think the community that has developed around Rust is awesome and somewhat unique for a technical project when it comes to the respect and civility that gets exhibited. But I think the module system is an area where having a talismanic figure capable of hearing input, deciding on a design aesthetic and then pushing the entire community to buy into it would yield superior results to a make-everyone-happy approach. And it feels like the changes that will be coming to Rust are more of the latter and limited to just the things that almost everyone agreed upon.

The difficulty is that I think the Rust process works better for changes that aren't as disruptive or fundamental to the language. The RFC process is not only open and fairly welcoming to people outside the core team, it's also a great educational resource. I've learned a lot about language design from reading the RFCs and the accompanying comments. So I don't think I'd trade the Rust model for a BDFL model. I just wish something more like Aaron's original proposal (the second original proposal, that is...the one that got turned into the most recent one) was being implemented, even though I disagreed with a good chunk of it.

[TeMPOraL]

Yeah, design by committee isn't a bad thing. See e.g. Common Lisp which, being defined by an ANSI standard, shows how great things can be when you have a committee made of smart people who inform their work by carefully evaluating what other smart people before them tested in the field.

[patrec]

    (setf (readtable-case *readtable*) :invert)

[TeMPOraL]

Yes. What about it?

It's a useful feature to have, as it makes Lisp reader more flexible. See the summary of the discussion by the committee here: http://clhs.lisp.se/Issues/iss286_w.htm.

[bad_user]

> I have a hard time being too impressed because more modern languages are doing this out of the gates

The problem with Java is that it's an already established and very popular platform and it's been so for the past decade at least. When starting from scratch, it's easy to just throw it all away and start fresh, it's easy to fix mistakes that were made in the past.

The irony though is that we still have a hard time learning from history. Just look at Go. Sometimes this industry feels like Groundhog Day, the movie.

Also, no platform or language that I know of has gotten "modules" right, with Java being one of the platforms that has gotten closest to fixing the problem actually (by means of OSGi). To see why modules are still a big problem and why we won't have a fix for the foreseeable future, I invite you to watch this keynote by Rich Hickey: https://www.youtube.com/watch?v=oyLBGkS5ICk

[jcadam]

OSGi is a bottomless pit of pain and despair. Never again will I work on any project built on OSGi modules (well, every man has his price, but you know what I mean).

[bad_user]

I agree, OSGi sucks, but it's one of the few attempts at fixing the problem with compatibility breakage in dependencies.

[chrisseaton]

> I have a hard time being too impressed because more modern languages are doing this out of the gates and with far fewer hacks

Well of course - it's easier to build something like this into a clean-slate language isn't it? It's harder to build it into an existing language and VM spec with an incomprehensibly large volume of existing code to be compatible with. It's backwards to say it's not impressive because someone else with zero constraints to work with also managed it.

[clra]

> Well of course - it's easier to build something like this into a clean-slate language isn't it? It's harder to build it into an existing language and VM spec with an incomprehensibly large volume of existing code to be compatible with. It's bizarre to say it's not impressive because someone else with zero constraints to work with also managed it.

I hope my original comment was relatively clear on this, but sure, it's a big accomplishment and will be a quantum leap for the ecosystem. However, given that languages elsewhere have had better systems for quite some time now, it's not like they're advancing the state of the art in dependency management.

But again, it wasn't an easy thing to do (a very hard thing even) and everyone involved deserves major felicitation.

[smallnamespace]

I guess the real question is, why do you find advancing the state of the art to be more impressive or deserving of acclaim than integrating improvements to existing systems?

Both activities seem valuable and complementary to me -- there's no point in advancing the frontier if nobody will bother to make those advances practically useful.

[watwut]

They are advancing Java itself. Those who work with Java on large projects (which is the Java use case) will find modules useful. There is huge community around Java and even more people who use it, but don't count itself members of any community.

[vanderZwan]

I think the distinction is GP makes is being impressed by the concept of a module system itself, or by being able to bolt on such a system to an existing language.

A module system in and of itself is not impressive these days. Getting one into Java is.

[coding123]

Can someone with an idea about this post other languages that actually attempt to modulerize it's own API? I'm not aware of any.

[_asummers]

Elixir is an application that runs on the Erlang VM. Libraries can be brought in and started as their own applications, with their own lifecycles. I'm not privy enough in the specifics of the Rust way of things, but it sounds like this is sort of similar if you squint a little bit?

Observer screenshot from a blank IEx session: https://puu.sh/xI3re/3701a0c64e.png

[pron]

> I have a hard time being too impressed because more modern languages are doing this out of the gates and with far fewer hacks and more out-of-the-box tooling

I'm not so sure about that. Java (both the language and the VM) is relatively unique in its mix of static (mostly in the sense of being typed) and dynamic (all linking is dynamic; code is often be loaded, unloaded and generated at runtime; reflection and general introspection of the runtime is pervasive). It's a blend of, say, ML, Smalltalk and Erlang. The module system is therefore more impressive than it may look at first glance. While not all the tooling is there yet, through the use of what it calls "layers", it makes it possible to dynamically load plugins or even upgrades to components in the system, each depending on different versions of the same library, while enforcing customizable levels of isolation between them. Whether or not there are any conflicts requiring a separation of "layers", can also be determined at runtime by introspection.

[catnaroek]

Other than (its botched version of) parametric polymorphism (due to how easily it can be circumvented, defeating the entire point to it), what ML-influenced features does Java actually have? There is no global type reconstruction, there are no abstract types, and “computation as expression evaluation” is a royal pain in the rear hole in Java.

[pron]

Oh, I simply meant any safe, typed language, and that was the first that came to mind.

Having said that, Java is a typed, safe OO language, and that alone, I contend, already makes it close in spirit to ML (in particular, its structures/signatures). I think interfaces (and the ability to override methods like `equals` are pretty close to abstract types. I also strongly disagree with your description of Java's parametric polymorphism being "botched", and certainly not that its "entire point" is defeated. That it can be circumvented is what makes code and data sharing between different languages with different variance models possible (compare to how badly that's done in .NET). So it's simply a matter of what requirements are more important to you.

I can understand those that think that Java not being fully static or fully dynamic may defeat the whole purpose of what they like in their preferred approach. But I think that Java combines static and dynamic aspects in a rather unique and novel way, and the new module system is not different, combining in an interesting way both static and dynamic aspects.

[catnaroek]

> Having said that, Java is a typed, safe OO language, and that alone, I contend, already makes it close in spirit to ML (in particular, its structures/signatures).

The whole point to structures and signatures is abstract types. Mere value hiding can be achieved with let, which isn't exactly the pinnacle of typing: Scheme has it.

> I think interfaces (and the ability to override methods like `equals` are pretty close to abstract types.

Please do tell how you would make two or more instances of an abstract type in Java, in such a way that:

(0) The client is aware that that the two instances have a common type.

(1) The client is not aware of the representation.

If it's not possible, then Java doesn't have abstract types, period.

> That it can be circumvented is what makes code and data sharing between different languages with different variance models possible

And unsafe. (No, memory safety alone isn't safety.) If it's going to be unsafe, then I better at least get my money's worth in terms of performance, which is why low-level languages like C and Rust are the only ones worth FFI'ing to.

> I can understand those that think that Java not being fully static or fully dynamic may defeat the whole purpose of what they like in their preferred approach.

Java is fully dynamic. “Type checking” in Java is basically a mandatory linter.

[pron]

> Please do tell how you would make two or more instances of an abstract type in Java, in such a way that...

By having two implementations of a common interface, like `ArrayList` and `LinkedList` both implementing `List`.

> And unsafe.

It seems like you're defining "safe" to be precisely what the languages you like provide, no more and no less. I can say that ML and Haskell are unsafe because they don't statically forbid erroneous behavior at runtime, like a sorting function that doesn't sort (or doesn't terminate). Java has no undefined behavior. In fact, it is completely unknown how much safer -- if at all -- is ML than Java in practice.

> If it's going to be unsafe, then I better at least get my money's worth in terms of performance, which is why low-level languages like C and Rust are the only ones worth FFI'ing to.

Of course it's a matter of specific requirements, but I think you get more than your money's worth in terms of performance in Java, and having decades of experience writing huge multi-MLOC programs in both Java and C++, I'm convinced that it takes significantly less effort to get a well-performing large Java app -- especially if it's concurrent -- than a C++ app, even though you could surpass Java's performance given considerable additional effort. In any event, I think that the success of the JVM shows that many people find supporting it to be worth it.

> Java is fully dynamic. “Type checking” in Java is basically a mandatory linter.

This is simply untrue. Java is mostly type safe. If you have a variable of type `Foo` in your program, it cannot reference an object of a type that is not `Foo` at runtime. You are right that this does not extend to generic types, but only if -- 1. you've intentionally tried to circumvent the type, or 2. you've fallen victim to an obscure bug that was found recently, and is very hard to reproduce accidentally.

[catnaroek]

> By having two implementations of a common interface, like `ArrayList` and `LinkedList` both implementing `List`.

That's not an abstract type, it's an object type. An abstract type has a single representation, determined by the type's implementor, which is hidden from the rest of the program. OTOH, objects with the same type may have different internal representations, determined by whoever constructs the object, just as in your example.

> It seems like you're defining "safe" to be precisely what the languages you like provide, no more and no less.

Safety is defined in terms of the language's semantics, not my personal preferences: are meaningless operations ruled out or not? (We can get technical and say that Java does, in fact, assing a meaning to invalid casts: to throw ClassCastException. But very few people would consider that a useful meaning: if you run into it, your program plainly has a bug.)

> Java is mostly type safe.

Then it isn't.

[ratmice]

>> Please do tell how you would make two or more instances of an abstract type in Java, in such a way that...

>By having two implementations of a common interface, like `ArrayList` and `LinkedList` both implementing `List`.

This is not i believe what he is talking about, here is a silly example in SML, it has 3 types which all share a type t... a list of t pairs, a vector of t's, and a function from pair of t's to t.

functor Foo (type t; val pairs : (t * t) list val f : t * t -> t) = struct val things : t vector = Vector.fromList (List.map f pairs); end

One thing to note is that t is never exported/returned. and thus, the thing returned exports t vector but not t.

we could export t in a few different ways:

type t = t; Export it, let its binding be known type t; Its a type, but what it is bound to is not known.

[geodel]

With the raging debate going on in Rust community about module system, it is clear they are still far from having well understood and usable module system for wider Rust community.

[catnaroek]

The meaning of “module” is pretty concrete and specific. A module is:

(0) A unit of encapsulation: Modules can export abstract types whose representation is hidden from the rest of the program.

(1) A unit of type-checking: Modules can be type-checked without knowing anything other than the interface of their dependencies.

(2) A unit of translation: Modules can be translated to target machine code separately from each other. (That being said, whole-program compilation is fine if you want it. The semantics of the module system shouldn't make it mandatory, though.)

Most programming languages don't have module systems at all. And most programming languages that do have module systems, have module systems that suck. Including Rust.

[bitmapbrother]

>I have a hard time being too impressed because more modern languages are doing this out of the gates and with far fewer hacks and more out-of-the-box tooling

I'm curious, is there another language that allows you to streamline its runtime by choosing only what modules your app needs?

[pjmlp]

This was already used in Lisp and Smalltalk to trim down images for production delivery.

Other than that, the majority of compiled languages do remove unused library code when static linking.

[ksec]

>I'd really love to see languages like Ruby tackle this next. It's sorely needed.

Could you expand on that? Why Ruby?

[riffraff]

not OP, but ruby's current "modules" are just namespaces.

What jigsaw and rust call modules are isolated components that you can plug and connect in various configurations, and that declare public interfaces both as dependencies and as exported symbols. Consider a module configuration like

  module java.sql {
    requires public java.logging;
    requires public java.xml;
    exports java.sql;
    exports javax.sql;
    exports javax.transaction.xa;
  }

So, they are actually slightly closer to rubygems + bundler, but ruby is always "promiscuous", i.e. code from one library can trivially mess up with code from other gems, while some isolation would be nice.

Also, by default ruby does not namespace things by package/module/gem so it's easy to have collisions, while in a more robust system this would/should be under the control of the calling code.