As far as I could tell, the big pain point was performance -- they were just too slow on median machines during the late 90s/early 00s window they had mindshare.
I actually enjoyed using them to get around a few browser limitations around the mid-to-late 00s, and they seemed feature/performance competitive with Flash unless what you were doing fit the media authoring model closely. But by then people were skeptical about Java and if you had to do anything to get it installed they wouldn't, and the direction was native web.
They were better than Flash (which needed a weirder runtime, and couldn’t interact easily with the DOM, and didn’t have variables until version 3). This is hugely improved over JavaScript at the time (pre-XHR) and had less lock-in than ActiveX.
I spent years working with GWT, it's not nearly as simple as you describe. It's not running "Java" in the browser, it's compiling down (a subset of) Java to Javascript.
That said, GWT was ahead of its time in a lot of ways, but it had warts galore.
I actually enjoyed using them to get around a few browser limitations around the mid-to-late 00s, and they seemed feature/performance competitive with Flash unless what you were doing fit the media authoring model closely. But by then people were skeptical about Java and if you had to do anything to get it installed they wouldn't, and the direction was native web.