[johnnycerberus]

You're lucky that you can do that kind of work. I've been working for a state company in Bulgaria and we had to maintain some Scala pipelines built with Apache Spark some years ago. It wasn't a pleasant experience, just a sea of vars/vals and cases of reinventing the wheel only to use some functional constructs when it could be simply done in Java by just adding a dependency that has already solved that problem like 10 years ago. It wasn't even some complicated logic in that, just calling different APIs and playing with dataframes, but the fact that they hid from me so much behavior only to reduce verbosity gave me some stressful days.

[non-entity]

Any partitular reason scala was chosen? Spark has Python and Java API's as well.

[rad_gruchalski]

Why not use that java dependency in scala?

[eitland]

Culture it seems.

Functional style is cool and some people will go to great lengths to use it.

Like shaving a few characters of a perfectly fine piece of code by using currying and instead make one single long line of code describing it.

[pjmlp]

Guest language syndrome. All libraries must be idiomatic instead of reusing the platform language based libraries.

[johnnycerberus]

Exactly that.