[seahawks78]

I very recently (about a year ago) joined a shop that is primarily into functional programming (FP) and I got my first exposure to Scala as well as F# through work projects. As a long time user of Java/OOP here are my thoughts on Scala and why it has so many things things going against it. Consider the following:

1) First, Scala is designed and developed by "ivory" tower academics primarily for research/PhD thesis purposes. Nothing wrong with that. But that also means language features are added not to solve a practical problem but mostly to show that "it can be done"/"prove a point". When you are awake at 3 AM early in the morning because a release made in the evening went awry and you are trying to grep though logs/code to see what caused the issue I can assure you that debates about "pure" versus "impure", Monads, Kleisli arrow will be further from your mind. So the problem that Scala seems to suffer from is that of "image".

2) Second, Scala is neither a fully functional (FP) nor object oriented (OOP). Support for mixed/multi-paradigm sounds nice on paper but let me ask this question: do you know of any hard core FP programmer who is excited about inheritance or abstract classes or other OOP features? Vice-versa the same thing can be said about many OOP programmers lukewarm response towards many functional features of Scala. I believe that support for mixed/multiple paradigms has hindered large scale adoption of Scala. The language seems to be caught at cross-roads not knowing which way to advance next.

3) Third is lack of niche. True, I absolutely love "Spark" and primarily use it to extract/manipulate data from HDFS/Hive storage but I am gradually gravitating towards "Python" libraries due to their ease of use. What about better Java? Kotlin check. What about sidecar systems used by DevOps? Go check. What about a better designed system language? Rust check. The days of pure general purpose programming languages are truly over and Scala will be in great peril if it fails to expand into newer niches. Without broadening its programmer base it will finally run out of people that can design newer language features in light of ever advancing progresses in processor, OS and memory architectures.

Just my two cents.

[dropofwill]

I work at a Java shop, we do ‘modern’ Java (11 or 14, some legacy 8), but I do a lot fp-style Scala dev at home.

Calling Scala an ivory tower with no interest in ‘practical’ programming is ridiculous. Scala and Java are remarkably similar in this respect (note Odersky was heavily involved with Java before Scala), albeit one is sponsored by a uni and the other a corporation. JEPs are just as likely to have actual PhD theses associated with them, and those that don’t often could have. Look at Project Valhalla or Loom or the GraalVM, these are novel programming language research projects with the end goal of bringing them to a widely used industry language in a backwards compatible and ergonomic way.

As for pure FP in production, just last week we had a brief bit of message loss in a system because some layer in an sdk swallowed a socket timeout exception and re-threw something totally unrelated and undocumented. This was hell to debug for other reasons related to imperative code, that simply wouldn’t have happened in a ZIO project.

That’s not to say you can’t make reliable software in Java, my team spends a lot of time doing just that. It just often feels like the whole ecosystem is fighting against you. Things are getting much better in Java land, but there are parts that can never catch up with Scala because of past design choices.

I agree with your other 2 points, multi-paradigm is both a blessing and a curse and probably becomes more of the latter in a large org.

Also, unrelated, but imo better Java is Java or maybe Java in 3 years, Kotlin didn’t go far enough.

[pjmlp]

Kotlin hit jackpot with Android, but given Google's reluctance to move Android Java beyond its Java 8 subset, JetBrains will eventually need to consider having two Java targets as part of their multiplatform Kotlin, or just focus being an Android language.

[z6]

Well put. I had to use Scala for a little and I wasn't a fan. It's maybe a fun language to play around with for a pet project, but for a reliable production system I would never pick it.

The language is too flexible with too many ways to do the same thing, increasing cognitive load and making things difficult to understand. Maybe you can limit that in your own projects, but what if you have to look at a library codebase that's structured completely differently. Trying to understand that can take a lot of time.

Also there is the problem with hiring. Maybe your current team can work well with Scala, but what if a key member leaves or a service gets transferred. Hiring for Scala turned out to be impossible, so you get all these people that need to learn Scala first. Normally that may be ok, but Scala has a pretty steep learning curve so it takes a lot longer to onboard people.

[mchisto]

On hiring: Python Paradox comes to mind (http://www.paulgraham.com/pypar.html). I've never met sub-par FPer or Scala programmer specifically. Using PHP or Javascript might give you a bigger top of the hiring funnel, but it doesn't mean you'll have easier time finding well-qualified applicant.

[Joeri]

I’m surprised you had such a bad experience with scala. Or do you think these are problems with how scala is perceived versus how it actually is to use?

Personally I liked it. I spent a while doing java web dev with the play framework, and also spent a while doing scala web dev with the play framework, which allowed me to compare directly. I have to say scala is a far nicer OOP language than java. The collection classes are better designed, the way futures and options work is so much more elegant than the java counterparts, and the overall syntax is much more efficient. IMHO if you just use scala as a nicer java, it absolutely trounces it. I couldn’t really say what OOP feature it lacks compared to java. I never tried kotlin, so I cannot tell how that compares.

I also spent a while doing FP style programming with scala in spark and akka. I cannot compare it against other FP languages because I never used them but I can say it was similarly very enjoyable and it was easy to write elegant and understandable code.

I would agree mixing both approaches in a single codebase is a path fraught with peril. If scala has a flaw then it is having too many features. Like C++ you need to be careful which features you use in which combination, or you can end up writing an intractable codebase.

[mtrovo]

I had the same experience as you on the beginning. I started with a project in play framework, I was really excited about an improved java, etc. Then some FP hardcore developers came and started to influence younger developers that OOP was not cool anymore, all our system should be composed of Actions and monads of all kinds, and all business logic should be just a big composition of map and flatMaps.

After a while the project became just a dick measuring context of who could do the best trick shots. Declaring a lot of implicit parameters, implicit conversions automatically applied, crazy type inference that makes compiling 10x slower, you name it. Just to make the code unreadable enough that other developers need days to understand a three lines of code change.

And after all of that the more they do it, the more they are seen as the best Scala developers on the team.

[JackFr]

That seems more a symptom of an organizational dysfunction than an indictment of the language.

To wit, 15 years ago I was on a team where it was “How J2EE can you get?” Ughh. Along the way they(we) basically forgot to solve the business problems.

And that sion was the end of that.

[jane_red]

Based on your preferences you definitely should try Kotlin :)

[KingOfCoders]

Thanks to Scala we've built, scaled and sold a startup (~150kloc, Lift + Play, SBT). Thank you.

That said, "First, Scala is designed and developed by "ivory" tower academics primarily for research/PhD thesis purposes." I totally agree.

Error messages when using libraries are horrendous with dozens of lines of type errors (take a look at Elm how it's done right).

Compile times are horrendous. There are whole consulting companies [1] which do nothing than help companies with compile time of their Scala projects.

The article talks about 2x speedups over the years. Which sounds nice, but if you're 10x slower than others (Go, Rust, ...), then with a 2x speed up you're still 5x slower. Which is fine far small projects but kills you with large ones.

Would come back if the compiler is rewritten in Go (which will not happen because Scala is primarily a research project).

I now use TypeScript with Wallaby.js and Quokka developing test first. It's like night and day. I do miss Scala, because I really love the language, but I would not go to the pain of compiling and executing unit tests again in Scala.

[1] https://triplequote.com/ "Compile Scala Faster"

[MetalGuru]

Did you have trouble hiring Scala devs? And if so, do you think the language benefits outweighed that. This is my main concern choosing to build new ventures with Scala.

[KingOfCoders]

1.) Yes 1a.) Scala was a bad choice, but it was chosen because I love the language.

2.) When founders ask me about what language to choose, I tell them: "Choose one where you easily find developers, then you can't go wrong".

[wpietri]

Totally. I think the right way to pick a language for a startup is to find some smart, experienced devs to hire and let them pick (modulo reasonable business constraints). The popular languages all can do a wide range of things effectively. The more important factor for productivity is how well the team knows and likes the tools.

As an example, we know that PHP is a "fractal of bad design"[1], but I can't deny that Facebook as a business is doing just fine.

[1] https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

[dominotw]

Agreed. I switched from scala to python and oh man what a relief.

Scala is a heavy weight languange. By heavyweight I mean,

1. Even intellij struggles to figure out the meaning of implicits in the given context. And then there are macros.

2. Way too many theoretical concepts, monads, monoids , isomorphisms ect that make you feel stupid. There is always a nagging feeling that you don't 'get it' when you program in scala. Here is an example of scala conference in 2020:

https://scaladays.org/2020/berlin/schedule

Features talks like these.

> a. Composition with Functional Effects

> b. Scala 3 and Abstract Algebra

> c. Describable, typesafe computations for the less monadically inclined

> d. Understanding Scala's Type System

> e. Roll your own Effect System

3. Every code base is written in its own flavor of scala.

I was spending absurd amount of time "learning scala", now i look back at that and cringe.

[spir]

Scala 3 will alleviate some of the pain you felt, for example around implicits. Scala's IDE tooling is getting better with faster compilation and language servers.

Scala is a fusion of object-oriented and functional programming. There are at least three major styles of Scala programs: OOP, functional, and reactive (actors). Scala indeed places the burden on the development team of choosing which patterns are right for them.

The theoretical concepts that you disliked are "available" in other languages too, you just don't hear about them much because functional programmers prefer languages with strong type systems. That's not to excuse the learning curve they create! I still don't understand them too well. Some of the people who do understand them will unnecessarily push them onto their projects or teams. Scala doesn't force these on you, you can ignore them. For example I write a game engine in Scala that is mostly imperative-style and has limited use of the theoretical constructs you mention.

Python's lack of a strong type system is a serious deficiency for large projects. Same with JavaScript. The costs it creates are large and often undetectable because they are the zeitgeist and death by a thousand cuts.

[azinman2]

Ahh yes, the old “the next version will solve everything!” and the “you must be doing it wrong.”

Or... have one way of doing everything (python, go, Rust, even Java) and simplify. That simplicity means 3rd party libraries will all largely have the same basic constructs for how things work, which makes it far easier to learn, which makes it easier to contribute.

[dropofwill]

None of those languages have one way to do everything, I would even wager none have only one way to do anything.

Simplicity comes at a cost, compare Elm to Purescript/Haskell for example. Elm lacks type classes, so it’s simpler and more explicit, but that comes at the cost of verbosity and duplication. This was an explicit decision to make the language more friendly for beginners, and that’s a reasonable trade off to make, but it’s not universally better.

[dominotw]

> Scala doesn't force these on you

Yea but one cannot write everything from scratch. I had to work on project that use cats, scalz and stuff. I felt like i was splitting an atom when i was actually writing a crud app :) .

Obsession with those theoretical concepts with dubious and unproven benefits is deeply interwoven in community. Cost of learning cats vs benefit is just absurd.

I would only recommend scala to programming language nerds that want to sound smart.

[tasuki]

> I switched from scala to python and oh man what a relief.

I did a bit of Python, then did a bit of Scala, and now do both at work. Oh what a pain Python is! And what a joy Scala is!

> Way too many theoretical concepts, monads, monoids , isomorphisms ect that make you feel stupid.

Concepts are theoretical in their nature, aren't they? Perhaps these concepts are useful and you just haven't yet found out why?

Wrt feeling stupid, there are two possible reactions: "gee, I'm not so smart" and "whee a learning opportunity" - I'll prefer the latter anytime.

> There is always a nagging feeling that you don't 'get it' when you program in scala

Yes I have that too - so much more to learn!

[dominotw]

> Perhaps these concepts are useful and you just haven't yet found out why?

Or maybe they are not that useful? If they were really that useful we would see those concepts everywhere.

[C4stor]

You do see those concepts everywhere.

monads : stuff which kind of encapsulate other things, but you can still nicely manipulate both the thing and the stuff. Let's say, lists.

monoids : set of things where you kind of can add one to another. Let's say, numbers

isomorphisms : functions where there is exactly as many outputs possible than there are possible inputs. Let's say, adding 1 to a number. It's kind of useful knowing whether or not a function is an isomorphism (hello, hash collisions !).

Those concepts aren't specific to any programming language in fact, and are in fact everywhere. You just don't need to explicit them to use them, expliciting them is an option (even in Scala) which allows you to do meta manipulations on them (which Scala doesn't require, and doesn't even really suggest since none are part of the standard lib).

[dominotw]

oh ok. Any thoughts on why scala conferences have this "Describable, typesafe computations for the less monadically inclined" but python conferences don't ?

[sideeffffect]

Because doing something like that without a type system (which Python doesn't have) to guide your hand is practically impossible.

I would expect Clojure programmers (while still functional) don't use these abstractions either.

On the other hand, typed languages, like Haskell, Scala, OCaml, F#, do employ these abstract concepts. Their type system makes these worthwhile endeavour with great payoff.

[TickleMeEno]

I very much agree with your point on academics pursuing programming language theory for the sake of theory, and not for language design. I don't want to disparage FP research as a whole, but it seems to be the area of language design that cares the least about practicality and I don't really know why.

Being an undergrad doing my dissertation into programming language design, it seems that FP evolved from an area of research that was very much tied to the theory of computation and never really danced with the practical side of software engineering. Of course there are a few languages that have tried to marry the two (OCaml, Rust to an extent) it seems to very much a mathematicians game. Of course I love the math but it gets annoying to explain that no programmer should have to have a working understanding of category theory to use a language effectively.

[hocuspocus]

1) Without concrete examples this is just an unsubstantiated rant. As a trivial counterpoint, just have a look at the most advanced and opinionated FP libraries out there. Most of their contributors are not in academia.

2) Plenty of FP developers enjoy having access to the JVM and its huge ecosystem. I can trivially use and wrap most Java libraries, even the ones that overuse outdated OO patterns.

3) There are still enough companies and teams out there that don't just want a better Java. We'd see a decline in job openings if they were massively going back to Spring with Java or Kotlin, but that's not the case

[yeshengm]

Nowadays most Spark users use either PySpark or Spark SQL. Spark is designed in some sense to mimic Scala collection library. Rust is nice but its interop with other open source big data libs (Hadoop/Zookeeper/HBase) can be painful.