I didn't mean that the language itself isn't functional - it certainly wants to be. But when I tried to naively apply it using the idiomatic constructs that I found online I got a lot of performance and memory issues. In fact some code that I wrote in Groovy (which I thought would be slow) was much faster than the naive port I did to Scala (which is commonly said to hit nearly native java speed). When I dug into those by profiling it turned out that I needed to have a deep understanding of how the compiler was treating Scala constructs to avoid performance pitfalls. A good example is here:
Another is that I've almost never managed to use recursion in my algorithms because Scala seems to have very limited ability to successfully optimize tail recursive calls.
Another problem is all kinds of unexpected boxing, unboxing, and implicit conversions of collections that I wasn't expecting.
Again - all the language features are there, just in practice it isn't working out for me very well when I try to use them idiomatically. I'm still learning. But I also learned Haskell and the experience was very different - once I figured out the idiomatic way to do something it usually was also well optimized.
ML sits somewhere in between Scala and Haskell. Like Haskell, ML has typed mutable data (`foo` and `foo ref` are different types). Like Scala, ML doesn't distinguish between effectful and effect-free procedures.
Scala is similar to Lisp and other higher-order-but-not-quite-functional languages in that it's littered with unwanted object identities. All you need to do is use the `eq` method to see when two “equal” objects are really not the same.
This sounds pretty misinformed. There are plenty of choices in between Scala and Haskell; Clojure and Elixir are two other relatively popular languages that come to mind.
You might have fallen victim to misinformation yourself.
In general, functional programming in Scala tends to be more FP, with code tending to be more pure than in Clojure and I have no experience with Elixir, but I have some experience with Erlang and FP code in Scala tends to be held at a higher standard than in Erlang.
Of course, you've picked 2 dynamic languages as examples and FP in dynamic languages is different than that practiced in static languages like Haskell or Scala. LISP developers for example don't think so much about monads or other abstractions with mathematical foundations, because LISP developers tend to work around such needs by doing macros (which then have composability problems) or by bending the rules a little, or in other words I've seen no LISP to make a serious attempt at reasoning about I/O in a pure way.
And IMO code in static languages tends to be more pure because of the types, because by having an expressive type system, the developers then want that type system to explain everything. Or in other words, dynamic languages are cool for your day job, but if you want to actually feel what FP is all about, you're better off going for a static languages like Haskell, or even Scala or OCaml.
Thanks for your reply, I think this is a matter of interpretation of what functional actually means. I personally consider the fact that Scala allows you to get away with immutability so easily (as evidenced with the implementation of all the immutable collections) a very bad thing. It might be just a matter of taste, though; Scala was my gateway drug to functional programming, and I considered implicit parameters and all these immutable containers something very un-functional.
You cannot get away with these things as easily in the Erlang VM (and thus Elixir). I agree with your assessment that Clojure with its macros is an ugly hack, and requires discipline to get right.
Having said that, Haskell also takes a lot of discipline to get right (no lazy I/O, for example), and allows you to get away with ugly things as well (unsafePerformIO). The type system makes it a lot easier to get right though (or in other words, more difficult to do the wrong thing).
https://issues.scala-lang.org/browse/SI-1338
Another is that I've almost never managed to use recursion in my algorithms because Scala seems to have very limited ability to successfully optimize tail recursive calls.
Another problem is all kinds of unexpected boxing, unboxing, and implicit conversions of collections that I wasn't expecting.
Again - all the language features are there, just in practice it isn't working out for me very well when I try to use them idiomatically. I'm still learning. But I also learned Haskell and the experience was very different - once I figured out the idiomatic way to do something it usually was also well optimized.