[smabie]

I've always loved Scala and Scala 3 is shaping up to be extremely exciting. Love the new ADTs and braceless syntax: really gives the language a cleaner and more ML inspired feel.

No other conventional language (including OCaml, F#, etc) seem to have the effortless kind of power Scala does.

The combination of the extremely powerful type system with the "OOFP" paradigm is a great combination. OOFP is such a great paradigm and it's unfortunate that no other language has embraced it.

It's ironic that I love Scala so much because I don't particularly enjoy dealing with the JVM and all its complexities and problem. But Scala is so good that it mostly makes up for it.

Scala does get a lot of hate (especially around here), but I've never really understood why. There's only one language I like more (kdb+/q), but that's a hard sell, especially because of the cost.

Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag. However, I do understand why non-CS/type theory people might find the language intimidating.

But to that I'd say, I've seen new junior (though very smart) developers with no Scala or FP experience get onboarded and write idiomatic Scala code within a couple weeks to a month. The short term cost is more than worth it considering the absurd productivity compared to Java/Kotlin/Python/blublang.

Compared to OCaml/F#/Haskell, the improvement is a little less stark, though still clear imo (especially when considering editor tooling and available packages).

[tastyminerals2]

Scala is weird. Sometimes it feels fluid and effortless but sometimes it simply cannot do things you would expect it to. For example, chaining custom methods. Why is it possible to do "arr.map(_.nonEmpty).filter(a => a % 2)" but not possible "arr.map(_.nonEmpty).myCustomFilter(_)"? In D thanks to UFCS and Properties you can chain all kinds of std and custom stuff together which looks more "efforless" and FP in my eyes.

Syntax aside I find Scala code hard to read due to how much implicitness it allows which stems from its "do FP or die" attitude which is again, weird for a language designed to incorporate both OOP and FP. Should I mention that in this regard D does a better job at serving the both worlds?

And finally, which threw me off completely is a real world case where we had to implement a word-entropy based algorithm to process tons of textual data and it appeared to be not much faster than a Python version despite all the optimization we tried to do. Scala is fast but there are many faster languages including Java itself. Considering the fact that its compiling times force you to leave your seat for a cup of coffee, you'd rather think of investing into another hard FP friendly language like Rust which simply has a wider applicability area. But yeah, this is all speculative but if you are thinking about what next language to pick up, these things start to matter.

Where Scala fits nicely is a small team which needs to build some business logic on top of the existing Java backend stack quickly and have a reliable working system. This is where it really shines. Scala provides a nice layer for JVM devs who have a soft spot for FP, safe and elegant solutions.

[valenterry]

There are no "special" methods in Scala. If you see something like "arr.map(_.nonEmpty).filter(a => a % 2)" in the standard library, you can use that syntax yourself too.

E.g. "arr.customlyFiltered()" is easy to do. Or even easier "arr.filter(myCustomFilter)". If something doesn't work out for you, please feel free to use e.g. scalafiddle.io and make it example, then you will be helped! :)

Scala is indeed not a high performant language. It is as fast as Java and compiles as fast as Java if you only use Java's features. However, then why use Scala in the beginning?

Scala is great for writing very maintainable and reusable code, especially with a big team and a lot of business logic. It is also great to "glue" things together like you can do it with Python. For these things it has more than sufficient performance in my opinion.

[tastyminerals2]

There is no straighforward elegant way of doing "obj.scalaMethod.myCustomMethod.scalaMethod.myCustomMethod" in Scala. I want to create a method and just inject it into the chain, no fiddling or ducktaping or using some black Scala magic.

"def myFun(a: String, b: Double): Boolean = {...}" and then "obj.mapValues.myFun.forall(_)" or just anything similar.

And yes, Scala is exactly the language when you need safe and maintainable code with far less unit tests required than in Java. But you have to invest into it and sometimes it is simply not worth it. The Scala2 to Scala3 migration situation makes things even worse unfortunately.

[vvillena]

There is a feature in Scala 2 that covers the specific thing you want: implicit classes. You can use them to add custom methods. It's clean and straightforward.

And it will be even easier in Scala 3, where this pattern is coded into an even simpler feature: extensions.

[esarbe]

In the Scala 2.x this can be done with an implicit class.

It's a bit cumbersome and not very use-case oriented (mechanism over intention) and so Scala 3 has introduced extension methods[0].

This gives you exactly what you want.

Since Scala 3 is pretty much completely backwards compatible (and with the use of Tasty you can use Scala 3 code in Scala 2.x projects), so I don't see the migration as a big problem. The Scala team has spent a lot of work to make the transition as painless as possible.

[0] https://dotty.epfl.ch/docs/reference/contextual/extension-me...

[ph2082]

Thank you for URL. Just when I needed. Any idea when "Scala 3" can be production ready ?

[esarbe]

From what I saw in the latest presentation by Martin, we can expect Scala 3 to be released in late Spring/early Summer 2021.

Part of the release testing is building the community release[0], which is a large part of the relevant Scala F/OSS code out there.

Whether you consider that to be production ready you have to decide for yourself, I myself am pretty happy about it.

Keep in mind that for now Scala 2 style code will still be compiled and you can use existing Scala 2 code in your Scala 3 project. The migration should be pretty smooth.

[0] https://github.com/scala/community-build

[shim__]

It's not really black magic https://docs.scala-lang.org/overviews/core/implicit-classes.... but I don't really see why having a filter as a method should even be encouraged.

[tastyminerals2]

Implicits are not encouraged in our code base and for a good reason.

This was just an example. It's not about filter at all but the general method chaining.

Here is example from D if I must:

    real[] fun(int[] arr) {
        return arr.map!(a => a.to!double / PI).array;   
    }

    void main()
    {
        int[] arr = 100.iota.array; // [0, 1, 2, 3, ...]
        real[] newArr = arr.map!(a=> a*2).array.fun); // [0, 0.63662, 1.27324, ...]
    }

I don't know how you can chain a custom method "fun" to the output of the "map" in Scala without duck typing. Why is this not possible when all conditions type-wise are met? Why you can chain std methods like map, filter, reduce, fold etc. but not custom ones?

[valenterry]

Ah, I understand what you mean now. You are looking indeed for the thrush operator. I think it should be built into Scala's standard library, but until that happens, you can use the mouse library or build it yourself. Here is an example:

    // Need to define this once somewhere in your project
    implicit class TrushExtension[A](anything: A) {
      def |>[B](function: A => B) = function(anything)
    }
    
    
    // Your application code
    def fun(arr: Iterable[Int]) = arr.map(_.toDouble / Math.PI)
    
    val arr = 0 to 100
    val newArr = arr.map(_*2) |> fun
    
    newArr.foreach(println)  // prints 0, 0.63662, 1.27324, ...

Execute or change the code here: https://scalafiddle.io/sf/WAKhZtJ/0

This is maybe not exactly as convenient, but it comes pretty close.

"someA.someB.someC" becomes "someA |> someB |> someC".

[vvillena]

Scala is more strict than D for this use case. In Scala, if you want a function to be available to a certain type as if it was a method call, you need to be explicit about it. You have to declare an "implicit class" that takes the base type as an argument, and define the function as a method of the implicit class. You also need to ensure the implicit class is in scope. Once these conditions are met, you can use it as a method.

    val t1 = MyType()
    def fun(t: MyType, argument: Int) = argument

    // can't do this yet
    // t1.fun(42)
    
    // In Scala 2 you use an implicit class to add methods to a type
    implicit class MyEnrichedType(t: MyType) {
      def fun(argument: Int) = fun(t, argument)
    }

    // In Scala 3 you use an extension
    extension (t: MyType)
      def fun(argument: Int) = fun(t, argument)
    
    // now it can be done
    t.fun(42)

Regarding your question about the stdlib methods being able to be chained: they are not special. They are defined for the type the methods return, so they can be used.

Rejecting all kinds of implicits and then complaining about Scala missing features is a bit unfair. "Implicit"is a single keyword, but not a single feature. Implicit arguments, implicit conversions and implicit classes are not the same thing. Fortunately, Scala 3 will clear this misunderstanding.

[valenterry]

Assuming that your object is a list of tuples, then you can do:

    val myFun = (a: String, b: Int) => b % 2 == 0
    List("a" -> 1, "b" -> 2, "c" -> 3).map(myFun.tupled).forall(identity) // false
    List("a" -> 2, "b" -> 4, "c" -> 6).map(myFun.tupled).forall(identity) // true

But I think that's not what you mean... are you maybe looking for the "thrush" / |> operator? Or do you have some example from another language that does a better job and show how it looks there?

[fluffything]

> you'd rather think of investing into another hard FP friendly language like Rust which simply has a wider applicability area.

FYI Aaron Turon did their PhD on Concurrency+Scala, and ended up leading the Rust core team for more than enough years.

[hocuspocus]

> Rust which simply has a wider applicability area

Rust is a great language but that's a pretty insane statement. The ecosystem is still tiny compared to the JVM.

[pkolaczk]

Rust is a great language, and Scala is a great language, but they are different. They optimize for a different thing. Rust definitely rocks at high-performance, resource management, fine grained control over all the aspects of the program, at the expense of developer's time. Scala rocks at developer productivity and building abstractions, sacrificing a bit of performance.

However, I must say that Rust is also a very productive and quite powerful language (albeit IMHO not as powerful as Scala) and Scala is actually not that bad at performance either - probably better than a vast majority of other languages out there (albeit not as powerful as Rust or C++).

As far as ecosystems are considered - this is really hard to say. Rust has the whole C (and most of C++) ecosystem at hand plus a few really amazing Rust solutions like Cargo. Scala is limited mostly to JVM and JS, which are great ecosystems, but it is not true they offer everything. And there are still some caveats when using Java tools with Scala (e.g. profilers, debuggers or build systems).

[hocuspocus]

Of course, general purpose languages have their limits and I'm not arguing that Scala can do everything. But it certainly can do a lot more than Rust (and its C/C++ interop) at the moment.

And I have no problem with the JVM interop. As long as you don't try to write Scala libraries that need to be consumed by Java applications, everything just works really.

[pkolaczk]

A lot more than Rust? Do you count the whole embedded, low latency and scientific computing? As far as I like Scala, I don't think it is better in these areas than Rust. Even the async I/O libraries are more advanced in Rust than in Java.

[lmm]

I'd say Scala's support for scientific computing is significantly ahead of Rust; there are good libraries for ML, linear algebra and what have you (including relatively good bindings for native LAPACK), and there's the whole Spark ecosystem.

Scala on a well-tuned JVM can reach much lower latency than you might think. Yes, in theory you'll hit a limit that Rust could help you go beyond. In practice, are you ever going to reach that limit?

Rust's embedded support is mostly theoretical at this point, and the use cases for old-school embedded are increasingly limited. Want to write some code for that cheap off-the-shelf SoC you bought? Scala is probably a perfectly good option, because it's probably an ARM with at least 1Gb of RAM and the JVM will run there just fine. If you're actually writing something that has to run for months off a single charge then yeah, Scala won't cut it whereas Rust may one day be able to. But most "embedded" these days isn't like that.

[haspok]

> Scala is limited mostly to JVM...

No, it's not. You can call native (C) functions from the JVM.

[pkolaczk]

Of course you can call C functions, but it is neither ergonomic nor performant. And good luck exchanging more complex data structures. Java can't use C structures directly, Rust can.

[esarbe]

You still need a JVM, which what the GP is trying to point out (I think...)

That said; nowadays you have the Native and JS runtimes as possible targets as well. Scala.js works surprisingly good with very few caveats.

[tastyminerals2]

Out of context yes. I should have said that with respect to which language pick up next and which language has a bigger future potential. Rust is so omnivore and hits on so many critical aspects that it is hard to deny it. Given it had rich and mature ecosystem now how many would pick Scala? Those who prefer GC pauses and have a hard dependency on JVM? But even then you are pitted against Kotlin and Java itself. I would say quite tough times are awaiting Scala 3 and not only on individual level but on company level. It is still unclear how much the migration will take and whether it is worth it in the first place.

[hocuspocus]

I don't exactly see how Rust can replace Scala where Scala is mostly used today. Or even Swift, despite Apple being more serious on the server-side roadmap lately and hiring a bunch of people from the JVM world.

Kotlin doesn't bring anything new and is full of ad-hoc design decisions, and Java itself is catching up (and even potentially surpassing it, see pattern matching for instance). I don't think it's a huge threat. Sure you'll see people who didn't really understand the point of Scala move to Kotlin because they just wanted a better Java really, and Kotlin does that well. But I think it's been clear that Scala shouldn't try to pursue this goal, for the past 5 years at least, after the hype around Spark cooled off.

I'm happy that Scala 3 is focusing on the biggest pain points. I'm also quite happy with the resources that have been added behind the tooling and ecosystem lately. The Scala center wouldn't exist if there wasn't a real demand for improvements in the mid to long-term future.

[aphexairlines]

> compiling times force you to leave your seat for a cup of coffee

This is not the case in a codebase split into multiple modules that compile in parallel.

[TeeWEE]

For me scala's feature set is absurdly big and there are always 10 ways todo things. Look at the spec of Scala. Its 200 pages long. That's insane.

I'd rather use Kotlin.

[mbo]

The Kotlin spec is...

275 pages: https://kotlinlang.org/spec/pdf/kotlin-spec.pdf

[aphexairlines]

And 796 pages for Java https://docs.oracle.com/javase/specs/jls/se14/jls14.pdf

[zaphirplane]

Yeah but the font ;)

[rgoulter]

> For me scala's feature set is absurdly big and there are always 10 ways todo things.

One of my favourite examples: of these 13 different ways of doing the same thing, only 12 are correct. https://nurkiewicz.github.io/talks/2014/scalar/#/22

[dtech]

I'd say that holds about as much value as criticizing C-style languages for allowing all these variants

    if(x) true
    if(x)
      true
    if(x) (true)
    if(x) {true}
    if(x) {
      true
    }
    if(x)
    {
      true
    }

etc. In practice you use a linter to enforce a style and it's not a problem.

For the curious. The combinations follow out of fairly simple rules:

    () and {} are interchangeable for expressions - altough {} can contain multiple statements and () only an expression.
    x op y is equivalent to x.op(y)
    Type abscriptions - x : Type - are optional and will be inferred if possible
    { case ... } is the pattern match construct and works similarly to a function with 1 parameter.

[valenterry]

How is that a bad thing? It's the same in other languages and it is nice to have some flexibility.

E.g. lisp: use whichever type of parentheses you want

Or Kotlin: use short syntax lambdas fruits.filter(it == apple) or long syntax fruits.filter(fruit -> fruit == apple) or with annotated types: fruits.filter{fruit: Fruit -> fruit == apple}

Sometimes brevity is good for the reader, sometimes more details are good for the reader. Not everything is a nail.

[misja111]

The flexibility is perfectly fine when you're programming something on your own. But when you're in a team, or worse, in a large company, it starts to be a problem that everyone can use their own style. It's much easier to read someone else's code when you have a common code style.

And in Scala there are not only many different ways to use the standard libraries or to structure your code, there is a pure FP vs OOP-style FP schism as well. Because of this, you can qualify as a senior Scala developer for one job but only a junior in another.

[thinkharderdev]

I've been writing Scala for the better part of 8 years now and I have my list of gripes about the language, but this is one I have never understood.

Yes, there are different code styles you can use with Scala (standard vs infix notation, parens vs braces, etc) but all of that can be standardized with code formatting tools.

In terms of FP vs OO style I don't think that is different for any other language. No matter which programming language you choose you have to make decisions about what sort of patterns you want to use in which scenario and enforce that across the team. I have seen many Java projects where a relatively small codebase has approximately every GoF pattern implemented somewhere (and a few novel patterns just for good measure).

[valenterry]

The opposite. If I write code on my own just for myself I use the highest level of brevity.

But if I write code for a team, then in some places I will use explicit type annotations and variable names to aid people unfamiliar with the code to understand what's going on.

What you are saying is pretty much "it's easier when everyone only uses nails, because then all I have to bring is a hammer". I think it's good to use screws sometimes.

However, your second point I agree with. OOP vs. FP is a different story. This is about paradigms not about mere syntax. So here, a team must be aligned, which can be a challange when using Scala. It's not a language for corporate drones.

[tastyminerals2]

If it was so simple as just using explicit types :) And this sort of flexibility is exactly why Scala is hard to read. Perl is another example of a great language that allows you a lot of flexibility. In fact, the ability to use just any symbol for a method is the worst thing about Scala. You quickly realize it once you start using Scala libraries some of which basically introduce you to a new Scala based DSL. This quickly becomes a nightmare to work with if several such libraries are used in the project.

[pkolaczk]

> But when you're in a team, or worse, in a large company, it starts to be a problem that everyone can use their own style

I was in many failed or semi-failed projects and code style was never the problem. Yes, people do use slightly different styles, but style is really easy to enforce - there are tools to do it automatically. And even if somebody misplaces a brace or writes `map(_.length)` instead of `map(x => x.length)` this doesn't impact the readability as long people in your team know the language. If a project fails to meet the deadline because of the code style it is not because of people using different styles but because of developers bikeshedding about the code style in code reviews instead of doing real work.

[vvillena]

The Scala ecosystem has tooling to enforce code style across teams.

Your second argument is true, Scala is a big language, and there are three main paradigms: better Java, OOP+FP, and pure FP. It's similar to how C++ is used, for some people is just like C with some improvements, but there are people using lots of C++ features.

[AzzieElbab]

That is when company wide style guide helps.

[MaxBarraclough]

> How is that a bad thing?

This doesn't quite answer your question, but, the Python community takes seriously the idea of There's Only One Way To Do It, as part of their philosophy on complexity.

https://wiki.python.org/moin/TOOWTDI

https://www.python.org/dev/peps/pep-0020/

[pkolaczk]

Seriously?

How to do data structures:

1. just put everything ad-hoc into a dict or a list (the "PHP" way :D) 2. use a namedtuple 3. define a class 4. define a dataclass (preferred) but works only if your Python version is recent enough

In Scala? Just use case classes and this is the only recommended way (#1 is very impractical so noone does that, #2 doesn't exist, #3 possible, but impractical when you have #4 in all versions).

How to map a collection in Python? - Start from an empty one and add mapped items in a loop - Use map + lambda - Use list comprehension - Update all items in place with a loop

Is it really any better than in Scala?

[valenterry]

Scala goes quite far in the flexibility, maybe too far. Agreed.

But is python really so good? I'm not a heavy python user, but there are loops and list comprehensions and map. There are also optional type annotations now, should you always use them, or not? How about "a is b" or "a != b"? How about environments and build tools...

I think that go might have been a better model student.

[aliakhtar]

This is like saying, airplanes are too complex. There are so many knobs and buttons. I'd rather just walk or ride a bike everywhere.

Scala is a very powerful, very expressive language. There are some features which you can just choose to leave out. If you do, you end up with a very clean, concise, and powerful language that makes you really productive.

I've literally had moments where I made my algorithm 5x faster just by adding 4 letters: '.par' in front of an operation. Instantly it got parallelized without my having to do anything, and the processing time got cut 5x.

[TeeWEE]

Its more like saying: I need to cut some paper ( i need to program): Swiss army knifes (scala) are too complex, I just need a pair of pair of scissors (lisp / other languages).

[aliakhtar]

Scala really isn’t that complex though. It’s actually pretty intuitive.

You CAN write bad and hairy scala code just like you can in any language. But that doesn’t mean the language is to blame.

[tastyminerals2]

And that's why there are far fewer pilots in this world than stewards.

[pkolaczk]

Having switched from Scala to Python recently, I must say Python has probably 5x more ways to do things than Scala and I can't see people complaining on Python's complexity.

This is something that really surprised me, because "one obvious way " was announced to be a part of Zen of Python.

[nsonha]

and 0 way to do a lot of things

[trumpeta]

IIRC one of Odersky's stated goal for Scala 3 was to address exactly this issue. That being said, it is developed in a research setting so I guess even if they restrict these features now, there will be new ones added over time...

[vvillena]

Scala has always been an opinionated language. The main difference is that this time is holding opinions against itself. Previously, opinions were held against Java, that's why there are features such as case classes, traits, and immutability by default. They answer to Java problems.

Scala is now mature enough that some issues have emerged. Typeclasses are cool, but programming typeclasses in Scala is a bit clunky. The type system is very powerful, but it could be even more powerful and that would actually help people write simpler code. And there are a bunch of unused features that could be removed. This is what Scala 3 is about.

[esarbe]

Scala 3 tries to be more opinionated and goes quite some miles (kilometers?) to make the language more regular and simple.

The implicits are gone, for example. Kinda crazy, since it was Scala's banner feature for so long. For the primary use-cases of implicits (extensions, type classes, conversion, ...) there are now language level concepts to make these use-cases accessible.

"Intent over mechanism" I think Martin calls it.

[pjmlp]

Yep, https://arrow-kt.io/

Just give it a couple of more years.

[willtim]

I am skeptical about the novelty of Scala's "OOFP" as you put it. It looks essentially the same as the original anonymous classes or SAM types in Java. Function values are objects with a single method and methods themselves are not first-class, but are wrapped in objects as required. The most important part of OO are the first-class modules; and in the literature, arguably more elegant examples of unifying first-class modules and FP exist, for example by starting with FP and adding objects as records containing functions, this way both "objects" and methods would be first-class. Subtyping also adds significant complexity to the language and weakens type inference.

Scala is, first and foremost an OO language which embraces subtyping, class hierarchies and imperative programming, see for example its own collections library. As Java gets more and more FP features, there is a danger it will struggle to differentiate itself.

[sjrd]

Your comment makes sense if we think of FP as being about "function values". In a world where all sorts of programming languages have function values, this is not a strong enough characterization anymore (if it ever truly was).

Functional programming is about programming with immutability first. So OOFP is about immutable objects. I go even further to say it is about objects with immutable interfaces, but the implementations can be imperative as long as encapsulation hides the mutable aspects from the user of the object. This is what you see in Scala's collections library: we use mutability inside the implementation but we expose an immutable API.

I gave an entire talk on that very topic at Scala Italy in 2019: https://vimeo.com/362001027

[willtim]

> Functional programming is about programming with immutability first.

I completely agree with this. So perhaps I am being rather hard on Scala.

[kitd]

It looks essentially the same as the original anonymous classes or SAM types in Java.

Decompiling a Scala .class file shows just how true this is.

[codeviking]

I really enjoyed writing Scala code. I'd agree with your evaluation with respect to the expressiveness and power provided by the language.

...but the tooling, at least at that time, was terrible. SBT was terribly overcomplicated, and full of foot-guns. I spent far too much time debugging dependency collisions, issues created by so-called "autoplugins" and other nuances that had nothing to do with getting real work done.

Now I'm writing a lot of Go. I'll admit, it's not nearly as fun to write. There's a lot less creativity and expressiveness -- though I think this is probably a good thing. But it also feels like I'm getting a lot more stuff done, and the tooling is amazing.

[cryptos]

I have the same feelings about Scala. It was fun to write Scala code and it was inspiring, but the tooling was terrible and there were way too many possibilities to express the same thing. Scalaz was essentially a language in a language that had nothing todo with "normal" Scala.

Now I'm using Kotlin. While a miss a more advanced Scala feature from time to time, I feel more productive because I don't have to think so much about the language itself. And Kotlin is much easier to introduce, plus the tooling is way better.

[lmm]

SBT is a fractal of awfulness and I have no idea why anyone ever uses it. But it's never been required. Just use maven and get on with your life.

[cryptos]

Yes, SBT is just terrible! It is the reference for usability nightmare in the category build tools.

[nsonha]

cbt and mill are just obvious scala code

[lmm]

How's their IDE integration? (Though TBH I'm coming from a starting point of "Maven isn't broken, there's no need to fix it")

[nsonha]

for mill intelij and vscode via the metals extension

[tlarkworthy]

I use maven to compile Scala.

[thirtyseven]

> Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance)

Compared to what? If you mean engines like Presto, I would say they have different use cases, plus Spark SQL puts them roughly in the same ballpark. Genuinely curious though, as a Spark user that is always interested in alternatives.

[smabie]

kdb+/q performance smokes Spark. Actually. most native code on a single machine smokes an entire Spark cluster.

[JD557]

> Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag.

I think it's not that bad. Sure, it loses to Python, but that's some tough competition.

Personally, I find that smile[1] covers most use cases. Breeze[2] also has a lot of love, but I'm not much of a fan of the `UFunc` abstraction.

I'm also really excited about polynote[3]. It's still a bit green and only supports Scala 2.11/2.12, but it's such a joy to be able to use Scala in notebooks :). You also get Python interop, in case you need to use some specific python library.

P.S.: I hope that Scala's ML/AI libraries other than Spark keep growing, because "embarassing" is a nice way to describe it's performance on everything that's not a "really-huge dataset".

1: https://github.com/haifengl/smile 2: https://github.com/scalanlp/breeze 3: https://github.com/polynote/polynote

[tastyminerals2]

Many if not most Scala data/ML libraries are one-person projects which are either abandoned or have compatibility issues. It's a barren land and loses to any similar Python library in terms of functionality. It's good that Java ecosystem can back Scala up otherwise it would be pretty sad.

[moonchild]

> only one language I like more (kdb+/q), but that's a hard sell, especially because of the cost

J? It's not quite the same, but is OSS.

[sireat]

When Scala, IntelliJ and SBT cooperate the language is so pleasant to work with.

There are some puzzling omissions though.

For example slice method is missing step. Coming from heavy data munging Python this really bites. Sure would be nice to have some syntactic sugar for slice.

Some seemingly simple tasks have no one way of solving them.

Let's take parsing JSON. Trivial in Python, painful in Scala. (almost the reverse in Python which has painful XML parsing and Scala's built in support for XML)

So far the easiest JSON library has been Li Haoyi's uPickle: https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html

Still, it does not parse JSON where objects have uknown arbitrary value types. Arbitrary value types are extremely common in real life JSON.

Due to pattern matching you see people suggesting you write your own parser! Sure it can be done, but then the next thing you'll be rolling your own crypto...

[sjrd]

> So far the easiest JSON library has been Li Haoyi's uPickle: https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html

> Still, it does not parse JSON where objects have uknown arbitrary value types. Arbitrary value types are extremely common in real life JSON.

Looks like ujson, which the article you point to talks about, is what you're looking for. uPickle is a layer on top of ujson for statically typed stuff, but ujson is working with raw JSON values, of arbitrary types.

[sireat]

Thank you! I will have to look into ujson deeper.

[lihaoyi]

You can in fact mix typed and untyped parsing, by having a `ujson.Value` field in the middle of your typed case class or collection. It just works

[sireat]

Thank you Li!

`ujson.Value` is in chapter 8.1.1. of your excellent Hands-On Scala book I've bought.

Turns out for arbitrary JSON you can do this:

  val rawData = read[ujson.Value](JSONstring)

For JSON where you know most of the structure but have mixed types for object values: https://jsonplaceholder.typicode.com/todos

  val todoData = read[Seq[Map[String,ujson.Value]]](todoJSON)

[vips7L]

Why wouldn't you just use Jackson for JSON? It makes parsing JSON trivial on the JVM.

[superyesh]

>Too bad there's not really a ML/AI ecosystem around (besides Spark, which seems to be losing favor partially because of its embarrassing performance), because using Python is such a drag.

Even if Spark might not be the best tool for ML, it is still the tool to beat for data processing and custom ETL.

[Thalarg]

> considering the absurd productivity compared to Java/Kotlin/Python/blublang

I have not see more unproductive language than Scala.

[newhkusers1]

Scala deserves all the hate. It might be a fun language for a throwaway pet project but I'd never recommend it in an enterprise setting.

It's an extremely unproductive language because it's too flexible. There are so many different ways to do the same thing which introduces unnecessary complexity.

[hocuspocus]

Yet plenty of big companies manage to ship software written in Scala just fine.