[indescions_2018]

It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS / D3 on the frontend. C# / Unity 3D modelling and scripting. Even low level C / Assembly if you need performance optimizations.

I think mastery of cloud development and deployment is crucial going forward. Start experimenting locally right on your laptop with minikube, microk8s, snaps, prometheus. And get a sense of designing scalable systems.

You should have the ability to immediately estimate project costs. So you can give potential stakeholders a ballpark figure to open negotiations. Billion item recommendation system running on spark that serves 100K+ requests per second deployed to 1000 node cluster. Will it cost under $100 per hour? Or do we jump to tensorflow and a few large gpu instances for half that. Once you gain sufficient mastery. It will become second nature. And you can concentrate on design and building.

Best of luck :)

[lmm]

> It's inevitable you will have multiple languages / platforms comprising your stack. Bash scripts for admin. Golang to write servers. Data science in Python. JS / D3 on the frontend. C# / Unity 3D modelling and scripting. Even low level C / Assembly if you need performance optimizations.

It's not at all inevitable. I would - indeed do - use Scala for all of the use cases you listed.

[ralphc]

Python is the de facto language for the deep learning part of data science, the interface to TensorFlow, PyTorch and Keras. Others may be possible but all the examples, books, etc. use Python.

[Brockenstein]

Hey, someone found a golden hammer! Better throw out all those other tools, who needs 'em.

[lmm]

Programming is not carpentry. The best programming languages (IMO) are general-purpose, and the gains from being able to know a single language in depth and reuse the same tools and techniques across different domains outweigh those from using different languages in different domains.

[AnimalMuppet]

Depends. Yes, learn a general-purpose language, and learn it well. But also, learn some other things well enough to be able to use them. Use them when they make the job easier.

For example, my main language is C++, and my second language is Java. But when the problem I have is picking snippets of text out of a text file, I reach for Perl, because Perl is far better at that than either C++ or Java. I don't want to "reuse the same [C++/Java] tools and techniques" in that domain - it's considerably more painful than using Perl to do it.

[ZirconiumX]

>But also, learn some other things well enough to be able to use them. Use them when they make the job easier.

I've found that even if I don't need to use a language after I've learned it, the skills and experiences I pick up while learning it improve my code as a programmer.

I've learned how to write fast code in C; learned how to write flexible template code in C++; appreciated the "batteries included" standard libraries of Python; wrote concurrent code in Go; used sum types and functional programming in OCaml.

My friend wrote an article[1] that says there is still no silver bullet for programming, and I agree with her.

Perhaps even if you think it is better to master a single programming language, you might better master it by seeing how other languages approach the same problem.

[1]: https://chameleon.kingdomofmysteries.xyz/articles/harmful/st...

[lmm]

> For example, my main language is C++, and my second language is Java. But when the problem I have is picking snippets of text out of a text file, I reach for Perl, because Perl is far better at that than either C++ or Java. I don't want to "reuse the same [C++/Java] tools and techniques" in that domain - it's considerably more painful than using Perl to do it.

Not convinced. I used to work in a similar way (indeed I would've made a similar argument), using Java for a primary codebase and Python for lightweight scripting around it. But I found that it was possible to adopt a general-purpose language that did both of those well.

[sghiassy]

I agree with Imm

Check out this boat car btw! (http://yachtpals.com/boat-car-9158)

It’s a boat. And it’s a car. It’s amazing.

People are so stupid that they’re still buying cars that can’t go in water. When will they learn?

[cpursley]

I first read that as "bloat" car ;)

[sghiassy]

Let’s rewrite the Linux kernel in Scala!!

[preordained]

I know we're just having fun here...but if we really could standardize on one language (or mostly one language) the development benefits would be insane.

[projectramo]

That is an interesting response.

I wonder if it were equally possible with any language.

Could a Python or Java programmer say the same? I think there are probably a handful of languages that you could use for "everything". The limiting constraint is the presence of a reliable javascript compiler, followed by data science libraries (I think).

[lmm]

I think most languages could get close - see e.g. https://www.teamten.com/lawrence/writings/java-for-everythin... . Of course if you want to actually execute code in the browser then you need to compile to javascript, but I'm not convinced that's required as often as people think it is (Wicket offers a very interesting AJAX model where you re-render UI fragments on the server in response to changes - all the code that runs is Java, but the behaviour ends up very similar to if you had client-side javascript making a server call to fetch data). "Data science" means many things to many people; you wouldn't want to write actual linear algebra in Java (or any language without operator overloading), but again the cases where you can't reuse a prebuilt model from your language of choice are rarer than one might think.

Scala was explicitly designed as a "scalable language" with both the strong safety guarantees that you need for large-scale development and the lightweight syntax that you need for throwaway scripts. But a lot of modern language design is converging on the same set of features (e.g. static typing for safety but type inference for convenience, pattern matching for the safety of the visitor pattern but the convenience of an if/else) and would have the same abilities. E.g. in a Reddit post about Rust a couple of days ago people were talking about writing web services in Rust, and part of the rationale there is that while Rust may not be the perfect web service language, it's good enough that if you already have Rust in your stack you might well write web services in Rust rather than bringing in a new language. I haven't seen anyone doing data science in Rust yet, but I see no fundamental reason why the language couldn't be adapted for it.

[_glass]

Rust is better suited as universal language than Scala for me as I am writing also music applications, which really hate the garbage collector needed in Java/Scala, things that run on the JVM. I never really needed recursive types, but that's also something limiting on the JVM.

[lmm]

> Rust is better suited as universal language than Scala for me as I am writing also music applications, which really hate the garbage collector needed in Java/Scala, things that run on the JVM.

That's the conventional wisdom about GC, but I'm not sure how true it is these days - prototyping a video streaming program in a previous job we were surprised to discover that Haskell performed significantly better than C++ in that role. The JVM GC is quite tuneable and if you're willing to sacrifice throughput for latency you can get it quite low.

(Rust has a lot of good points but I won't be interested until it gets HKT).

> I never really needed recursive types, but that's also something limiting on the JVM.

What do you mean? Recursive types in the sense I'm used to work fine on the JVM; there is even e.g. a recursion-schemes like library for Scala (matryoshka).

[TooBrokeToBeg]

> That is an interesting response.

It's a terrible idea. Writing and running scripts from Scala is incompatible with certain actions. Specifically provisioning and kernel configuration. There are lots of tasks that Scala is a bad tool.

[sghiassy]

Web browsers only run JavaScript, so in fact if you wanted to write front end code, then you would indeed need to know JS.

Even if there’s a Scala transpiler or WASM target - if you think you’re going to ship to production and blissfully never need to know JS...

[lmm]

I "know" javascript in the same way that I "know" SQL, or JVM assembly. It's part of the stack, you do want to be able to read and understand it, but you don't use it for programming (that is, expressing business logic) in.

[sghiassy]

Which proves the commentor’s first point:

“It's inevitable you will have multiple languages / platforms comprising your stack.”

[lmm]

Different senses of "stack". It's inevitable that there will be multiple technologies involved in the running of your code, which will involve multiple programming languages, and you will need to understand the behaviour of those technologies (though less and less as they mature - I wouldn't be surprised if it were possible to debug a compile-to-js language without understanding JS these days). But that's not the same thing as being able to program in those languages.

[mi100hael]

https://www.scala-js.org/

[passivepinetree]

Scalia for frontend work? How do you do it?

[lmm]

https://www.scala-js.org/

[hobls]

People who want jobs don’t usually have the luxury of picking them by the languages used.

[lmm]

People usually get hired to use a particular primary language (though even then, I've worked in jobs where I was hired for Java that ended up switching to Scala mainly due to developer preference), but often have broad discretion over which languages to use for secondary tasks. Certainly in a lot of "Java shops" the pattern I've seen is that devs will not be second-guessed (or at least, will contribute to team consensus) in their choice of bash/python/ruby/scala/... for things like testing scripts or data migration tools, nor in their choice of coffeescript/typescript/... for frontend parts.

[dangom]

Data science?!

[lmm]

Oh yeah. Spark has lots of good aspects (though also some bad ones), and personally I find it much easier to think in terms of code with map/filter/aggregate/... than in terms of tables.