When someone says they want to work in Perl6/Raku, they likely have vastly different problems than someone using Erlang/Elixer. There is overlap of course (both general purpose), but I can't imagine using Erlang for scripting, while Raku is first class here.
I can't imagine scripting in Erlang. Or rather, I've seen it done and it wouldn't be my first choice.
Elixir, on the other hand, has many nice features for scripting, such as better string handling, process pipe-lining, and an excellent set of first-class generic data structures (beyond just map and list). It also happens to have Erlang's fantastic concurrency model.
As a testament to this statement I wrote a Google maps API scraper to get a list of driving distances from a pair of csv tables - shortname + address and shortnames to and from, in about 1 hour, and the whole thing clocked in at 90 lines of code including about 40 lines of comments.
That seems about right for a competent Python coder. It or Powershell or Linux command line tools would have very little code and be very easy to whip something up. Perl would be great here as well or honestly any dynamic language with great library support.
Elixir still has to borrow arrays from Erlang and it's a pretty ugly implementation at that so no - I would not agree that Elixir has an excellent set of first-class generic data structures.
Neither Elixir or Erlang have real arrays, unless you want to count tuples. That said, I don’t see how that correlates to scripting, the data structures that are first class in Elixir are very ergonomic and certainly flexible enough to tackle pretty much any task you’d use a scripting language for.
I guess it depends on what you're comfortable in? I myself have tons of Elixir scripts that I use to automate much of my life. It's replaced ruby as my go-to for one-off scripts.
Oh man I just love and adore Crystal. I am a long time rubyist and I have to say they just nailed it with Crystal. I just wish I had more opportunity to use it professionally!
It must be time to take my Ruby skills and go and learn this. Is it really as easy as it sounds like it should be for a lazy, dynamic-typing sort of Ruby dev to pick up?
If you have never worked with a strongly typed language before there might be some getting used to that, but any pain is well worth it. Catching all your mistakes at compile time and not at runtime is so amazing. Oh and it’s literally 100x faster than Ruby in some cases.
Is a 100ms startup time really a big deal for something that is either going to be manually triggered or cronjobbed and expected to take on the order of minutes?
You know that you are timing the compiler and not the compiled program here?
Rather try "echo 'puts 1' > puts.cr && crystal build puts.cr && time ./puts"
BTW, you can dramatically decrease the cold startup times of ruby if you call it with the options '--disable-gems --disable-rubyopt' if that's an option to you. For many scripts it certainly is.
But even on a 8 years old machine with spinning rust, after the ruby stuff is cached in the OS filesystem cache, my startup time are smaller than 10ms.
Elixir/Erlang isn't a general purpose language. You pointed-out one reason why - scripting. Computation is another area where the language is deficient. I'd go so far as to say Elixir's failure to challenge mainstream languages for mindshare is down to to it occupying a niche - lightweight concurrency and distributed systems.
Their point was that elixir is a good language for scripting. I'm not sure if this is true but I don't see any reason it couldn't be. I don't think "distributed systems" are niche... I think every web application I've worked on fits that description, which is basically my whole career at all different kinds of companies.
For concurrency I think Raku has slightly more developed async support than Python, but the bigger advantage, I think, is in parallelism where, aside from the CPython GIL limiting practical parallelism in the main implementation (which is a big deal), Python as a language lacks the parallel iterables produced by the hyper (parallel but constrained to return in order) and race (parallel and not constrained in order) methods on the Iterable role in Raku, or anything like Raku’s hyperoperators that are semantically concurrent and parallelizable at the discretion of the optimizer. (Come to think of it, while also parallel, all those are also high-level concurrency constructs that Python lacks equivalents to.)
Python as a language can support parallelism via threads, and CPython as an implementation can via multiprocessing, but those are both very low level abstractions; Raku has higher level abstractions which allow much more succinct expression of parallel operations.
Sounds like multiprocessing.Pool.imap and imap_unordered? When dealing with io you also have async/await equivalent iterators like asyncio.as_completed().
sub foo ( Int \n where 1..100 ) {
# ___ start is a prefix operator that creates a Promise
# V
start { sleep n.rand; (1..n).pick }
}
my $result = foo(10);
say $result.status; # Planned
say await $result; # 6 (after a delay)
# this is nearly identical to foo above
sub bar ( Int \n where 1..100 ) {
Promise.in( n.rand ).then({ (1..n).pick })
}
(Note that `start` does not need a block unless you are grouping more than one statement as I have above.)
There are combinators like `Promise.allof` and `Promise.anyof`.
You usually don't need to use `Promise.allof`, because you can use `await` on a list.
my @promises = Promise.in(1.rand) xx 10;
my @results = await @promises;
(Note that the left side of `xx` is _thunky_ so there are ten different Promises with ten different random wait times.)
---
You should see the `react`/`supply` and `whenever` feature.
I am not that familiar with Python, but the GIL has long prevented any real in-process concurrency. Perl has concurrency but it's complex, heavy, and poorly supported. Raku's approach to this is built to avoid all these problems (like Elixir).
I agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?
If it's entirely CPU bound, you can use multiprocessing to negate most of the GIL issues, and transparently send inputs/outputs between the parent and child processes. If it's I/O bound, then AsyncIO is a great way to express asynchronous workflows. If it's a combination of both I/O and CPU bound workloads, there are ways to mix multiprocessing and AsyncIO to better saturate your cores without losing the simplicity or readability of Python: https://github.com/jreese/aiomultiprocess
Very true, I had not even thought about the multiprocessing package; it's sometimes not as convenient as multithreading but it'll get those other cores working.
Indeed and as a Perl developer I make use of XS/external libraries, cooperative multitasking (event loops/promises), and forking to cover these use cases. It doesn't preclude wanting the additional option to take advantage of threads in a useful way, since they do exist.
Why should first-class concurrency needs be required to script in Elixir? This question seems to imply that Python is somehow a default language and special requirements must be needed to justify writing in something else. Elixir is general-use and pleasant to write scripts in so seems reasonable to me for someone to do so if that's their thing.
lliamander replied that Elixir does too and it's worth a look
To that, 7thaccount replied that Perl6 and Elixir fill different niches.
So far, it seems Perl6 fills a niche that requires scripting and first-class concurrency.
My question then is: what is this niche that requires very solid concurrency but also scripting. In other words, what does Perl6 have in terms of concurrency that Python does not (given they are both scripting languages)?
I don't know of many instances where scripting and concurrency would be needed in the same application. But if you wanted to use single language both for scripting tasks and applications that require concurrency, then Raku or Elixir would work.
One instance I can actually think of, that would be specific to Erlang/Elixir, is if you have a long running service that you sometimes run batch jobs against (like importing data from a CSV). An Elixir script can hook directly into an Elixir/Erlang service and interact with it using message passing. It's a heck of a lot simpler than RMI, and avoids the need to expose new HTTP endpoints specifically for such tasks.
More to the point, does Python 3 have operators that transform sequential operations into operations that are both concurrent and parallelizable, and, in the case of iteration, provide control of parallelism parameters and whether results are constrained to be in-order or not.
To which the answer is “not only does Python not have them, but with the GIL CPython couldn't do much with them even if the language had them.”
I'm embarrased to say I write scripts in node :P I used to write scripts in python but I've been writing so much JS that it's just easier for me in node. Plus, node defaults to locally installed deps so I don't have to deal with virutal environments.
Well just from experience untangling async calls in python is a nightmare and sometimes hard to reason about. The red/blue function problem is real. Meanwhile dispatching concurrent long-running scripting tasks is basically trivial in elixir (Enum.map over Task.async, then Enum.map over Task.await)
I'm guessing because Python's concurrency relies on the Global Interpreter Lock. Although I think concurrent.futures might address that. Haven't worked with python concurrency libraries in a bit.
I agree the GIL is a problem but it's only an issue for CPU-bound problems. Is there really an important amount of CPU-bound work that is written in a scripting language? If it's CPU-bound, wouldn't you want to use something lower level?
tldr; Using a scripting language that allows for native threads or has a strong concurrency model builtin to the core would be beneficial for any CPU bound scripting task...
If your program is CPU bound the GIL will slow you down. I'm sure since the python ecosystem is quite vast there are ways around the GIL... but then you have to worry about every library you use not supporting "real" multi-threading, or (b)locking for no reason and shitting all over your efforts.
As I've posted above, I'm a bit confused by CPU-bound work being processed in a scripting language. If you're planning on doing intense CPU-bound work, maybe use a lower-level language? I'm not saying abandon Python: you can extend Python with C or just use IPC to transfer data between a Python front-end and a computation back-end.
When I have a bit of Raku code that is too slow I complain (send a bug report) and then someone else figures out why the JIT isn't doing a better job and fixes it.
Then bingo-bango it ends up even faster than using NativeCall to call into the C code.
Of course there may be a delay before someone gets around to figuring it out; so in the meantime, NativeCall is actually very easy to use.
---
I would like to note that someone wrote Perl6/Raku code and the equivalent C/C++ code. They reported that the Perl6/Raku code was faster. (It was before Raku was brought up as an alias/name.)
I'm fairly sure that the reason the C/C++ code was slower is that it copied strings around rather than using pointers and indexes into other strings like MoarVM does.
At one time it was an obvious dichotomy that you would not use a scripting language for CPU bound work, but these days it is a much more blurry line. Partly because modern efficient languages are becoming ergonomic enough to work well as scripting languages while still giving you very good performance.
I actually love doing CPU bound work in Groovy which is usually described as a scripting language. But it gets converted straight to java byte code which is JIT'd and ends up as machine code. It only takes a few static typed hints here and there and it runs in the same league as C++ implementations. And it gets Java's excellent concurrency support for free.
I totally you feel you, I guess I thought your question was substantially more surface level than it was. My apologies.
I'm personally with you. I also don't tend to think object boxing is really the performance bottleneck for most applications, and if/when it is, likely the other requirements should've already ruled out using one (a scripting language).
It's like writing Nifs for Elixir, yeah sure you _can_, they have their purpose, but you could also just write another application to do that one thing and like you said, use IPC.
So in summary, we agree with each other, here's to:
Erlang is great, and I miss working with it, but I'd never want to write much in the way of 'quick scripts' with it. Something like Ruby feels much more productive for that.
Even if GraalVM fixes startup times aren't JVM languages just too long-winded for scripting? Perl, Python and Ruby dominate the scripting world for a reason - standard libraries for file, dir and pathname manipulation written in a concise language. Scripting is a style of coding, not just a means to an end. It is here that dynamic languages excel. Clojure is the leanest of the JVMs but doesn't it still rely on Java for file, dir and pathname manipulation?
> Even if GraalVM fixes startup times aren't JVM languages just too long-winded for scripting? Perl, Python and Ruby dominate the scripting world for a reason
Ruby is a JVM language, in that JRuby is a very complete competitive, current, and widely-used-in-production Ruby implementation.
You should definitely educate yourself about Groovy. It's my favorite scripting language, particularly because it enables a true scripting style, but actually works well as a full application development language too (though I would always want a backbone of static typed code).
Joker is a Clojure dialect which is interpreted, thus it starts super fast (but runs slower, but fast enough for scripts). Its design is to be batteries included for all things scripting. So it's just a self contained executable with everything you need for scripting. It's implemented in Go.
I use it wherever I would have used bash or powershell prior.
There is also Babashka: https://github.com/borkdude/babashka which is a similar idea, it's an interpreted dialect of Clojure with fast start times designed for scripting. The difference with Joker is that it is newer and more experimental, and it is implemented in JVM Clojure, compiled with GraalVM and has no Windows support for now.
Anyways, I really recommend the use of Joker. Its awesome for scripting. I just put its binary in all my bin folders and script with it. It's great.
GraalVM does in fact fix that. There's a (new-ish) project called babashka that lets you do some basic scripting for example: https://github.com/borkdude/babashka
Oh certainly. I've done production work with Erlang, and it's great for building systems. I also love the Prolog syntax (I've also since done some side projects in Prolog for great fun).
Why I suggested Elixir is because it has all the strengths of Erlang but is also a better scripting language, and would compete better with Raku on a more equal footing in that respect.
In fact Elixir and Erlang are the only reasonably known dynamic languages that differentiate on concurrency. The rest either do higher order event loop concurrency, which is ok, but not a differentiator, or do worse, shared memory multithreading (threads, coroutines with synchronous channels, mutexes, etc.)
I don't really know what "differentiate" means, but Clojure has support for CSP-channel style with core.async, shared-memory with transactions using atoms/refs, and actor-ish style with the agents.