[willghatch]

Shell is my favorite domain-specific language. But many (including myself) would argue that domain-specific languages are generally better embedded. Many projects aiming to mixing shell with general purpose languages find a nice embedded DSL for subprocess/pipeline management. Some others find a convenient way to run shell commands or pipelines by mixing grammars and trying to disambiguate them. [Shameless self-promotion] I've been working on a project that aims to not only have a nice DSL for running process pipelines, but focuses on making a syntax for using all functionality of the host language as a command language, called Rash[1]. It is hosted in Racket, and can be embedded in Racket at the module or expression level, and also normal Racket expressions can be embedded in Rash (they can alternate arbitrarily deep). It supports process pipelines, Racket function/object pipelines, and mixes of the two. It's still alpha and has a TODO list a mile long, but I've been using it as my daily driver interactive shell for months and have loved it so far.

[1]: https://github.com/willghatch/racket-rash

[chubot]

FWIW I have your project on my wiki page :)

https://github.com/oilshell/oil/wiki/ExternalResources

I guess what you mean by embedded is that it should be an embedded DSL in a full-fledged programming language? I don't quite agree, since there are at least 20 projects like that on the wiki page, none of which is popular.

Probably the most popular one is eshell, in Emacs Lisp?

But if there's something I don't know about I'd be interested in hearing it. This idea goes back at least 20 years, e.g. to scsh. And it hasn't taken off.

But certainly I don't begrudge anyone if their favorite environment is Racket and they want to stay in Racket. That's a perfectly reasonable thing. It's just not what I would expect anyone besides racket users to use.

One reason I'm interested in shell because it's the lowest common denominator between different groups of programmers. C/C++ programmers, use it heavily, Python, Ruby, JS programmers, Go, etc. Everybody uses it.

[willghatch]

Yes, actually I've looked over all the shells on that wiki page. I think most of them haven't taken off because they either their host language has been poor or unpopular, their design or implementation wasn't great, or they haven't really solved the right problem. I think the idea of embedding a shell into a general-purpose language still has a lot of merit. Most of those projects trying to embed a shell DSL into a general-purpose language, scsh included, are basically for programming and not interactive use. Shells that are only interactive or only for programming end up fulfilling less than half of the purpose of shell, in my view, because the interaction and scripting feed back into each other in a virtuous cycle. The only ones on that list aside from Rash that try to be embedded in a general-purpose language while being good at both interactions and programming are Xonsh and eshell. Xonsh is pretty new (I wasn't aware of it until after I had made Rash), and eshell is in Emacs Lisp (which is not a very good programming language or platform to build on for anything except extending the emacs editor).

Rash also tries to be better than the competition by adding object pipelines (much like Powershell, it makes it much more reasonable to make system administration commands in the host language, and have rich interaction and inspection of command results), user-defineable pipeline operators, and generally tighter integration with the host language while still having a light syntax for basic commands.

I would like to be able to have my command language tightly integrated with my programming language, and be able to define my system administration commands (and interactive shell plugins and completion) as functions in the host language (while still being able to naturally use external programs). And I would like my shell scripts to be able to grow and potentially even turn into full-fledged programs, or modules of some larger program. I think there are a lot of benefits to the approach I'm using (which would be too long for a comment here).

That said, I'm not holding my breath for it to catch on widely any more than I'm holding my breath for Racket to take off as a popular programming language (although I frankly wouldn't mind either one). I think a better Posix shell is certainly a noble effort, because whatever better shell does become popular, we certainly need one. And an automatic upgrade path for existing scripts sounds great. So I salute you and wish you good luck with it. Also, as someone looking at shells and their features to copy the best ones, your wiki is a great resource. So thanks.

[chubot]

OK great, glad you have made use of the page. We had an "alternative shells" thread about a year ago between the authors of Elvish, NGS, Oh, and mash. Those were the main active/nascent shells I could find.

It might be time to start that again to see what ideas people have and what has changed. If you're interested e-mail me at [e-mail].

I'm also interested in the Shill shell, which as I understand was written in Racket, and then somehow they moved away from Racket? I'm not sure. I think it was because of the runtime. I also saw some efforts to move Racket to Chez Scheme.

I very much like Racket as an idea -- a meta language -- but I haven't gotten a chance to play with it too much.

And I did experiment with femtolisp as a basis for Oil -- the lisp used to bootstrap Julia -- but I decided against it.