[rendaw]

Elvish had some very cool ideas, which is why I tried it out! Like the built in script checker! But it also has a lot of very basic issues that have been open for years, and TODOs in the documentation as I mentioned. People are going to read your message and put N hours into it and get burned, and I think this is a fair warning.

Nushell also had very minimal background task support, so I rejected that. They explicitly say use some other program for background tasks in their docs.

I actually looked at Murex after seeing it in previous threads, but I bounced for some reason... I just took another look though skipping the tutorial and I see you have `bg` and `fg` support! But does `bg` return the `fid`? Can you use those in scripts, or are they hobbled the same way bg/fg are in bash?

It's been a good 4-5 months since I went down this rabbit hole, but IIRC the basic things I wanted to do and got blocked in multiple shells were:

- System-wide interactive-use config file, I use Nixos and manage my system config using that

- Background task support - I need to start an ssh tcp proxy, pipe a command over it, then kill ssh once the command is done (all in a script).

- Post-command hook, to send a notification when a long command finishes

- Async iteration of command output, i.e. streaming events with swaymsg subscribe and running a command when certain events occur

- Value/call arity safety - i.e. a clear distinction between a single value and multiple values that doesn't rely on stringification hacks. I.e. in `command $x` `command` should always have one argument, regardless of the contents of `x`, and making that plural should be explicit.

And then other standard evaluation criteria, like I looked at xonsh but it seemed like a massive hack, despite handling a lot of the above.

[hnlmorg]

> does `bg` return the `fid`

There's two kinds of process IDs in Murex: FID (function IDs) and PID (process IDs)

Forking is expensive in POSIX and has a number of drawbacks such as the inability to share scoped variables without resorting to environmental variables. So a FID is basically a PID but managed inside the scope of Murex's runtime. You can manage FIDs in much the same way as you can manage PIDs, albeit using Murex builtins rather than coreutils (though PID management tools in Bash are technically builtins rather than coreutils too).

What this means in practice is you can have entire blocks of code pushed into the background, eg

    » GLOBAL.name = "rendaw"
    » bg { sleep 5; echo "Hello $name" }; echo "not bg"
    not bg
    Hello rendaw

You can see the FID as well as the job ID the usual way, via `jobs`

    » jobs
    JobID  FunctionID  State      Background  Process  Parameters
    %1     2109        Executing  true        exec     sleep 5

...and you can kill that entire `bg` block too

    fid-kill 2109

But you'd also see any non-builtins in `ps` too:

    » ps aux | grep sleep
    hnlmorg   72749   0.0  0.0 410743712   1728 s012  S+    4:24p.m.   0:00.00 /usr/bin/grep --color=auto sleep
    hnlmorg   72665   0.0  0.0 410593056    432 s012  S+    4:23p.m.   0:00.00 /bin/sleep 5

> Can you use those in scripts, or are they hobbled the same way bg/fg are in bash?

While the above seems very complicated, the advantage is that `bg` and `fg` become much more script friendly.

> - System-wide config file, I use Nixos and manage my system config using that

This isn't Murex's default behaviour but you could easily alter that with environmental variables: https://murex.rocks/user-guide/profile.html#overriding-the-d...

The latest version of Murex (v7.0.x), which is due to be released in the next few days, makes this even easier with a $MUREX_CONFIG_DIR var that can be used instead of multiple specific ones.

> - Background task support - I need to start an ssh tcp proxy, pipe a command over it, then kill ssh once the command is done (all in a script).

Murex has another layer of support for piping in addition to those defined in POSIX, which are basically channels in the programming language sense. In murex they're called "Murex Named Pipes" but the only reason for that is that they can be used as a glue for traditional POSIX pipes too. This is one area where the documentation could use a little TLC: https://dev.murex.rocks/commands/pipe.html

> - Post-command hook, to send a notification when a long command finishes

There are two different events you can hook into here:

- onPrompt: https://dev.murex.rocks/events/onprompt.html

This is similar to Bash et al prompt hooks

- onCommandCompletion: https://dev.murex.rocks/events/oncommandcompletion.html

This hooks into any command name that's executed. It runs the comment in a new TTY and buffers the command's output. So, for example, if you want a command like `git` to automatically perform a task if `git push` fails with a specific error message, then you can do that with onCommandCompletion.

> - Async iteration of command output, i.e. streaming events with swaymsg subscribe and running a command when certain events occur

I'd need to understand this problem a little more. The channels / Murex Named Pipes above might work here. As might onCommandCompletion.

> - Value/call arity safety - i.e. a clear distinction between a single value and multiple values that doesn't rely on stringification hacks. I.e. in `command $x` `command` should always have one argument, regardless of the contents of `x`, and making that plural should be explicit.

This one is easy: scalars are always $ prefixed whereas arrays are @ prefixed. So take the following example:

    array = %[ a b c ]
    
    » echo $array
    ["a","b","c"]  # a single parameter representation of the array

    » echo @array
    a b c          # the array expanded as values

-----

This is quite a lengthy post but hope it helps answer a few questions

[rendaw]

Thanks for the answer! I think we may be talking past eachother a bit, but it's good to get confirmation on a lot of those!

AFAICT named pipes have nothing to do with ssh tcp proxies, or at least that bit is tangential to the key point - the key point I was making is that ssh is running in the background while I'm running another command (with no relation between them, as far as the shell is concerned).

You didn't answer my question about if `bg` returns a `fid` or not, and the documentation doesn't answer this either, nor did you say if those could be used in scripts... but it sounds like you're saying I have to parse `jobs` to get the `fid` of the command I just launched?

> Async iteration

This isn't about objects but about how operators evaluate. Maybe "streams" or "async streams" would be a better description? Actually, maybe this is where the named pipes you mentioned would be useful?

    while true; do echo hi; sleep 1; done | while read line; do echo $line 2; done

in bash, prints "hi 2" once a second. That is, the body of the while loop is executed asynchronously with the pre-pipe command.

AFAICT elvish doesn't have a `read` command, and `for` waits for the argument to complete before executing the body at all. This is a pretty common pattern, so I was surprised when there's no mention of it in the elvish docs. I don't like `read` in bash since it's yet another completely different syntax, but maybe Murex has something similar?

TBH I just looked at the Murex docs again and I'm lost. Where's the reference? There's the "user guide" which contains the "user guide" (again) and also the "beginners guide" which to me are synonyms, "cheat sheet" and "read more" which both appear to be a bunch of snippets (again synonyms?), "operators and tokens" which are a reference of just a subset of the language, etc etc. I couldn't find anything on loops, which I'd expect to be a section somewhere. Clicking on "read / write a named pipe" in "builtin commands" somehow teleports me to an identically named page in a completely different "operators and tokens" section.

In the end the read/write pipes page only shows examples of reading/writing pipes to other pipes, not using them in a for loop or anything. One example appears to use some unique syntax in `a` to send array elements to the pipe, but I couldn't find anything about that in the `a` section when I finally found the `a` section.

Also are named pipes always global? Why can't they be local variables like other structures?

It's great you have so much documentation, but I think it actually turned me away - it seems bloated with every bit of information split into multiple pieces and scattered around. Simplifying, consolidating, and reorganizing it from the top might help.

> Arity safety

Thanks! Yeah, I wasn't talking about arrays here, but if e.g. `x` in my example contains space-separated "these three words", will `command` receive 3 arguments or 1. In bash, `command` would get 3 arguments, if you don't invoke the variable quote hack. I just tried this out though, and it gets 1 argument, so great!

[hnlmorg]

> You didn't answer my question about if `bg` returns a `fid` or not, and the documentation doesn't answer this either, nor did you say if those could be used in scripts... but it sounds like you're saying I have to parse `jobs` to get the `fid` of the command I just launched?

Everything in the documentation can be used in scripts.

`bg` doesn't write anything to stdout. The documentation does actually answer that if you look at the Usage section. In there it doesn't list <stdout>. eg compare `bg` it to other builtins and you'll see the ones that do write to stdout have `-> <stdout>` in the usage. Also none of the examples show `bg` writing anything to stdout.

You wouldn't want `bg` to write the PID to stdout anyway because then it would be harder to use in scripts, because you'd then need to remember to pipe stdout to null or risk contaminating your script output with lots of random numbers.

However the good news is you can still grab the PID for any forked process using <pid:variable_name>. eg

    » bg { sleep <pid:MOD.sleep_pid> 99999; echo "Sleep finished" }
    » kill $MOD.sleep_pid
    Sleep finished

($MOD is required because bg creates a new scope so any local variables created in there wouldn't be visible outside of `bg` block. MOD sets the scope to module level (also supported are GLOBAL and ENV).

It's also worth reiterating my previous comment that `bg` blocks are not separate UNIX processes but instead just different threads of the existing Murex process. In fact all Murex builtins are executed as threads instead of processes. This is done both for performance reasons (UNIX processes are slow, threads are fast...relatively speaking) and because you cannot easily share data between processes. So if `bg` was a process, it would be impossible to grab the PID of `sleep` and then share it with the main process without then having to write complex and slow IPCs between each UNIX process.

> That is, the body of the while loop is executed asynchronously with the pre-pipe command.

Are you just talking about each command in the pipe executing concurrently? That's the default in Murex. for example the following would only work if each command is working as a stream:

    tail -f example.log | grep something | sed -e s/foo/bar/ | tee -a output.log

The loop example could be achieved via `foreach` too. eg

    while { true } { echo hi; sleep 1 } | foreach line { echo "$line 2" }

The only time `foreach` wouldn't stream is with data-types that aren't streamable, such as JSON arrays. But that's documented in the JSON docs (https://murex.rocks/types/json.html#tips-when-writing-json-i...)

> TBH I just looked at the Murex docs again and I'm lost. Where's the reference? There's the "user guide" which contains the "user guide" (again) and also the "beginners guide" which to me are synonyms, "cheat sheet" and "read more" which both appear to be a bunch of snippets (again synonyms?), "operators and tokens" which are a reference of just a subset of the language, etc etc. I couldn't find anything on loops, which I'd expect to be a section somewhere. Clicking on "read / write a named pipe" in "builtin commands" somehow teleports me to an identically named page in a completely different "operators and tokens" section.

I'd generally refer people to the language tour to begin with https://dev.murex.rocks/tour.html It's pretty visible on the landing page but sounds like we could do more to make it visible when elsewhere on the site. I can take that feedback and work to make to the tour more prominent in the menus too (the pages you described do also link to the language tour, but clearly not in a visible enough way)

> Also are named pipes always global? Why can't they be local variables like other structures?

The only reason for that is because named pipes were one of the original constructs in Murex. They came before modules, scoping, and so forth. It would be possible to make them scopeable too but careful thought needs to be made about how to achieve that in a backwards-compatible yet intuitive way. And there have been other requests raised by users and the other contributors that have taken priority.

> It's great you have so much documentation, but I think it actually turned me away - it seems bloated with every bit of information split into multiple pieces and scattered around. Simplifying, consolidating, and reorganizing it from the top might help.

We'd welcome some recommendations here. Organising documentation is really hard. Even more so when the people who organise it aren't the same people who are expected to depend upon it.

> Thanks! Yeah, I wasn't talking about arrays here, but if e.g. `x` in my example contains space-separated "these three words", will `command` receive 3 arguments or 1. In bash, `command` would get 3 arguments, if you don't invoke the variable quote hack. I just tried this out though, and it gets 1 argument, so great!

I know you weren't talking about arrays, but the next question people ask is "how do I now expand one variable to be multiple parameters?"

[rendaw]

Awesome, thanks for the answers! It looks like Murex will be the first shell I try once I recharge my "rip everything up and start over" batteries again.

> I know you weren't talking about arrays, but the next question people ask is "how do I now expand one variable to be multiple parameters?"

I held back from asking that, but yes, exactly :P

[hnlmorg]

Awesome. Good luck if/when you do. And feel free to pester me on Github with any questions / feedback / complaints you have :)