[ygra]

Probably not. In my experience it's not so much about things being impossible elsewhere, just that PowerShell can often be better at fiddling in a REPL until you got the results you wanted, in a way. The point where you need (or want) to upgrade to a more powerful language is IMHO earlier in bash than in PowerShell².

PowerShell handles objects like Unix utilities handle text. You get a lot more orthogonality in commands, e.g. there's just a few commands dealing with JSON, CSV, XML, etc.¹ – they mostly do just the conversion of a specific text-based format to an object list or tree representation and back. Everything else after that can be done with just the same set of core commands which deal with objects. Filtering, projecting, and iterating over object sequences is probably the most common part of PowerShell scripts and one-liners and it's a part that's useful everywhere you need the language. Note that we have a bit of that in the samples here, too. ConvertTo-JSON is used to convert a hashmap to a JSON string to use in a request, and Invoke-WebRequest already handles parsing HTML for us so the following line can just filter elements by certain attributes.

Where the ideal format for the core Unix tools you use most frequently, is free-form text, you often have special commands that work on other formats by replicating a few core tools' features on that format, e.g. a grep for JSON, a grep for XML, etc. There are others, of course, that do the conversion in a way that's friendly to text-based tools, but the representation still lacks fidelity. Finding XML elements with certain attributes quickly becomes an exercise in how to write robust regexes. Cutting up CSV by column numbers is a frequent occurrence – and since the tools do not understand the format it makes for not a pretty read in the script's code. Personal opinion here, based on lots of PowerShell written, some shell scripts written, lots of horrible things read in either (sure, awful PowerShell scripts do exist, but I'd argue that discovering a nice way is much easier where you can cut most of the ad-hoc parsers written in regex or string manipulation in pipelines).

(One last point about orthogonality of commands: ls has a few options on how to sort or output the results, for example. Sorting a list of things? That's Sort-Object's domain. Formatting a list of things? Format-Table, Format-Wide, Format-List. That's quite a bit less each individual command has to do and it's all just for nicety to the user. For working with the output programmatically you don't (well, and can't) need them at all.)

I have a few posts on SO where I tried to steer people into actually learning how the language works and that you should use the pipeline as much as possible, e.g. http://stackoverflow.com/a/7394766/73070 or http://stackoverflow.com/a/3104721/73070. That's not to say you can't write un-understandable things: http://stackoverflow.com/q/1018873/73070.

And finally, let's not forget that PowerShell exists on Windows where text-based tools are mostly useless. Want to query the event log? The registry? WMI? Good luck. Windows has a long history of having non-text formats everywhere in the system and for administration it's a bit hard to pretend they don't exist. Jeffrey Snover elaborates a bit on that here: http://stackoverflow.com/a/573861/73070.

There may be a lot of developers getting by with Unix tools on Windows, but they're not the target audience for PowerShell³. And the previous approach to scripting things on Windows servers and domains was either batch files or VBScript/JScript.

This ... uhm, got longer than anticipated and probably a lot less coherent than planned. Apologies for anything that makes no sense. I didn't have caffeine yet.

______

¹ ConvertFrom-JSON and ConvertTo-JSON for JSON for example. For CSV there are also convenience commands that directly work on files as well, so there's four of them. XML processing is built-in since .NET can do that easily already.

² I probably don't get to live the rest of the day now, I guess.

³ I am a developer, though, who uses PowerShell daily for scripting, as a shell, and a .NET playground. The PowerShell team at MS was a bit surprised once when I told them that my background was not server administration (the Scripting Games were quite focused on that part and often involved doing things with WMI or AD – stuff I rarely, if ever, do.)

[aidos]

Your comment was useful, so not totally in vein :-)

Just incited me to have a little look at powershell (read through [0], useful intro). It looks nice, I can definitely see the utility in have a simple object model for transferring information between processes. In nix land you get pretty good at extracting data from simple text forms, though sometimes it's harder than it should be.

One thing that jumped out at me there is the overhead of the commands.

    430ms: ls | where {$_.Name -like "*.exe"}
    140ms: ls *.exe
    27ms : ls -Filter "*.exe".

Not so much the absolute numbers but the fact that there are 3 different ways of doing it and the more flexible choice is over a magnitude slower.

What happens when you add another command to the pipeline? Do they buffer the streams like in linux?

I guess the situation will improve over time but how complete is the eco-system at the moment? One area nixes will always shine is the total ubiquity. Everything can be done over commands and everything works with text.

[0] https://developer.rackspace.com/blog/powershell-101-from-a-l...

[ygra]

You found three ways of doing things that all do filtering at a different level. The -Filter parameter employs filtering on the provider side¹ of PowerShell, i.e. in the part that queries the file system directly. Essentially your filter is probably passed directly to FindFirstFile/FindNextFile which means that fewer results have to travel fewer layers. The -Path parameter (implied in ls .exe as it's the first parameter) works within the cmdlet itself, which is also evident in that it supports a slightly different wildcard syntax (you can use character classes here, but not in -Filter) because it is already over in PowerShell land. The slowest option here pushes filtering yet another layer up, by using the pipeline, so you get two cmdlets, passing values to each other and that's some overhead as well, of course. Note that the most flexible option here is by combining Get-ChildItem with Where-Object, whereas the direct equivalent in Unix would probably be find which replicates some of ls' functionality² to do what it does, which probably places it in a very similar spot to ls, performance-wise.

It's not uncommon for the most flexible option to be the slowest, though. In my own tests my results were 18 ms, 115 ms and 140 ms for doing those commands in $Env:Windir\system32, so the difference wasn't as big as in your case. For a quick command on the command line I feel performance is adequate in either case, unless you're doing things with very large directories. If you handle a large volume of data, regardless of whether it's files, lines, or other objects, you probably want to filter as much as you can as close to the source as you can – generally speaking.

As for buffering ... I'm not aware of, unless the cmdlet needs to have complete output of the previous one to do its work. Every result from a pipeline is passed individually from one cmdlet to the next by default. Some cmdlets do* buffer, though, e.g. Get-Content has a -ReadCount parameter that controls buffering in the cmdlet (man gc -param readcount). Sort-Object and Group-Object are the most common (for me at least) that always need complete output of the stage before to return anything, for quite obvious reasons.

However, even though I did some work on Pash, the open-source reimplementation of PowerShell, I'm not terribly well-versed in its internal workings, so take the buffering part with a grain of salt.

As for completeness, well, the Unix ecosystem has an enormous edge here, simply by having been there for decades and amassing tools and utilities. Since PowerShell was intended for system administrators you can expect nearly everything needed there to have PowerShell-native support. This includes files, processes, services, event logs, active directory, and various other things I know little to nothing about. Get-Command -Verb Get gives you a list of things that are supported directly that way. It seems like even configuration things like network, disks and other such things are supported by now. At Microsoft there's a rule, I think, that every new configuration GUI in Windows Server has to be built on PowerShell. Which means, everything you can do in the GUI, you can do in PowerShell, and I think you can in some cases even access the script to do the changes you just made in the GUI – e.g. for doing the same change on a few hundred machines at once, or whatever.

Of course, you can just work with any collection of .NET objects by virtue of the common cmdlets working with objects (gcm -noun object). For me, whenever there is no native support, .NET is often a good escape hatch, that in many cases isn't terribly inconvenient to use. You also have more control over what exactly happens at that level, because you're one abstraction level lower. As a last resort, it's still a shell. It can run any program and get its output. Output from native programs is returned as string[], line by line, and in many cases that's not worse than with cmd or any Unix shell.

_____

¹ Keep in mind, the file system is just one provider and there are others, e.g. registry, cert store, functions, variables, aliases, environment variables that work with exactly the same commands. That's why ls is an alias for Get-ChildItem and there is no Get-File, because those commands are agnostic of the underlying provider.

² So much for do one thing – but understandable, because ls' output is not rich enough to filter for certain things further down in the pipeline.

[aidos]

Oh yeah, I can see how the different filtering spots would make a difference.

Was just a bit surprised at the overhead of adding a command to the pipeline. A similar setup on linux would be like the following I guess (on a folder with 4600 files, 170 matching files).

    20ms time ls *.pdf
    35ms time find -maxdepth 1 -iname '*.pdf'
    60ms time ls | egrep '.pdf$'

I was more wondering if adding each additional command added so much overhead. Your numbers looks much more reasonable. Maybe the article I read had a big step due to filesystem / caching or something.