[chubot]

Right, a real shell obviously can't use strtok. If you're leaving out pipes, redirects, and any control flow, then separating a shell string into words for the argv[] array is fairly similar to lexing a C-escaped string (e.g. in C, Java, Python, JavaScript).

You have backslashes, single quotes, and double quotes basically. Traditionally this is done with switch statement in a loop in C.

But that is not a good approach for a real shell. Even inside double quotes you can have a fully recursive program, like:

    $ echo "hi ${v1:-A${v2:-X${v3}Y}B}"
    hi AXYB

Once you have recursion then you need some kind of parser, not just a lexer.

[laumars]

I've been mocked on HN for saying this before but Bash and other shells of it's ilk are programming languages in their own right. I mean sure you're dependant on the suite of tools in $PATH to do anything useful, but that's not that much different to the standard libraries that make modern languages so powerful.

[Spakman]

I have have a hard time seeing what there is to mock about your opinion of shells. I absolutely consider them languages - better at some things, worse at others.