[hyp0]

  You have to be a dev to understand what your system is doing, but you don't have
  easy access to all the standard dev things.

Interesting, this mismatch may be a good description of the problem of visual languages.

Curious: what do you think is the minimal subset of unix tools to do this? i.e. instead of pretending the problem is simpler than it is, accept the complexity, but minimize it.

I'm thinking of a tool like "jq" (sed for json) for json data sources... but I don't think its raw-text manipulation is up to the task (and of course you need tools to monitor the feeds etc).

[TylerE]

Trying to manipulate structured data as text makes about as much sense as parsing XML with a regex.

http://stackoverflow.com/questions/1732348/regex-match-open-...

[shavenwarthog2]

Python :-) there are libraries specifically for parsing malformed html. I'm happy using Unix tools for scraping and parsing, but you run into a brick wall rather quickly. Python is more reliable, flexible, and easier to integrate.