[alanthonyc]

Not sure what your main goal is, but in my compilers project class, we used lexical analyzers to break out tokens from the input stream.

Try looking up "Lex" or "Flex"...these were the tools we used. There may be better ones around now.

Here's a quick google: http://dinosaur.compilertools.net/