[tokenizer]

You did accomplished the first point, which I agree with. Here's the link to the TED talk for those interested: http://www.ted.com/talks/lang/en/zimchallenge.html