Now I know where those annoying challenge pages come from. Was planning to add CloudFlare to my site but if it comes with those challenge pages, I'd rather pass.
i contacted them after seeing that page a couple of times and got a reply saying they had been having some false positives; i've not seen it since. so if you only saw them in the last week or two, and not in the last few days, then it may be something that has been fixed.
Challenge pages cannot be totally disabled. From CF's admin page:
Adjust your basic security level to modify CloudFlare's protection behavior. A low security setting will challenge only the most threatening visitors. A high security setting will challenge all visitors that have exhibited threatening behavior within the last 14 days. Essentially off will act only against the most grievous offenders. We recommend starting out at medium.