It answered "How many frogs does a horse have?" correctly, with perfect reasoning. No model I've tested has ever answered that correctly without 3-4 hints.
"A horse has one frog per hoof. The frog is a part of the horse's hoof, located on the underside, which acts like a shock absorber for the hoof, assisting with circulation and traction. Since horses have four hooves, they typically have four frogs, one in each hoof."
GPT4 seems to handle this fine, unless I'm confused?
GPT-4-turbo-2024-24-09 (temperature = 0.7) just told me a horse had one “frog” per hoof and went on to clarify that a frog does not refer to the amphibian but to a part of the horse’s hoof.
Gemini Pro (the current web chat version) gave a similar answer, either no frogs or four depending on the intended meaning, and showed a photo of a hoof. All 3 drafts agreed on this.
Other models I have tried said a horse had no frogs. That includes gemini-1.5-pro-api-0409-preview as provided by the Chatbot Arena (temperature = 0.7, 2 tries).
I mean, i just tested this prompted, and have no clue what side is correct lol?
How many frogs does a horse have?
Oh it appears
Model B: gpt2-chatbot
is correct?
A horse has one frog in each of its hooves, making a total of four frogs. The frog is a part of the horse's hoof, located on the underside. It is a V-shaped structure that helps to absorb shock, provides traction, and aids in the circulation of blood through the hoof. The health and shape of the frog are important for the overall well-being of the horse's feet.
GPT4 seems to handle this fine, unless I'm confused?