[szanz]

(Disclaimer: I'm a Haystack maintainer and I helped creating this demo)

I had to try it out the questions you asked, because your first seems totally answerable to me. And indeed I do get the right answer in the first position (60 million). Did you ask exactly the same question you posted?

For the second, unfortunately we included only country pages and capital city pages, so it's likely that the information about the population of Cambridge simply wasn't there.

In general though I agree this task is not perfect for a demo. It's hard to tell whether the model is wrong because it doesn't have enough info, or whether it does have the data but couldn't find it. The best way to evaluate it will always be to try it out on your own data :)

[timomo]

What is the population of Cambridge?

for me the demo returns that the model did not find an answer...

[anentropic]

ah, it seems to matter if you use upper/lower case

"What is the population of italy?" returns Rome still

"What is the population of cambridge?" returns Kingston, Jamaica, circa 1788

I guess I typed the questions nicer in my original comment above than I did in the input box for the demo :)