[ActVen]

Opus just crushed Gemini Pro and GPT4 on a pretty complex question I have asked all of them, including Claude 2. It involved taking a 43 page life insurance investment pdf and identifying various figures in it. No other model has gotten close. Except for Claude 3 sonnet, which just missed one question.

[zooq_ai]

Did you compare it with Gemini Pro 1.5 with 1 million context window? (Ideal for 43 pg pdfs)

I have access to it and I can test it against Pro 1.5

[technics256]

I am curious on this. can you share more?

[ActVen]

Here is the list of the questions. https://imgur.com/a/D4xwczU The PDF can't be shared. But, it looks something like the one here: https://content.naic.org/sites/default/files/call_materials/...

[spaceman_2020]

I tried Sonnet with a question about GANs and it seemed pretty good, better than GPT-3.5

[uptownfunk]

Really? I tried the sonnet and it just was not very good.