[antihipocrat]

I think the idea is that once built it would be a service that could parse a question, then automatically develop and run any query in response.

Sounds cool until it produces the wrong results.. then you'll need to hire an analyst to check every query just in case.

[skissane]

Put the requests in a queue. Have the bot generate the response. Then forward the response to a human analyst to double-check. A human can surely double-check a response much faster than they can produce one from scratch.

In many professions, it is common to have junior staff members do the grunt work, and then the more senior staff just review their work and either sign off on it, correct it, or send it back to be redone. You could use the same pattern here, replacing the junior staff with an AI, but keeping the senior one.

[kqr]

As if the analyst doesn't get the results wrong! For 1/50 of the price, maybe a few more errors are acceptable, even.

[desert_rue]

Which errors are you okay with?

[shinycode]

Yeah and whose responsibility is it when not catched in time and there this consequences / damage ?

[kqr]

The consequences would be accounted for up front and paid out of the savings from using GPT.

[shinycode]

Which price tag are you willing to put on a loved one’s life ? Some consequences of fully automated systems can go deep into human life cost.

[kqr]

The ones for which I would refer the question to GPT. We are still in control of which questions go to GPT/the intern analyst (less critical ones, where a fraction erroneous are okay) and which go to the resident expert analyst.

[gampleman]

Also it could possibly remove the (dreaded) on call aspect of it.

I think a lot of business owners would be relatively happy with automated instant answers, or get carefully considered answers in a week.

[stkni]

This is a good point. If the users know the difference the costs and benefits between using GPT and not using it then it certainly has value if those users are also willing to accept that not every answer needs to be 100% accurate.

In my experience business people often have a 'nose' for the right number and will bluff it out if the numbers are wrong and they're challenged.

Blue sky things or stuff you're putting in the annual report should be left to hoomans IMHO.

[guru4consulting]

If there are extensive test cases with static dataset, this may help with query modifications (optimize query, fine-tune, etc.) Of course, this may not feasible for new queries as you can't have test script until the query is ready.