Hacker News new | ask | show | jobs
by dazzeloid 1116 days ago
If we're doing our job right then HH should only be answering based on its source content (and not background knowledge). So bad answers coming from incorrect source content would need to be corrected in the source content. Citations should help with this, e.g. as a human admit, review answer -> notice it's bad -> click citation -> locate incorrect part -> change it.

Also thinking about ways to ensure the bot answers common questions correctly while still being able to personalize responses. Working on something called "answer shaping" where an admin can write out a response and tag with the question it responds to. Then the bot would first check to see if the human question matches a cached question, and if so would prioritize using info in the cached answer in its response. Seems like this can give the bot freedom to personalize the answer but make sure it includes the right stuff.
1 comments
jonahx 1116 days ago
> e.g. as a human admit, review answer -> notice it's bad -> click citation -> locate incorrect part -> change it.

I understand why you want the flow to work this way (single source of truth, fix things at the root, simplifies everything), but, respectfully, it is really bad from a UX perspective. Here are the main reasons:

1. Not all admins will have the ability to edit all source pages. Both from a permissions perspective (eg, a zendesk ticket or slack message created by someone else) or a technical ability perspective (eg, you need to edit html and create a PR).

2. People are busy and lazy. If I can see the problem in the answer, notice it's wrong, and correct it right now on the page where I see it, I will. Otherwise, I often won't. Think busy CS agents, developers in the midst of problem solving, etc.

Yes, supporting this workflow makes life harder on you, because it's technically more complex, but it's the way people will want to use this product.

link
dazzeloid 1116 days ago
I guess my concern is being a system of adjusted record on top of a system of record. Wouldn't it be a cluster if changes are being made to docs in CommandBar but not [Zendesk]?

That said maybe there's room to store bot-specific stuff in CB. For example, tagging passages with "exclude this from training data" if they're causing bad answers for some reason.

link
jonahx 1116 days ago
Indeed, it will be a total cluster.

From a pure engineering perspective, it is obviously the wrong solution, and your initial suggestion is the right one.

Yet....

I strongly predict that the forces of market demand and human behavior will push your product inexorably in this direction.

link