[graeham]

"I'm AI but humans help train me"

The implication to me is that the chat is with a human, who is using an AI tool with the intention of training that tool. What better way to train a new service than to launch it, then answer all the weird, unexpected questions with humans? Gradually more of the questions get answered, the AI gets better trained, and the human-AI becomes increasing more AI.

Further, as the AI gets better, the human working with it has to do less, so they can roll out the service to more users without requiring more staff. Perhaps eventually, no human is needed.

[PaulHoule]

Any good chatterbot has a script or set of scripts to answer questions about its nature, relationship to its creator, etc.

The presence of a delay does not mean there is no A.I. there. Not everything is as fast as Google search, for instance IBM Watson would think about a problem for a few seconds, which is fine because it only needs to be about as fast as a human.

[abritishguy]

The AI can assist the human as well. It may not be confident that its response is accurate and the human just clicks send.

[PaulHoule]

Great point -- if a system has a good probability estimator it can send questionable answers for review to raise accuracy up to the commercially useful level.

[jhpriestley]

AIs are already trained on huge corpuses of natural human communication, but they still aren't able to reliably analyze human language. Adding a few more gigabytes of chat logs isn't going to do much.

[hodwik]

They weren't trained to read natural language, and then respond with some programmatic action filtered into natural language:

  Bot reads "What's the temperature like near me?"
  Person calls "$get user-local-temperature"
  API responds "{temperature:{f:77},{c:99}}"
  Human writes "It's 77 degrees outside!"

Training set now contains that relationship between that question, that API call, that response, and that natural language response (and probably the users location, age, gender, and so on, all captured in the meta-data about the response in the corpus).

  Bot reads "What's it like outside?"
  Person calls "$get user-local-weather"
  API responds "{weather:{now:Sunny},{today:Cold}}"
  Human writes "It's sunny now, but will be cold later today."

And so on. I think the goal here is training on standard API calls as the response, and taking their data return and converting it into grammatical sentences. It's a two step training process. Know which API to call, and know how to convert API response to natural language.

There's no serious corpus yet for that -- if this is real, it is important work.

[lhc-]

Facebook's M is powered by Wit.ai (who they acquired a while back), which basically does what you're talking about. They translate natural language into machine readable data structures. Honestly, its interesting tech and being able to feed in facebook level data volume should prove very good for them.

[assholesRppl2]

thank you for being one of the few people who actually knows what's going on underneath. Wit.ai's software computes meaning via statistical modeling of semantic roles as they relate to "intent". It loosely resembles the linguistic theory of Frame Semantics (Fillmore), embodied today in the FrameNet project at ICSI in Berkeley.

[hodwik]

Thanks! Was just guessing, glad I wasn't too far off the mark.

[jhpriestley]

The translation of unrestricted human language into a restricted, "logical" language which could be understood mechanically, is another long-standing problem - going back as far as Liebniz - which is unlikely to be solved by a few gigabytes of chat logs.

[hodwik]

This is not a problem of structural semantics, each phrase is itself a semantic atom -- think Zork.

It doesn't need to see "What|WP 's|VBZ it|PRP like|IN outside|IN ?|."

Just "What's it like outside?", and know that it will always respond to that with the same call.

[hodwik]

(Can't edit.)

Warning: This is mere conjecture.

[SilasX]

All true, but it doesn't need to solve that bigger problem to have succeeded at building a reliable M: it just needs a high success rate at converting requests into desired responses, not translating everything into some master language for arbitrary use.

[gohrt]

Are you claiming that chat logs won't contribute at all? Or are you claiming that progress is irrelevant unless someone launches a 100% solution all at once?

[tonylucas]

I think the chat logs will help, but it won't just be about that data.

The data around how users interact will also be important:

Do they prefer a back and forth conversation, or do they want to say everything in one go.

Do they want to start a conversation, drop it, come back to it several hours later, or do they like completing it in one go.

How do they handle switching back and forth between different contexts, if certain requests take time, or do users not switch context.

What data are they happy to share, and what are they not.

What are the typical response times that a user considers acceptable 5 seconds, 1 minute, 5 minutes, 60 minutes? Does it vary depending on scenario.

Is there particular services or information that there is a trend towards, for example local search requests, research/information, particular types of purchase etc.

We've just spent 6 months going through a very similar process to this, which has helped drive the development of our Converse platform, which allows people to build semi or fully automated conversational messaging services, so this is fairly closely related.

(From our point of view, the NLP data we gathered was useful, but it wasn't the most important part)

[giancarlostoro]

Yeah I feel like some responses are pulled up, and then "M" gets assisted to pick the best answer, in the case an answer is not generated someone types one out. I would call it "Assisted AI" if anything, whereby the AI is monitored for accuracy. It makes sense to me, eventually in theory it would not need as much assistance, and even if it still would need assistance, it would still be one of the more reliable AI systems yet to some degree.