Really cool! Perhaps you could also take a look at what we are building at https://snips.ai (disclaimer: I'm a co-founder) to make a 100% on-device and private-by-design Voice AI which runs even on a RaspberryPi 3
it would be awesome to do this on a raspi, without requiring internet connection!
Great idea. I really like idea of not requiring an internet connection. Self contained projects are more reliable and portable. Also this project is aimed towards students so privacy is a real concern.
The way I built it should be easy to use another voice ai input. The Alexa module of the code just passes the commands to the Core logic. It's all built in node.js.
Is it possible to get started without creating an account on your website? I couldn't figure out how, but I figured there must be a way if it's "100% on-device".
Hi spacehome, you need to create an account in order to use the webconsole to build the assistant (create intents and queries), then once this is done you can use it 100% on-device!
You guys are are also incentivizing for app work done via the console / added to platform right? For a serious dev with experience in app deployment what could I look forward to?
Most of my background is in data security so this is definitely up my alley.
You do not create an account on the website. Right now the only way is to open the Alexa skill and it will give you a 4 digit code for your game. After you open your game on the website you can continue adding rules to your game via Alexa. New game logic will update in the game in real time.
I would like to make a version 100% on device. A few have suggest how.
This would be so awesome! I'm wondering how Snips.ai functions with no internet connection? I'm an early stage programmer with not much RaspPi experience unfortunately. Is it just a closed circuit for home automation. How does it fetch data for tasks that need external servers to update?
The creator/main developer is a friend of mine and is one of the smartest people I know. The project can be used to control IDA for reverse engineering, play video games, program, and more.
Thanks for the plug! Couple notes: Talon is free, but there are closed source bits, and this Alexa project is "voice coder", which is different than the paid and abandoned voicecode project.
The difference between this and Talon, is Talon can be used for real programming and uses very low latency offline recognition, while voice coder looks like a guided tutorial and the user experience is probably a fairly high latency single command call/response based on my previous Alexa experience. I'm not sure Alexa can recognize a streaming series of commands without large recognition pauses between each command.
Correct. Voice Coder is a guided tutorial for beginners, not a replacement for developing production code. It does allow for multi step answers, but is a high latency back and forth dialog with a smaller set of commands. The goal is to be extremely human friendly, not productive, to better understand the basics of coding.
Silvius isn't really state of the art. It's just newish. On their website it even says the project that inspired them has better performance. It references Aenea (which is a convoluted system of using Dragon on Linux in a VM) and calls out to Tavis Rudd's 5-year-old presentation and Dragonfly, none of which are state of the art.
Silvius is unique in being the only project I know of to use Kaldi, which is very cool because it's open-source and doesn't require a GPU, but the recognition accuracy of Silvius isn't great and the voice model is fixed and has no feedback from the grammar.
The game template supplies the available events/action/value. The Game Creator guides you through the available options supplied by the template. It then tries to use a human speech like interface to receive the input, the game logic. Currently there is only one game template. Each new game template would need to supply it's own possible events/actions/values to suit it's needs.
This game template was design for students with zero experience. So the create new objects command was omitted in the user defined rules. The game itself controls the box placement for this game. The goal of that game was to learn about events and actions.
I plan to add more game templates with the create object command for more advanced users. You can set it's properties, game physic values and so on. The game itself will need to implement and execute that command. The game template just tells the Game Creator what that current game supports. So the magic happens in the game code. Hit view source on the game if want to see it.
The project was built in a 2 weeks; server, Alexa integration, mini-game, and game creator logic. If the project gets any traction, I'd love to keep working on it.
Mostly yes. The difference is rather than changing values, you are creating event handlers, actions, and their values. You can only trigger actions on events the game already has and execute actions the game supports which is defined in the game template. With that said, there is no reason why it couldn't support all the events and actions from the core game engine it uses. For this release the focus was for beginners and the options were minimized.
I tested your code and it works for me. Make sure you type 0IRK and nothing else like spaces or periods. or click this link https://voicecoder.net/game.html?id=0IRK
The project is designed to help students to understand the basic of coding logic using their voice. Think of it more like a coding lesson for beginners. Many at this level use drag and drop coding. Code.org has many drag and drop coding lessons. The difference here is it uses voice. The recordings are not part of the code produced, just plain text. This version cannot make production like code, just add game logic to an already created game template.
I have personally been playing with Snips. It has been an exciting project to work with. I do need some help with certain things, but reddit has been a good source of information.
Hi i'm not quite skilled in coding but is the barrier of entry high? Would someone like me with not a lot of knowledge be able to pick this up and start programming?
Nothing in life is impossible. But unless you have some sort of physical disability that prevents you from using a keyboard and monitor, you are much better off learning programming using standard tools that have tons of noob-friendly documentation, rather than something experimental like this.
I guess things have moved on from this Perl-Vista nightmare, I was personally looking forward to having all the coders around me screaming at their desktops in a fit of frustration and rage.
it would be awesome to do this on a raspi, without requiring internet connection!