[Adam13531]

Hey, I'm the guy who made this! This all started from a random idea I had, but then I was tickled by the thought of becoming faster than the average typist on a piano. It all culminated with a livestream today (https://www.twitch.tv/videos/404585365) where I did typing tests for about 3 hours.

I'll try to address the comments on here, so ask any questions you have!

[etaioinshrdlu]

Do not be afraid to release bad source code. It's fine.

[rijoja]

You beat me to it! I've been working on similar thing, so I'm a bit jealous that you published this before me! Then again it makes me happy that I'm not the only one with far out ideas like this!

My approach is a tad different though. My approach works by using the microphone. And I have the code to extract notes from a microphone via a FFT. Allowing for typing with say a guitar.

Extracting single notes works quite well. I haven't gotten around to test chords just yet, but I think I have the fundamentals there.

I'm a bit short on musical knowledge and instruments so so far I've only tested it with my voice (ugh), synthesized sounds and partially filled water bottles.

Also I have a suggestion for a graphical interface that would make the learning curve easier and making it language/alphabet independent.

Please reply to this comment if you are interested in integrating this.

[jackxbritton]

Been doing some pitch detection recently myself, but I went for an average magnitude difference function over an FFT. My understanding is that FFTs are a bit more intensive than autocorrelation-like solutions, but they work better with polytonal signals. Typing seems suited to single-tone instruments - did you consider using AMDF (or a similar function)?

[rijoja]

To be honest all of this is on the border of my knowledge horizon. No I've never heard of AMDF but I will check it out for sure.

In the long run as I wrote I am aiming to support chords so that'll be necessery.

Would AMDF make the response time quicker?

For what purpose are you doing pitch detection?

[Adam13531]

Sounds like a fun project!

RE: integration with what I did - I'm hoping to release the source code one day, then you'd most likely be free to use that code however you want (I'll probably MIT-license it). But as for my involvement, I'd like to be done with this project.

[brownbat]

Love the details for completeness, like modes, and your discussion of adjustments you found necessary along the way. Great job on the project and great explainer video too.

Slight tangent--I've often wondered if the principles behind audio keyloggers, where you record typing sounds and try to reconstruct what was typed, could be used to make a wireless computer keyboard that never needs batteries. Seems like you're attacking an inverted version of that problem.

Interesting stuff, thanks!

[rijoja]

That's a really cool angle. As a matter of fact I'm working on a system like this only that it uses FFT to get the notes instead of midi.

[Insanity]

This was really cool to watch! The youtube video offered a nice explanation and I actually enjoyed the audio/visual-feedback from watching you type on it.

Kudos!

[mindcrash]

If you love these kind of things you will be blown away by this: https://codeklavier.space/

Which can be seen in action here: https://www.youtube.com/watch?v=HrUNrb3ihl0

And no, no April Fools here ;-)

[Feuilles_Mortes]

It might be fun/interesting to pipe pieces transcribed to MIDI into your system and see what falls out. I'm sure it'll be nonsense, but perhaps fun nonetheless. (e.g. http://www.piano-midi.de/chopin.htm)

[Adam13531]

I made a little explanatory document (https://goo.gl/Vq3Fsb) to go along with the typing tests that I streamed today, and that was one of the things that I mentioned (copy/pasting from there):

Q: Can you play a real song so that we can see what it types out?

A: This isn't as satisfying as you may think since the entire typing range is only about two octaves, and most songs exceed that range, so nothing actually gets typed when you play those notes. Anyway, here's Wonderwall: Iuiuigaezobzgbxgebiuiuigaezobzgbegz

(that's actually the right hand of Für Elise)

[Rychard]

> Anyway, here's Wonderwall

There are many things that I would like to say to you, but I don't know how.

[coldtea]

Write them to him using a piano keyboard

[rhizome]

Gah, I tried to get here before someone suggested rendering Moby Dick.

[kian]

What were the key or note chording patterns you used to make this?

[Adam13531]

I'm not totally sure I understand your question. My original design document (from apparently a year ago!) is here: https://goo.gl/U1HnMu ← that was part of what I produced in the "six hours" that I mention in the video. Perhaps that will help, but I don't expect you to read through 11 pages to find your answer, so with clarification, I'd be happy to answer you!

[kian]

Sorry i phrased that poorly - what was the mapping of piano chords to characters you settled on?

[kian]

I went through and read your design document - thanks for the link! That answered my question. For anyone else wondering, one piano key per character within a given modifier palette (like letter, number, etc). With 24 major and minor chords, 4 times that in minor and major sevenths, another 12 dominant chords, 12 augmented, and 12*11 interval pairs, there is a very large space to map interface structures to. The trick is making it harmonically pleasing - however, with a graph of harmonic transitions and cadences, it ought to be possible to find isomorphisms that preserve pleasing sound if you matched unikely letter combinations to unpleasing sounds. Perhaps in the same way that i9en used to infer what you had typed based on probability and scrolling, perhaps we could do something similar with arpeggios or chords played within a certain span.

[Adam13531]

Ah, good, I'm glad you found the answer!

As for making something like this harmonically pleasing, it's been touched on here in other threads, but I think the whole project would require a redesign. If you were still aiming for a one-to-one mapping of piano keys to letters, then I think you would end up sacrificing both learning speed and typing speed.

[yomly]

My friend is a professional pianist and wants to see how high he can get his WPM on typing - is the source available?

[Adam13531]

Unfortunately no, not yet, although I'll try to make a post if I ever open-source it!