[andrewflnr]

I wonder if you could get more musical sounds by considering frequency of pairs of letters as well, and tweaking the layout to put them on nice intervals like fifths, fourths and major thirds.

Edit: apparently "bigram" is the word I was looking for instead of "pair of letters".

[Adam13531]

Something I didn't mention in the video is that I had to tweak the positioning of certain letters due to common digrams in English. For example, I originally had "v" as a right-hand thumb key due to its frequency as a standalone letter, but some common English words have the digram "ev" ("ever", "every", "never"), and "e" was also a thumb key, so I remapped the "v" to my pinky to make those words much more smooth.

To do that same process for musicality would be challenging since your average English sentence contains so many different letters. I would probably do something like assign each letter a set of sounds, then pick one of those sounds based on the rest of the sentence.

[andrewflnr]

It's definitely not something you'd want to do manually. I'd want to come up with a representation of the layout that lets you run a genetic algorithm or similar on it, and score them on musicality when used to type some representative corpus of text. That's a much bigger research project, though (read: extra insane :D).

[Adam13531]

RE: "much bigger research" - fully agreed! Sometimes, I daydream and wonder what it would be like if it were my job to do something like this for, say, two whole months. Then I could try tackling issues like making this more musical. Then again, I know that if I had two months, I'd try to break the 80 WPM barrier.

In the end, I don't think there are many practical applications to something like this. I only had one real problem that could potentially be solved by this—it's when I'm working in a music program with my piano and limited physical desk space. The keyboard would be out of reach, so I could press an escape chord on the piano to enter Keyiano Mode, type whatever I needed, then reenter Piano Mode. The learning curve could be decreased by making on-screen overlays to show you what each key would do at any given time.

But yeah, that's solidly in daydream territory since I've got bigger fish to fry for now.

[rijoja]

I am doing exactly this. As a matter of fact I am working on something eerily similar. I took a year of just to muck about with my own project to see what happens. In particular the field of user input.

Seems like we're thinking very similar on this problem as my approach would be to have a constant onscreen HUD.

Initially my target was gamepads and other things viable for Virtual Reality. To try to demonstrate the universality of the method however I am working on a musical input method. Where musical keys can turn into keystrokes.

When you think about it you'll find that having a HUD onscreen gives a lot of room to implement clever algorithms that can speed up the process. Think huffman trees and other things that'll make the necessary keystrokes compressed.

What I have in the pipeline as of now is an FFT based algorithm that extracts the notes from the microphone, opening up for all instruments basically. The FFT part was surprisingly simple when dealing with one note at least, the tricky part might be chords and noisy environments...

What's so cool about far out ideas like this that you do for fun is that someone somewhere might find a usecase for it. Say for changing note cheats while playing a guitar or something similar.