Show HN: I made a body-interactive music app

[maweibezahn]

I made this iPhone app called "Affine Tuning" which is a collection of body-interactive musical pieces: https://affinetuning.com/ It uses body tracking (Motion Capture from RealityKit) to shape dynamic compositions. The app is completely free and will be updated in the future with more songs and other interactive features.

[wintermutestwin]

Bravo! I have waited for something like this for a long time.

Back in the early 90's I saw this interesting band called D'Cuckoo at Stanford that used a variety of midi triggers to redefine instruments relative to performed movement. https://en.wikipedia.org/wiki/D%27Cuckoo Now this can be done with AR.

Your implementation is very tight. I can see a lot of potential here. I'd throw money at in app purchases of other setups and even better would be the ability to load in my own samples and customize the associated movements, etc.

Is this a side project or are you planning on monetizing?

Anyone know of other AR music generation tools like this? I tried some searches, but didn't come up with anything.

[maweibezahn]

Thank you! I did not know of D'Cuckoo, so thanks for the pointer.

During my research I found a lot of projects that connect body-movement and interactive music which could usually be put in one of these categories: 1. research projects that never left the university 2. art installations or performances - So both were tied to a specific time and place. My goal was to create something mobile, bringing this kind of experience into the home, or wherever.

Right now this is a non-commercial product, like a test balloon. I can imagine to collaborate with musicians and offer their creations as IAP in the future.

About bringing in your own samples etc.: I have started to work on an editor (native Mac app) with which people could create their own interactive compositions. They could then import them into the app and ideally even share them with others. I hope to have a first version out in a couple of months; luckily I got a tiny bit of funding for that project.

[wintermutestwin]

Yeah, I have seen a couple of #1 and #2. There is so much potential here. Congrats on bringing out a #3 version.

>I have started to work on an editor

Godspeed on the editor! I could see joint collabs with musicians and dancers creating an essentially net new art form.

There is a lot of potential here.

[maweibezahn]

The only other AR "musical" tool I know is "here hear": https://apps.apple.com/us/app/here-hear/id1249413969?ls=1

[gregsadetsky]

Congrats on the launch!

A few ideas at random:

- could be neat to have the ability to send OSC (i.e. send the same values that you're using to generate the music i.e. limb position / angle / velocity, etc.) so that computer-based synths, video softwares, Max MSP, etc. could ingest it in real time as well

- it would be fun to have different synth "presets" or "patches" which connect limb movement to different parts of the audio generator -- i.e. connect right arm with LFO rather than the oscillator pitch for instance

- just curious, does it support multiple dancers?

[maweibezahn]

Thanks! It actually launched some months ago but this is the version I feel confident about to share.

on OSC: I already have implemented this feature in my development version. It would just send the raw body tracking data (joint 3d location). I was not sure though if it would be used by anyone and where to put it, so I did not ship it.

About the presets and general interactive/creative features, please read my answer to wintermutestwin, I am working on it.

It does not support multiple dancers because the "Motion Capture" feature (which is proprietary) does not support it… :/

[iseanstevens]

I’d love OSC for use with Max to control some of my interactive art. Currently using Leap Motion but previously Kinect (too high latency), Wii-mote (great till it gets lost/walked off with) etc.

[maweibezahn]

Hm, I have used the Kinect (2) in the past and thought the latency was quite good actually. If that is too slow for you the iOS body tracking might be as well, I find it has even an even bigger delay than the Kinect. It is also less precise because it relies completely on 2d camera input and a machine learning model. Now I am thinking of putting the OSC feature back in for a future update :)

[wintermutestwin]

I am imagining being able to track a guitar's headstock and sending the OSC to act as an expression controller for things like vibrato, etc.

[masaav]

Very cool, congratulations!

[mashaole]

Damn I can’t use it on My iPhone 8 Plus it’s not compatible

[aymeric]

Great idea!

[maweibezahn]

Thanks!