IMHO SOUL is a game changer in audio just like OpenGL/Direct3D/etc. was a game changer in the graphics world.
Think of all of the graphics technologies that were enabled by OpenGL/Direct3D/... and consider the equivalent for sound/music.
The trend is clearly going into the direction of application-specific processors and a lot of hardware already has dedicated DSP chips for audio. It is high time that a standard language is proposed to access them in a unified way just like OpenGL did.
Full disclosure: I worked with Jules and Cesare at ROLI/JUCE where Jules was my mentor and colleague. If anyone can pull this off, it’s them.
> Our intention is to make SOUL entirely free and unencumbered for developers to use. All our public source code is permissively (ISC) licensed. We’re currently keeping some of our secret sauce closed-source, but the EULA allows use of it freely to encourage its adoption in 3rd party hardware and software. Ultimately, we plan to commercialise SOUL by licensing back-end drivers and other IP for use by vendors who are building SOUL-compatible hardware products.
How would it be possible to allay your concerns? I think we've tried to be as clear as possible with our intentions - how could we rule out such possibilities with our license agreement?
Sadly, I’ve been bitten by this before, too, and even if I love the product, I stay away.
It isn’t that I don’t trust you or your intentions now, it’s that intentions (and requirements) change.
You asked about allaying fears. Personally for me this means actually take-to-the-bank licensing that I could get through a legal audit, even if hypothetical.
Talk of “secret sauce” and future closed source and IP licensing of future maybe-must-have magic... that’s way too much future risk for me.
An example of what I think is good licensing is JUCE (https://juce.com). Everything from “free” to “paid with support”, all open source… but the point is I always know exactly where I stand legally, what I can and cannot do... and what I can and cannot do in the FUTURE.
Well, as the person who also created JUCE and its license, I can't really argue with that. :)
We're very keen to make sure that the end-user developers are totally unencumbered and license-free, because we want lots of people to adopt it.
All our licensing is more aimed at device and driver builders - think of it like e.g. openGL - the industry making GL cards and drivers is legal complex, but none of that stuff affects the coders writing GL apps.
Disclaimer: *I'm not your target audience*; I don't plan to use your product even if you made it public domain.
I never want to agree to an EULA before starting to develop something outside of a professional setting. That's an immediate dealbreaker for me. But even if I did - I wouldn't be able to disassemble or reverse engineer your software (2.2)? Also - why are there so many clauses in the EULA telling me to obey really specific laws? Aren't these redundant to section 2?
- end-user developers. This is kind of like being a user of any other language, there's no EULA, nothing to sign, you just write SOUL code, test and debug it with whatever tools (which may themselves have a EULA, but probably nothing heavy)
- device and host developers: These are the people writing DAWs, plugins, hardware that can run SOUL code, audio device drivers etc. These are professionals, and licensing is a normal part of life when you're doing this kind of work.
If there's no mechanism to bind those intentions to the intellectual property, what you're building is just another time bomb which will eventually detonate at the expense of your users.
Real answer: Mainly the JIT engine, and the very complex rewriting algorithms that turn multi-threaded soul code into a form that can actually be executed.
We're certainly keen to open-source everything when possible, just been advised by our lawyers not to do that yet. C'est la vie.
- using Faust/SOUL on Bela and having SOUL as the intermediate language to JIT compile Faust code is certainly possible, but not the easiest way ! Bela developer Giulio experimented a more direct Faust JIT support on Bela (since Faust can directly generate LLVM IR code and JIT it) here: https://github.com/giuliomoro/bela-faust-jit, but this projet is a bit frozen by lack of time to improve it. Feel free to bring it to life again.
"Faust (Functional Audio Stream) is a functional programming language for sound synthesis and audio processing with a strong focus on the design of synthesizers, musical instruments, audio effects, etc. Faust targets high-performance signal processing applications and audio plug-ins for a variety of platforms and standards."
"The core component of Faust is its compiler. It allows to "translate" any Faust digital signal processing (DSP) specification to a wide range of non-domain specific languages such as C++, C, JAVA, JavaScript, LLVM bit code, WebAssembly, etc. In this regard, Faust can be seen as an alternative to C++ but is much simpler and intuitive to learn."
I'm not affiliated with either but I've used Faust a lot. I believe SOUL intends to be directly supported by various platforms, whereas Faust compiles to whatever kind of code runs on different platforms. So if you like Faust, you might end up compiling Faust to SOUL for some platforms.
Faust also has a more advanced syntax. SOUL basically looks like C, while Faust is more functional and designed from the ground up to represent graphs, as such it can look pretty alien. In particular, partial application in Faust is great for graphs but it also means there are 100 ways of writing a program.
For those interested in audio-specific languages, I've had some success with Vult, a transpiled language which runs everywhere, compiles to Pure Data and runs on the Teensy. I used it to make some really powerful, extremely performant filters.
I'm definitely not the target audience for this, but I can't help wonder: Why introduce a new turing-complete language, instead of building on top of an existing language suitable for "gluing things together"? Python and Lua come to my mind.
The use-case for SOUL is about as far from a "glue" language as you can get!
It's a language for writing the absolute lowest-level, close-to-the-metal, bit-twiddling realtime code which you'd then glue together using a higher-level language like C++, javascript, python, lua, etc.
* Can I use SOUL to create freestanding interactive performance environments a la Pure Data?
* What prospects are there for beefing up hardware to run more DSP and bring down latencies? Are there, or will there be open-hardware DSP projects which run SOUL, or open-source runtimes which achieve high performance on standard issue CPUs? If the company behind SOUL goes under (Roli, I think), what happens?
* Could I write a program in SOUL that does real time beat detection given an audio signal, say a steady drum beat (to make things simple)?
> Can I use SOUL to create freestanding interactive performance environments a la Pure Data?
SOUL does have a JIT compiler and you can live-code it, and you can certainly use it to write programs that generate musical patterns etc.. So sure, the language could be used like that.
Our background and focus has always been more from the pro-audio side of things rather than performance, though, so the tools we've built so far aren't really targeted at performance use-cases, it's more about development of apps and plugins.
> What prospects are there for beefing up hardware to run more DSP and bring down latencies? Are there, or will there be open-hardware DSP projects which run SOUL, or open-source runtimes which achieve high performance on standard issue CPUs? If the company behind SOUL goes under (Roli, I think), what happens?
Right now we've not had the resources to get stuck into that side of things deeply yet - hoping to be able to do more on it this year. (And ROLI aren't going to go under, ha!)
> Could I write a program in SOUL that does real time beat detection given an audio signal, say a steady drum beat (to make things simple)?
Yep, totally the kind of thing we're expecting people to do with it.
What prospects are there for beefing up hardware to run more DSP and bring down latencies?
Well that's the million dollar question. We're imagining things moving (hopefully) along similar lines to how GPUs have developed, with dedicated processors for audio processing. It's pretty clear though that the current model of distributing binaries for the CPU is not magically going to enable hardware vendors to sell audio accelerator cards, and hence why we feel a change of direction for the audio industry is required.
The way we see this being enabled would be a JIT based approach where the audio driver translates device independent code to run on their given hardware (and the 'soft' rendering on the CPU which is where we currently are for machines without dedicated audio accelerators).
The design of SOUL has this sort of support in mind, with per sample processing and the parallel structure of the DSP still visible within the language, which allows the driver to make threading decisions at the JIT stage, enabling parallelism.
> Are there, or will there be open-hardware DSP projects which run SOUL, or open-source runtimes which achieve high performance on standard issue CPUs?
The binary release includes a build of the soul command for Bela, which is a very cool low latency board + OS for audio (https://bela.io/) based on xenomai. It's probably as close as we get to open hardware for DSP, and it puts you in the sub millisecond latency range.
Does it support matrix multiplication, matrix inversion, SVD, etc.? These are very important operations for array processing. I cant seem to find them in SOUL's documents.
No, the language does not currently cover matrices. Our primitives include int, float and complex in 32 and 64 bit sizes, and we also have vectors of these types. We have spoken about extending this to matrices, but it's not in the current language.
It would be very nice to include this, as it would open up some useful patterns for analog modelling and ML runtimes.
I have a few questions, I hope are not stupid. I'll confess to not having delved into the heart of SOUL source yet.
1) what are these "audio processors" that computers already have embedded in them? (DSP cores and associated extensions or an MCU controlling the audio-HW-codec state machine and providing a buffer for the computer or?)
2) Does SOUL engage in the audio sub-system hardware in any way or perform any firmware like activities? DMA engine configuration typically driving the audio engine I/O, right?
3) Do external audio DSP hardware like UAD provide, for example, pipe audio out and then back into the computer, typically? Could such hardware be addressed if it weren't proprietary?
Full support for wide streams (i.e. streams of overlapping windowed blocks, suitable for frequency-domain work) is still to-do (though we've planned for it all along!)
We've tried to make embedding it as easy as possible - there's a DLL, and a simple COM interface and a few header-only C++ classes to load and JIT-compile a patch dynamically. We have an example project showing how to do this. FWIW Tracktion Waveform is doing exactly this, and only has a few hundred lines of glue code to enable patches in a full-blown DAW.
Good to hear! I'll stay in C/C++ until its ready then - I look forward to not needing to prototype in one language and implement in another, but for now the biggest motivator is how easy it is to do spectral/cepstral things in MATLAB (bit of a pain to write windowing code in C++, hard to test, and easy to get wrong).
Why COM (and is it actually COM, or is it COM-inspired like VST3)? It's a bit of a pain to interface with it from hosting languages (C is much more straightforward, particularly when worrying about C++ exception safety and garbage collection).
Why COM? Well, I hate COM in general, but it was a much better fit for this particular API than flat C functions. It comes with a bunch of C++ helper classes to make it easy to work with, and the resulting code actually ends up looking OK.
It seems odd to spend a bunch of time working on a C++ alternative for audio DSP but to rely on one of the most annoying C++ paradigms to deal with to embed it in anything useful. I don't really care how the library code looks (if the secret sauce is proprietary and closed source anyway, that presents zero value to me as a potential licensing customer!), I care how it functions.
I want to script together many soul patches into a complex systems written in higher level languages. Integrating C++ dependencies is avoided because of the headache of FFI, C is quite straightforward. I suspect users like myself will need to wrap your COM API in its C-equivalent through manually written vtables regardless, which is its own minefield rather than a proper API.
COM is relatively common and accepted in audio (VST3 / AAX).
Usually it doesn't require the COM runtime from Win32, it's just an ABI for being able to virtually call and delete shared objects.
I'm certainly not at a level to be able to use this kind of thing fully, and I've had several attempts in the past to try to learn to create audio plugins/instruments (as audio is my main domain, I'm a dabbling programmer), but this looks like it could help get me where I want to go.
I have a (probably dumb) question. I've looked through the docs, and I can't see how you would play a sample back (i.e. to create a simple sampler, which is one of the projects I'm most interested in creating). Am I missing something, or is that something that wouldn't be present at that level, and you'd need to roll your own?
EDIT: I've just read the code for the simple piano, and I see that it implements the sample playback itself, so I guess that answers my own question!
Current tools for audio coding are complex and non-portable
Thank you for addressing this. I was meaning to get into making VST plugins of my own, but currently in order to do that one has to jump quite a few hoops and often as the work grows in complexity so does the required expertise.
Well that's definitely something we've been wanting to help with.
One of the supported features of the soul system is to export a JUCE project from a soulpatch - it cross compiles to a C++ project supporting all the standard plugin formats, so you can get AU support for OSX, and VST3 support on Windows out of that one project.
JUCE is one of Jules' previous projects and is pretty much the standard for building cross platform audio plugins used by commercial plugin vendors (https://juce.com/)
Thank you for all the effort, you and your team do inspiring work. After building a pretty big C++ audio app, I think it's time to make a change, and try something a bit kinder (SOUL).
I would also have appreciated a code example above the fold, even on the press release. When I read about a new language, the first thing I want to see is what the code looks like.
Yes - the command-line tool will take a SOUL patch and emit a full, ready-to-build JUCE project that will create all the plugin formats (and a standalone app).
We've had it running on Bela for ages now. This year we'll be releasing the SOUL network protocol, so that we can properly control remote "SOUL venues" on hardware like Bela (our current Bela network is a bit of a bodge to get it running).
Faust has also had SOUL support for a while. Not sure what's new over there but maybe Stephane will see this and comment :)
Nice, it's great to see this project go from C++ rant to reality. Can't wait to never write C++ ever again :)
On licensing, how will it work for creators of open source hardware? Will they still need to pay a licensing fee if they are targeting a specific device with closed driver IP? Or could there be an exception there?
Do you also see yourselves supporting open low-level hardware (RISC-V, FPGA, ...?), and in that case would you consider opening your driver IP for those targets?
Right now, we haven't got an official position on how we'll handle the back-end licensing, but it's a sensible assumption is that we'll have an open-source reference implementation that people can use, and then our own optimised or custom versions that we can license.
RISC-V is an easy one to support, as our JIT uses LLVM so it's just a case of adding a build flag for that.
FPGA is more interesting. I think there could be some really clever tricks we can do for soul programs which are highly parallel, but that kind of rocket-science stuff is probably going to have to wait until we have a couple of PhD students to work on it :)
Hi there, the faust binaries include a faust2soul command which from faust dsp generates a soulpatch. We include a bela build of the soul command line tool (it's basically a tech demonstrator of the JIT and a backend). So at this point, the bits are all out there to take a faust.dsp file and JIT it on Bela.
Yes, the soul command when playing a patch watches for updates and recompiles when changes are found (then it switches across from the current DSP to the new one when compilation is complete). This isn't totally seamless, as although parameters are retained, any state held within the DSP code (say delay lines) will be reset to initial conditions, but it's pretty good.
So then, update your faust code, re-run faust2soul and the soul command will see the updated patch and recompile.
Even though this already sounds great, hot swapping state would be the really big deal here. Do you think the Faust program itself might be able to save/load state somehow?
faust isn't in the loop at that stage. Saving/restoring state will be something the soul driver will have to do, and of course, this will then mean this ability works for all code using the driver, not just faust sourced soulpatches.
We've sketched out how this would work, and it's on the project backlog as something we want to support, so at some point we'll improve this area of the runtime.
Interesting! I wonder if the patch format has support for OSC messages for external interactions - so far I only found MIDI support, but maybe I missed something...
The patch format is for the same use-case as VST, AudioUnit etc, where OSC isn't really a thing.
OSC is more something you'd want to use for a stand-alone program, which needn't be written as a patch. But yes, would be nice to add some OSC wrappers one day. Maybe it's the kind of thing that someone else will contribute before we get around to it.
EDIT: ah, I misunderstood you.. You mean you'd found the examples but the playground page didn't fit on a phone. Yeah, it's tricky to squeeze all the stuff needed into a small space! We're planning to do a huge redesign of the site in 2021 to turn it into a much more powerful developer portal, and we'll certainly look at other form-factors in doing that.
Think of all of the graphics technologies that were enabled by OpenGL/Direct3D/... and consider the equivalent for sound/music.
The trend is clearly going into the direction of application-specific processors and a lot of hardware already has dedicated DSP chips for audio. It is high time that a standard language is proposed to access them in a unified way just like OpenGL did.
Full disclosure: I worked with Jules and Cesare at ROLI/JUCE where Jules was my mentor and colleague. If anyone can pull this off, it’s them.