Hacker News new | ask | show | jobs
by PaulDavisThe1st 1734 days ago
It's not hard to stress test plugins. There are a few good tools for some formats out there. They can verify things that most DAWs will not (e.g. by deliberately cross-threading calls and so forth).

The problem is getting people to care. For years, on macOS, the "standard" for plugin developers has been "does it work in Logic?" There were even plugins that would fail auval(1) (the command line AudioUnit validator), but somehow pass in Logic. As far as their developers were concerned, the plugin worked. Working on Ardour, I've seen at least a dozen plugins that used ambiguity in the AU spec to justify why "well, it works in Logic even if it doesn't follow your interpretation of the spec" was the end of their interest.
1 comments
lavabiopsy 1734 days ago
That's what I mean. The solution there would be to get Logic to run "auval" before loading the plugin which ensures that developers don't do that. But for users of Logic doing that serves no benefit as it doesn't actually guarantee the plugins run more reliably, so it won't happen. And even if it did happen, there would be no guarantee that the stress testing tool wasn't also written against other ambiguities in the spec.

Edit: I also think testing is hard in general for plugin developers to use correctly. I personally would prefer to see plugin APIs designed in a better way that make it so it's hard to accidentally cause race conditions.

link
PaulDavisThe1st 1733 days ago
Logic does actually use auval these days, so things are improving. But this still doesn't catch the cases where the issue is something that auval can't or doesn't test (e.g stuff related to GUI interaction).

Plugin API design is an art, indeed. There was recently a brief bubble of activity on KVR among a number of independent plugin devs who found many things to dislike about VST3. A long discussion ensued, there was some talk of picking up LV2 instead, then it all evaporated and nothing was left. The last time the industry tried this was in around 2003, and nothing came (directly) of that effort either.

link
lavabiopsy 1733 days ago
That's also another aspect where the plugin API vendors have not much incentive to fix things, because users can just use another thing on top of it (JUCE, DPF, etc) that makes better progress towards solving these issues from the developer perspective, but still DAWs are left with the tractability problem for other plugins that don't use that.

To me, if there is any interest in solving that, I would just expect someone to put a new plugin backend in JUCE/DPF that is specific to the DAW and then compile that together with the plugins into a big giant build. That's more what I mean by "dropping plugins", it's how you avoid the MxN problem too. But I think that many DAWs (including Ardour) gain little benefit from doing this at this time, so if that was what your original sentiment was, I agree.

link