And don't forget, the plugin API has some super strong limitations. I wish that was fixed, otherwise platforms "similar to wordpress" can never become a thing on Go
Actually, folks usually just use gRPC or Yaegi in Go. You can write an SDK for your software that abstracts all of that away for the plugin developer.
See Terraform[0], Traefik[1], or OctoSQL[2].
Though I agree plugins would be welcome, especially for performance reasons, though also to be able to compile and load go code into a running go process (JIT-ish).
Well aware of all that, but as you point out, real plugins would be better.
And to be clear they exist, but you have to compile them with the same go version and if any dependency is shared, they need to be exactly the same version. There are also serious limitations in the sharable code
I have had cases where Terraform would OOM because each Terraform provider instance maps to a fresh new Go process.
Most of the time this approach is not an issue, and it works really well. But you need to be aware that it's best to keep the number of providers in the 0-20 range per root module.
If I was going to try to make something "similar to Wordpress" with Go I would replace plugins with optional packages triggered by build flags (or possibly a light codegen step), not an actual dynamic plugin system. The Wordpress model is solved with plugins in PHP (for reasons that were to some degree bad even at the time), but it's not actually "I want to inject some external code into this existing process environment", it's "I want a bespoke N of M features because some Ms are mutually exclusive or have overhead I can't afford".
If you have a language that has a module ecosystem, compiles fast, and generates a single binary, take advantage of that; don't try to reimplement fragile dynamic linking.
I'd like to point out also that wordpress can install plugin directly from the UI, which is a lot more involved if it requires the whole software to be recompiled (totally different machine too)
But it’s not so expensive you can’t do it. If you cache it’s probably not even so expensive anyone will notice compared to the equivalent PHP process. And it would be a massive operational, usability, and performance improvement to the actual server.
Otherwise you’re just saying “I can’t write Go like PHP” - no shit, that’s a good thing.
WordPress is an example. For C# videogame bots, the software is delivered as a compiled executable file (no sources) and plugins are sold as DLLs that you can just copy-paste in a directory.
Kerbal Space Program (videogame) is a big compiled executable and plugins are loaded as DLLs from a directory.
I'm not trying to reimpostare anything, I just wish it was available.
I've written plugins in Go. You don't really need to use the plugin package, and you probably shouldn't anyway.
One way I've done it in the past is via sub processes talking via stdin/stdout. Another way I've done it is just via a regular REST API.
You'll never get enough performance for a lot of tasks if you need to pass a lot of data between applications. But depending on your needs it's more than serviceable. The benefit of this is that plugins don't need to be written in Go, as you have a simple interface.
A good example of this is something like LSP. LSP plugins are abundant and each one is written in a different language. VSCode is Typescript, the Go language server is written in Go. Both can speak to eachother fast enough.
Unfortunately, the performance as you point out is not even close. Not only that, but the API is way more limited by the transport protocol, instead of being able to directly use language interfaces.
If you write plugins for PHP, they are just as fast as the basic software. If you write plugins for C# (dlls), they are as fast as the language.
Writing using grpc has substantial development overhead and performance overhead, so it makes Go a bad choice by default
Plugins for a compiled binary are just plain hard. Changing the code works better with an interpreted language. Usually you see the software implement it's own interpreter or a scripting language on top to handle this.
Plugins for ELF- and Mach-O-based systems aren't any different than regular shared libraries, modulo slightly different symbol precedence. AFAIU, while a quite different architecture, Windows PE linking treats them quite similarly, too.
Perhaps you meant statically compiled binaries? Even then I'm not so sure about that. Right now I'm working on a project that statically compiles a plugin so its various libraries can't leak or be overridden by the loading application, yet which itself can load other plugins. OTOH, this is in C, C++, Objective-C, and Lua (basically C from the perspective of binary linking), all of which have mature linking semantics. (Well, at least this is true for C. Controlling symbol namespace pollution by C++ and Objective-C code requires more complexity than for C, but at least the necessary compiler and linker flags exist and are mature.)
When you use languages or toolchains that don't invest in an ABI, or which make linking too automagic (with no or little recourse for exposing various linking features a platform may offer), then runtime linking is likely to become a major limitation in some solution areas. For example, there's probably no way to coax XCode to compile my plugins properly, without outsourcing some logic to additional scripts. Ultimately a Makefile is likely the best solution--and the one I chose, anticipating these headaches--as it keeps most of the build transparent. Fortunately, XCode seems to just be a wrapper around a bunch of command-line utilities, so you don't actually need XCode at all.
The C ABI is nowhere near enough for passing arbitrary Go structs, all you need is a map or a sync.Mutex field and you need to exactly match plugin compilation to caller compilation.
I wish https://pkg.go.dev/plugin was clearly documented as not being a viable 3rd party plugin solution.
It's not enough to pass arbitrary C++ classes, either. The point is that you can still design a fairly expressive API involving structured data within the constraints that the C ABI imposes on you - and that C ABI will guarantee stability. This is how plugins were usually handled in the C++ world, historically speaking.
(There are also higher-level ABIs like COM, but they are usually reducible to the C ABI in practice - e.g. COM can be described entirely in terms of struct and function pointers.)
I know, I was thinking C# does something like that though, so I was hoping it would be achievable.
The annoying part about the scripting language is that if something is written in Go, you want developers to also work in Go, otherwise the people writing plugins for your software, knowing better the API and use-cases, have a disconnection with the language actually used for the software, losing potential opportunities for contributions.
See Terraform[0], Traefik[1], or OctoSQL[2].
Though I agree plugins would be welcome, especially for performance reasons, though also to be able to compile and load go code into a running go process (JIT-ish).
[0]: https://github.com/hashicorp/terraform
[1]: https://github.com/traefik/traefik
[2]: https://github.com/cube2222/octosql
Disclaimer: author of OctoSQL