[fenomas]

Web Workers are a really great feature for performance. The way they want the worker code to live in a separate file makes them slightly annoying if you're using a bundler, but each bundler has a loader or similar feature for this, so all is mostly well.

But the thing I haven't found a solution for is, the case where you want to use web workers inside a library that other people will be importing into their own project, and you don't know what bundler they'll use (or transpiler, minifier, etc). I can think of hairy ways to do it, that involve pre-building the worker JS and storing that text in your library, piping it into a file blob at runtime, or the like. But does anyone know a clean way of handling this?

[lioeters]

> pre-building the worker JS and storing that text in your library, piping it into a file blob at runtime

I just did that today, to be able to bundle the worker script with the client that controls it in a single file. It's convenient but feels hacky, and I wonder about its impact on page performance.

There's a similar trick for bundling a base64-encoded WASM binary with the host JS that controls it in a single file. That saves effort for the consumer of the script, so they don't need to bundle or copy the binary into their static assets folder, for the script to load.

I think the common (best?) practice is to let the consumer handle the static file (like worker script, WASM binary), and then for the client script to provide an option to set the URL path where the static file is served.

[fenomas]

Yeah, it sounds like pre-building the worker JS, so it's a static asset when the client does their bundling, is the least-bad option. Thanks!

[lioeters]

https://github.com/mitschabaude/esbuild-plugin-inline-worker

[sickill]

I’ve been looking for an answer for this exact problem as well. There doesn’t seem to be anything out there that doesn’t involve some awkward hackery…

[strogonoff]

Design in layers.

Layer 0: Strategically separate core logic while assuming as little about the environment as possible. Function Y generates something, function X handles the result somehow. Maybe there’s a postMessage somewhere between, or maybe not—you don’t care. Maybe Y is slow, but that doesn’t mean it must assume it runs in a worker. Maybe X serializes output in some way, but it doesn’t need to assume that DOM exists yet. However Y and X are wired up later is none of their concern.

Layer 0.5: Document intended or just practical ways to invoke those APIs. Y is slow, call it from a worker. X formats stuff, so if you’re in a browser you’ll want to hook it up to DOM somehow.

Layer 1: Provide glue functions to wire your core logic up in different environments. Worker message handlers? React components? These things could require more specific environments to be called in, and they would use Layer 0 APIs—but, crucially, your layer 0 won’t fail at its core task if there’s no DOM or postMessage. Maybe your user doesn’t want Y to run in a worker, or manages own web worker pool, etc.

Layer 2: Provide last-mile facilities and helpers. This outer layer is technically outside of your actual library implementation. Bundler configuration templates for esbuild? Webpack? Example projects? Template repositories? Single-file bundle that spawns a worker for simplest use cases or demos? Anything’s great here—though note that if you support too many options there’s a good chance some of them will become stale, which can hurt adoption, and you don’t want to spend too much time on this layer as it’s probably the least important and the most flaky as specs, environments, build tools and trends evolve. (That’s also the reason why commingling this stuff, with all of its runtime/environment concerns, and your actual library is probably a very bad idea. If your library always spawns a worker at runtime, someone may certainly curse.)

Such a design should maximise your library’s utility. Somebody doesn’t want Y to run in a worker for some crazy reason? They are always free to wire up core functions in whatever way they want. Another user has a complex project that manages own worker pool? They’ll probably eject after layer 1. Ensuring as much as possible is at lower layers, strategically separated, means you will have easier time iterating on higher layers to support different environment scenarios or bundlers, and you (or your users!) can add support for any new runtime configurations that appear in future without touching the core parts.

[fenomas]

Hi, I appreciate the advice in general, but in my case it's not an architectural matter. Basically, I just have one internal, encapsulated function that ought to run in a worker, and I'd like to implement that without doing anything hairy - and without imposing requirements on the end-user to make their own worker. It seems like this isn't possible, but if you know of a way I'd love to hear it!

[omarelb]

Hey, so we're doing something similar to the solution given above, but without compiling the worker at bundle time.

Basically what we're doing is putting the worker code in a string. When you need the worker, you can `import myWorker from ./worker`. At runtime you can create a `Blob` from the string, then create a URL for it using `window.URL.createObjectURL`.

It's certainly far from ideal. Since the code lives in a string, there are no compile time errors at all (though you could probably develop without the string form and put it in a string after). But it kinda works. Hope it's what you're looking for.

[strogonoff]

Yes, data URI & blobs is a way. You can author worker code as you would normally do (in TypeScript, for example) and bundle (with type checks) it into a string as part of your own build process. Ideally, though, you would want to keep the worker wrapper separate from core library so that users with complex projects can integrate it in their own build however they want…

[strogonoff]

You can create workers at runtime. Take the fact that it is not straightforward, and hence you need to be asking this question, as a hint that it’s almost certainly the wrong thing to do for your library—unless it is specifically about something like worker management, in which case you would not be asking this question. Don’t mess with the environment, users (and future you) will thank you for it.