Nice post. I like the focus on developer experience rather than just performance.
Workers' implementation of pipeTo() is indeed optimized to avoid entering the JavaScript isolate at all during the operation. I agree this probably explains the performance difference here.
For setting `Content-Length` on a streaming response, Cloudflare has a non-standard extension called `FixedLengthStream`. It works the same as an identity `TransformStream` except you pass the expected stream length to the constructor.
Wow thanks so much for pointing out the fixed length stream. I had been talking with a few people on your team and no one pointed me to that. That entirely solves our problem to getting things into production. Thanks a ton
Yeah it's not well-known nor well-documented since it's non-standard. There's a bit of debate about whether and how to document it... but don't worry, it won't disappear.
I didn't dig into the benchmark, but this seems to use typescript which fastly I think claimed in the inital post was "beta"? I would be interested to see how this compares to fastly with rust/wasm as their more mature system, even if that would also inherantly not be apples to apples (not that you can ever have that with something like this anyway)
Workers' implementation of pipeTo() is indeed optimized to avoid entering the JavaScript isolate at all during the operation. I agree this probably explains the performance difference here.
For setting `Content-Length` on a streaming response, Cloudflare has a non-standard extension called `FixedLengthStream`. It works the same as an identity `TransformStream` except you pass the expected stream length to the constructor.
Example: https://cloudflareworkers.com/#c2a014c7f67df83f9ad3ea90e9c60...
We would of course prefer to implement a standard API here, but at present there doesn't seem to be one.
(I'm the tech lead for Cloudflare Workers.)