[pgo]

So under the hood it's chromium headless and you are streaming screenshots back using an extension

Curious why not run normal chromium and vnc the whole screen back using a web client such as novnc, would have solved most of your "Normal Browser UI things not yet implemented"

[graderjs]

Not an extension. Just your browser, no download needed. So a "clientless" remote isolated browser. it runs headless Chrome, yes (but eventually I want to repurpose it to support other browsers that can be driven by DevTools/WebDriver protocol).

Short answer is because a lot of the work of this was building a custom "head", so having a browser engine without a head made sense. Slightly more detailed answer below... :) ;p xx

I might experiment with xvfb chrome and vnc in future. Other people certainly have success with this, e.g Reflect (YC S20)[0]. Maybe I got here accidentally. Original purpose was for mapping web app structure, and recording and replaying of web action sequences (automation) from any device. As most of the time the browsers were going to replay without anyone using them having xvfb seemed unnecessary. And doing the mapping and recording in one platform (real chrome + xvfb) then replaying in another I thought would risk many unnecessary, difficult bugs. Also having my own wire protocol based on DevTools, rather than use what a VNC web client uses, I believed would assist in more reproducible recordings and playback for automation.

Also I liked the flexibility and control provided by headless+devtools to change the UI, easily work on mobile, custom handlers for tasks like file upload and other modals, as well as the control over the viewport streaming. On streaming, I experimented a lot to get something working across a range of bandwidths, and settled on: a per-frame ack (to avoid congestion), frames over webrtc or binary werbsocket channels, and periodically switching to whichever of webrtc of websocket is faster.

I also want it to be really scalable. And I thought that headless+devtools would be much less resource heavy than chrome+xvfb, but I have no hard data on that and can't find any.

Plus I just liked the challenge. Plugging in Chrome+xvfb+VNC just seemed too easy, and I wanted something "meatier" (apologies to vegetarians, I'm on my way there, 'Proteinier'?) I could dig my hands into :)

[0]: https://news.ycombinator.com/item?id=23897626

[ktpsns]

Are you aware of pure-JS in-browser VNC protocol implementations such as http://guacamole.apache.org/ ? This works over websockets, so even encryption is covered. On the server side, run a Xvnc, fullscreen chrome without window manager or decorations. As the OP said, this should give superiour performance, responsiveness and features. And no need for a client side browser plugin.

Another thing you could look into is https://www.brow.sh/ -- they also have a pretty good performance and feature coverage. Maybe there's something your project can learn from them :-)

[graderjs]

Yes, that's a good idea thanks! I'm really impressed by these web VNC projects, like guacamole and novnc. One thing I think I could learn is how do they send the frames?! They seem to achieve some interesting compression. I think their framerates are higher than mine, and also responsiveness. I definitely could learn something from them. For features, they're not really superior because I probably need all that control, and browser internals instrumentation I get with devtools protocol. E.g how do I upload a file over VNC? But it's a great idea, and thanks for sharing it with me! :P ;) xx

For brow.sh it's kind of a sad project. I mean because it's so popular, with > 13k stars, but the poor guy's only making USD44 a month from this work. I feel so sad to see that. It's very niche, and too niche and restricted for what I'm doing, but one thing I can learn from them is not to be them. I don't want to be doing this labor of love and putting in all that work, and then at the same time asking for donations and only getting 44 a month.

:(

I like the text thing but it's just not what I'm looking for with this. I mean there might be but I don't think there's anything more I can learn from it :P ;) xx

[graderjs]

I just tested out https://www.kasmweb.com/demo.html

They use a custom web VNC client with WebP. Definitely they have higher perceptual framerate and slightly higher responsiveness (actual lag) but the responsiveness is about the same. The real difference is the higher perceptual framerate. I think this is because of how VNC works by sending rectangles of only the changed region instead of the entire screenshot in WebP as I do.

When you compare the framerate on scrolling, or on full screen video, there's not much difference between the Kasm Web demo above, and my demo. With that said, there's definitely room for improvement in my framerate. And I could improve responsiveness with geodistributed endpoints (right now, I'm just using a server in Ohio).

So in future I could add support for checking the changed region, and sending only that region (possibly using the chrome LayerTree to only get and send the changed layers). So I could do that to increase framerate, while still getting all the benefits of my approach, such as full control of browser UI, seamless text pasting, file upload, co-browsing and seamless working on mobile. None of these things are supported by Kasm (at least in the demo above).

However, my focus is not on improving framerate right now. It's on building other features on top of this platform (such as the co-browsing support I added recently), and marketing.

Also I noticed both brows.sh and the Kasm web VNC demo take a long time to load. My app loads basically instantly, and has essentially the same responsiveness, making it highly useful for all kinds of work, as long as you are OK with the lower framerate.

I could also easily "increase" framerate by decreasing the image quality, and I'm considering adding a slider for that.

One thing that really impressed me about the Kasm demo was the audio was instant. I haven't been able to work out how to get rid of the 3 - 13 second lag I get with my pulseaudio / parec / lame setup. I wonder how Kasm is capturing and sending audio !! :P ;) xx

[hjek]

Yo dawg. I heard you like browsing the web, so I put a web browser in your web browser so you can browse the web while you browse the web.

[graderjs]

Exactly!! :P ;) xx I need to add that meme to the page :p :)xx

[atatatat]

Jesus, please don't. The memey-ness is detracting from your presentation, a bit.

Knock it down 25%, unless its 100% genuine. Seems a bit forced from here.