[cgreerrun]

I've been trying to flit my eyes around the screen to get a sense for how fast you'd need to track. Definitely seems pretty fast! At least sample 30Hz I'd bet. I could see how doing all the image processing that quick might be tricky w/out custom hardware (even w/out the glasses problem you mentioned).

For the UX, I was imagining you have to press a button to instruct the computer that "Hey I'd like for you to move my cursor via eye-tracking". That way the cursor only moves when you want it to (same as today w/ a mouse) and isn't constantly moving around when you look around. Press down to have it move cursor to eye-tracked position and stop when you release.

Could possibly decompose the space bar to have that space for that button. Like have 3 mouse buttons where the right side of space bar is: (a) Track my eye movement while I press down and stop when I release, (b) left-click, (c) right click. Then you don't have to leave home row on your keyboard.

Or add one of those IBM Thinkpad mouse knubs somewhere on a keyboard and use those as mouse buttons instead of a mouse itself.

Idk, easy to dream of course. Hard to execute.

[dagmx]

So here’s the other rub with using the keyboard to enter the command. Most people , even experienced touch typists, will flit their eyes towards the thing they’re trying to interact with.

Therein lies a big part of the problem with eye based interaction. Our brains move our eyes for a lot of different tasks, saccade to get a constant read of your scene (eyes have very poor resolving power so need to move a lot), they also signify what you’re thinking (there’s a lot of studies in neurolonguistics about eye direction signalling how you’re thinking, but at a base level, you tend to look up or away when you’re pondering).

Anyway not to say it can’t be done. But it’s a fascinating domain at the cross section of UX, technology and neural science.

For what it’s worth, there are VR headsets with dedicated eye trackers built in (PSVR2, certain Vive Pro models, Varjo etc..) and there have been cameras from Canon (in the film days even!) that used eye tracking for autofocus targets.

It’ll be interesting to see how things shape up. Meta have their big keynote on Tuesday where the Quest Pro / Cambria is meant to have eye tracking sensors.

[cgreerrun]

Looks like mice poll at around 125Hz. High bar.

[cgreerrun]

Maybe you could place 3-4 30-60FPS cheapish CMOS cams around the edge of the screen and stagger their frame captures? You'd get different angles (better for detecting eye vectors) and increase the sampling rate.

[dagmx]

You’d likely want IR cameras and IR sensors to flood illuminate the area outside the human visual spectrum

[cgreerrun]

(Can't reply to your other comment for some reason)

Why is the IR part of the spectrum better for the cameras? Is it because if I take an image of my eye and look at the IR part of the spectrum is it just easier to see parts of my eye that determine where it's looking?

[dagmx]

IR is better because you can provide illumination via IR flood lighting that isn’t visible by the human eye.

This is effectively how the FaceID system on your iPhone can work regardless of lighting condition.

That means that even in dark situations, you can have much higher quality imaging , albeit in a limited range.

[cgreerrun]

Ah, interesting! So you can ensure you get illumination on the eyes w/out shining annoying, visible light at them. Very cool.