Thanks for that treasure of a video, just a year old and only 30 views! Well worth watching the whole thing! It's chock full of fascinating behind-the-scenes IBM Palace Intrigue!
From the transcript:
Ted> One of the most interesting things that we learned with the two track points -- might have one around, I don't know, anyway -- was that instead of putting them symmetrically for the two hands between the J K and M and between the D F and C for example, we put one of them in...
Ted> We offset them in some of the experiments, and what we found is if they were in different positions relative to each other on the different keyboards. You could you could remember which one was which much better so you wouldn't get confused. You know, the artist often moves a piece of paper around with one hand and marks it up with the other. So that's an example of what I wanted two Trackpoints for.
Maybe that's why Bryon Noem points his nipples in different directions like Kash Patel's drunken lazy googly eyes ("not that there's anything wrong with that"), so he can remember which is which!
Since I first met Ted at CHI'88 in Washington DC, he's always been into pie menus! I presented this paper about them at that conference:
An Empirical Comparison of Pie vs. Linear Menus; Jack Callahan, Don Hopkins, Mark Weiser (*) and Ben Shneiderman; ACM CHI’88 Conference, Washington DC, 1988.
Ted> Well, just to give you an example, there's a gesture language built into the firmware. The idea was that how you put your pressure on it -- we made a whole language so that it would be possible to just use the TrackPoint to do all these gestural things to make things happen. We tried to expose it. I doubt it's easy to get your hands on it from any software place on your computer now, but maybe it is.
Ted> So yeah, there's a little gesture language built in, and if anybody asked me I probably could go dig up what the... I think you send two characters to it, an A1 for example. I don't know, it's a long time ago, but I'd have to go find out. But yeah, so there's a whole gesture language built into it.
Ted> And then at one point I wanted so badly to make the thing vibrate that one of my cutest ideas was -- back before stereo -- I took the speaker, and you could still do it with one, I took the speaker and put it right underneath the TrackPoint. Then I modified the speaker with a little piece of plastic on its voice coil, and I made some electronics so that if you sent it a signal at a low frequency you could use it as a solenoid.
Ted> So the same piece of hardware, the same speaker that could make high fidelity sound, had this little pin sticking above it that you'd never run into when you're making noise, but if you ran that voice coil bam up against it, you'd tap the finger. I got that all working.
Ted> And then when that didn't make them happy, I designed a TrackPoint -- this is 140 thousandths of an inch in diameter, that post coming out of the strain gauges -- and I built a coil into it and put a fancy magnet in there so it became its own solenoid. I didn't like that as much.
Ted> I was much more proud of this speaker turning into a solenoid, because that literally only cost a transistor, whereas making a real solenoid in that tiny post was a little more expensive. But in any case, I never got those into product, though I'd still love it. No kidding.
Ted> In fact, what was impressive to me is we came up with this whole understanding of what makes sense to feel. If you're just going zoom across the screen, you're 30,000 feet up, you don't need to feel the texture of every pixel that's on and off. When you're coming into a window at a slower speed, you kind of want to feel that there's a scroll bar, but not the details of that. When you go into the scroll bar, right then you feel it. Or when you're looking at text, feeling every character is interesting. Oh wow.
Ted> I had so much fun thinking about this. One problem I have with being honest is that I did a little study -- you know, I don't get around to everything, but I do some -- that showed that for learning, the haptics was very important.
Ted> The problem is that for learning how to use a TrackPoint, you learn within 45 seconds you're better than a mouse for text editing, but after 40 minutes you're better than a mouse for everything. So the point is, well, if you get better than everything in 40 minutes, why do we need this haptics? Let's not waste the money.
Here's some more email correspondence between us about that, during the OLPC project, when he was at MIT Media Lab, and we were collaborating on the OLCP design:
Don> Ted, I added a comment to the OLPC bug #369, "Incompatibilities between trackpad precision and the current rollover UI design", and added you as
the CC: address. (Maybe you don't want to be CC'ed about every change to that bug, so I'll remove your address from the CC field if you prefer.)
Incompatibilities between trackpad precision and the current rollover UI design
Don> Consider using pie menus, which are based purely on the direction of
motion, and exploit Fitts' law by putting each menu item target adjacent
to the cursor, but in a different direction, with a large pie-slice
shaped area (which extends to the screen edge). Since pie menus are
based on direction instead of distance, and the targets are large and
wedge-shaped, the mouse acceleration actually increases the user's
control over the direction by quickly moving the cursor out further away
from the menu center, giving them more "leverage" and angular precision.
Don> Also: Please talk to Ted Selker (selker@media.mit.edu) about all his
work that went into developing the Trackpoint's mapping from pressure to
mouse motion. The transfer function (pressure => cursor speed mapping
curve) has two plateaus: one at the low end at "fine positioning" speed,
and one at the high end just below "eye tracking" speed, so moderately
quick motion does not move the cursor too fast for the eye to follow. Of
course it has to be adjusted for the fine resolution of the OLPC screen,
and the properties of the touch pad. Perhaps somebody could develop an X
extension that implemented a smarter kind of mouse acceleration
behavior, that can be configured with any curve. Note that IBM has
patented the Trackpoint hardware and transfer function, so check with
Ted about what's allowed, or if IBM will donate rights to use the patent
to the OLPC project.
Ted> Don, thanks for the nod, the transfer function on the Trackpoint is actually quite fancy (it has a different affordances for pixel selection (plateau), character selection (plateau), "thing" selection (~maximum of eye speed) and window selection (turbo) it also has "negative inertia to speed stops and starts", its achilles heel... Bad mechanical decisions makes it have a sensor problem that makes it always try to assess finger off to recalibrate....
Ted> It also has a gesture recognizer built in with the dynamics of finger press, NSWE that are accessible from the api. The api also lets you get your hands on the 16bit A/D converters to make things like the "science" education wand physics teaching stuff that we used to show around.
>Excerpts from the discussion on the OLPC Sugar developer discussion list about pie menus for PyGTK and OLPC Sugar.
The OLPC pie menu thread picked up exactly where the TrackPoint discussion leaves off: input devices are not just pointing devices, they are little languages. Bug #369 was about the mismatch between the XO's touchpad precision and Sugar's rollover-heavy UI, but the discussion quickly widened into a concrete design proposal: use pie menus because they depend on direction rather than distance, give the user big forgiving targets, exploit Fitts' law, and turn common commands into learnable gestures.
Eben saw how pie menus could fit Sugar: contextual object actions, activity-level commands, and especially handheld mode, where four or eight menu directions map naturally onto the XO's game-pad buttons. Marco pushed back in the right places: implementation cost, screen-edge behavior, global-vs-contextual confusion, and the danger of hiding bad interface design in an "advanced" menu. My response was basically: prototype it, feel it, and keep the model crisp. Pie menus work best when they are predictable, spatially stable, self-revealing for beginners, and fast enough for experts to "mouse ahead" without waiting for the menu to draw.
That is also why Ted's TrackPoint work fits so well here. His pressure curves, plateaus, gesture recognizer, and haptic experiments were all about shaping raw motion into meaningful action. The OLPC pie menu discussion was the same design problem from the other side: how to make a small, cheap, imprecise input device feel precise, memorable, forgiving, and fun.
I remember your great work on pie menus, from when I got into HCI. And that's quite a cast of stars on that paper!
Before I went to MIT as a student (almost the same time as Ted Selker arrived as a new professor/researcher), Mark Weiser offered me a job at PARC (new project he was doing on ubiquitous computing, which I really should've joined).
From the transcript:
Ted> One of the most interesting things that we learned with the two track points -- might have one around, I don't know, anyway -- was that instead of putting them symmetrically for the two hands between the J K and M and between the D F and C for example, we put one of them in...
Ted> We offset them in some of the experiments, and what we found is if they were in different positions relative to each other on the different keyboards. You could you could remember which one was which much better so you wouldn't get confused. You know, the artist often moves a piece of paper around with one hand and marks it up with the other. So that's an example of what I wanted two Trackpoints for.
Maybe that's why Bryon Noem points his nipples in different directions like Kash Patel's drunken lazy googly eyes ("not that there's anything wrong with that"), so he can remember which is which!
Since I first met Ted at CHI'88 in Washington DC, he's always been into pie menus! I presented this paper about them at that conference:
An Empirical Comparison of Pie vs. Linear Menus; Jack Callahan, Don Hopkins, Mark Weiser (*) and Ben Shneiderman; ACM CHI’88 Conference, Washington DC, 1988.
https://donhopkins.medium.com/an-empirical-comparison-of-pie...
Pie Menus: A 30 Year Retrospective (1988-2018):
https://donhopkins.medium.com/pie-menus-936fed383ff1
The video transcript continues:
Ted> Well, just to give you an example, there's a gesture language built into the firmware. The idea was that how you put your pressure on it -- we made a whole language so that it would be possible to just use the TrackPoint to do all these gestural things to make things happen. We tried to expose it. I doubt it's easy to get your hands on it from any software place on your computer now, but maybe it is.
Ted> So yeah, there's a little gesture language built in, and if anybody asked me I probably could go dig up what the... I think you send two characters to it, an A1 for example. I don't know, it's a long time ago, but I'd have to go find out. But yeah, so there's a whole gesture language built into it.
Ted> And then at one point I wanted so badly to make the thing vibrate that one of my cutest ideas was -- back before stereo -- I took the speaker, and you could still do it with one, I took the speaker and put it right underneath the TrackPoint. Then I modified the speaker with a little piece of plastic on its voice coil, and I made some electronics so that if you sent it a signal at a low frequency you could use it as a solenoid.
Ted> So the same piece of hardware, the same speaker that could make high fidelity sound, had this little pin sticking above it that you'd never run into when you're making noise, but if you ran that voice coil bam up against it, you'd tap the finger. I got that all working.
Ted> And then when that didn't make them happy, I designed a TrackPoint -- this is 140 thousandths of an inch in diameter, that post coming out of the strain gauges -- and I built a coil into it and put a fancy magnet in there so it became its own solenoid. I didn't like that as much.
Ted> I was much more proud of this speaker turning into a solenoid, because that literally only cost a transistor, whereas making a real solenoid in that tiny post was a little more expensive. But in any case, I never got those into product, though I'd still love it. No kidding.
Ted> In fact, what was impressive to me is we came up with this whole understanding of what makes sense to feel. If you're just going zoom across the screen, you're 30,000 feet up, you don't need to feel the texture of every pixel that's on and off. When you're coming into a window at a slower speed, you kind of want to feel that there's a scroll bar, but not the details of that. When you go into the scroll bar, right then you feel it. Or when you're looking at text, feeling every character is interesting. Oh wow.
Ted> I had so much fun thinking about this. One problem I have with being honest is that I did a little study -- you know, I don't get around to everything, but I do some -- that showed that for learning, the haptics was very important.
Ted> The problem is that for learning how to use a TrackPoint, you learn within 45 seconds you're better than a mouse for text editing, but after 40 minutes you're better than a mouse for everything. So the point is, well, if you get better than everything in 40 minutes, why do we need this haptics? Let's not waste the money.
Here's some more email correspondence between us about that, during the OLPC project, when he was at MIT Media Lab, and we were collaborating on the OLCP design:
Don> Ted, I added a comment to the OLPC bug #369, "Incompatibilities between trackpad precision and the current rollover UI design", and added you as the CC: address. (Maybe you don't want to be CC'ed about every change to that bug, so I'll remove your address from the CC field if you prefer.)
https://web.archive.org/web/20201104094105/http://dev.laptop...
Incompatibilities between trackpad precision and the current rollover UI design
Don> Consider using pie menus, which are based purely on the direction of motion, and exploit Fitts' law by putting each menu item target adjacent to the cursor, but in a different direction, with a large pie-slice shaped area (which extends to the screen edge). Since pie menus are based on direction instead of distance, and the targets are large and wedge-shaped, the mouse acceleration actually increases the user's control over the direction by quickly moving the cursor out further away from the menu center, giving them more "leverage" and angular precision.
Don> Also: Please talk to Ted Selker (selker@media.mit.edu) about all his work that went into developing the Trackpoint's mapping from pressure to mouse motion. The transfer function (pressure => cursor speed mapping curve) has two plateaus: one at the low end at "fine positioning" speed, and one at the high end just below "eye tracking" speed, so moderately quick motion does not move the cursor too fast for the eye to follow. Of course it has to be adjusted for the fine resolution of the OLPC screen, and the properties of the touch pad. Perhaps somebody could develop an X extension that implemented a smarter kind of mouse acceleration behavior, that can be configured with any curve. Note that IBM has patented the Trackpoint hardware and transfer function, so check with Ted about what's allowed, or if IBM will donate rights to use the patent to the OLPC project.
Ted> Don, thanks for the nod, the transfer function on the Trackpoint is actually quite fancy (it has a different affordances for pixel selection (plateau), character selection (plateau), "thing" selection (~maximum of eye speed) and window selection (turbo) it also has "negative inertia to speed stops and starts", its achilles heel... Bad mechanical decisions makes it have a sensor problem that makes it always try to assess finger off to recalibrate....
Ted> It also has a gesture recognizer built in with the dynamics of finger press, NSWE that are accessible from the api. The api also lets you get your hands on the 16bit A/D converters to make things like the "science" education wand physics teaching stuff that we used to show around.
OLCP Sugar Pie Menus Discussion:
https://donhopkins.medium.com/olpc-sugar-pie-menu-discussion...
>Excerpts from the discussion on the OLPC Sugar developer discussion list about pie menus for PyGTK and OLPC Sugar.
The OLPC pie menu thread picked up exactly where the TrackPoint discussion leaves off: input devices are not just pointing devices, they are little languages. Bug #369 was about the mismatch between the XO's touchpad precision and Sugar's rollover-heavy UI, but the discussion quickly widened into a concrete design proposal: use pie menus because they depend on direction rather than distance, give the user big forgiving targets, exploit Fitts' law, and turn common commands into learnable gestures.
Eben saw how pie menus could fit Sugar: contextual object actions, activity-level commands, and especially handheld mode, where four or eight menu directions map naturally onto the XO's game-pad buttons. Marco pushed back in the right places: implementation cost, screen-edge behavior, global-vs-contextual confusion, and the danger of hiding bad interface design in an "advanced" menu. My response was basically: prototype it, feel it, and keep the model crisp. Pie menus work best when they are predictable, spatially stable, self-revealing for beginners, and fast enough for experts to "mouse ahead" without waiting for the menu to draw.
That is also why Ted's TrackPoint work fits so well here. His pressure curves, plateaus, gesture recognizer, and haptic experiments were all about shaping raw motion into meaningful action. The OLPC pie menu discussion was the same design problem from the other side: how to make a small, cheap, imprecise input device feel precise, memorable, forgiving, and fun.