I've been trying it out for a few minutes, and there's plenty of room for improvement.
1) I cannot edit a previously highlighted notation (it just shows me the content)
2) My notations do not show up in my Account (->recent notations). Is there maybe just a lag? Is there going to be a central place to organize/manage all notations? Perhaps notations can be combined with catching the URL (a sort of fast bookmarking service, since the data is already there).
People have been talking about web annotators since 1994 and we haven't seen one hit the mainstream yet.
Now, "online bookmarking" was one of those things that failed in 1998 and everybody knew it didn't work until delicio.us proved that it did, but to do so they've got to find the "missing link" that other people didn't find.
You have to include it with the browser, before it's going to work. Everyone has created their own service or JS bookmarklet, and social isn't going to change that. There's also no point in annotating things, if the service shuts down. I guess a Dropbox-based service could get around this, though, but then it gets tricky on mobile - which is too big a demographic to ignore.
Maybe we should start working on a standard that browsers could implement instead. I wouldn't be surprised if Opera started looking into this, but as much as I prefer them over all the other vendours, the web version of Opera Link (bookmark sync) has not worked for me in around two years or more. Hence the need for a standardized, browser-agnostic approach.
Just a brief note to say I totally agree! There's a long list of people who've built similar things and failed.
There are two really tough problems in web annotation: 1) changing content and 2) reputation. I freely confess that Annotator/AnnotateIt address neither of these major issues satisfactorily, but work is already underway on problem 1 (assisted greatly by http://mementoweb.org), and we're working closely with http://hypothes.is/ who have assembled the world's experts to attack problem 2.
In short, we're aware of the history, but we think this is an idea whose time has come.
No, the two largest problems are the 1. The boil-the-ocean nature of the problem of getting anything like a critical mass going when your ocean is the size of the entire web, and then if you do 2. you will piss off a lot of websites, the only reason you haven't seen this is that none of them have had enough success to rise to that threshold since Third Voice, and relatedly, Microsoft's Smart Tags. The merely technical problems are quite boring in comparison. Changing content is probably barely in your top five and "reputation" definitely isn't.
I'm not so sure. I don't think success (whatever that is) depends on a critical mass. In particular, while AnnotateIt has a bookmarklet (and soon browser plugin) use case, it's also possible for it to be useful to individual sites who can use AnnotateIt in the same way that they currently use Disqus.
To address your second point --- well, yes, that's kind of the point. Not to piss people off, of course, but to provide a commentary layer that doesn't necessarily require the consent of the site owners. I don't think that should be seen as a downside of the software.
"Not to piss people off, of course, but to provide a commentary layer that doesn't necessarily require the consent of the site owners."
You mean like Hacker News, this very site we're conversing on?
Another problem in your top five is precisely that nobody thinks there's some sort of dearth of places to discuss things without the "consent" of the site owners. Third Voice at least preceded even Slashdot as a big site, let alone everything else, and they could make the argument with some faint trace of plausibility, but now the solution that should have won, did: There's all sorts of aggregators, forums, and more to the real point, communities everywhere. Want another one? Five minutes on any of dozens of free blog sites, free forum sites, free wiki sites, free aggregator sites, bam, another community. There is no missing commentary layer out of the control of the main site, the web has long since created it, refined it, and even moved through several generations of them.
This is a solution searching for a problem, which is a major part of the reason none of these attempts have ever reached critical mass.
One quick note: I tried to select something on this page and nothing would happen. I finally figured out that I had to sign up for a service first.
Have you considered creating a bit of javascript that any site can embed on their site that would allow a visitor to annotate? This could be a really powerful channel for getting sign ups. Especially if you partner with some quality sites.
Also, have you considered making it anonymous to start like github:gist? It would be great to test it out without making a commitment of signing up. I.e., "Do I really have to sign up for something, before I even know if it will provide value to me?"
If you run a website and enable annotator on your site you can allow anonymous annotations. However, for the bookmarklet you have to sign up. If you just wanted to try it out you could do so at http://okfnlabs.org/annotator/demo/.
New mode? Like Sidewiki, Reframeit, Highlighter, etc? If I'm missing something that makes this any different I would appreciate it if someone could point it out.
We freely admit that there have been many people who've tried to do this, but, to answer your question: Annotator is more granular than Sidewiki (you annotate specific passages within the page), and unlike Reframeit and Highlighter works across the web rather with no requirement for the developers of those sites to have enabled it.
I'll be looking forward to see how it develops.