| With respect, I think this is the same innovation that occurs in every industry and it's silly to bemoan the increasing complexity of the web. In transportation: It used to be that everyone could buy a horse and build a buggy and get around. Then cars came along and it got a lot more complicated and expensive to build a vehicle that was state-of-the-art, but tinkerers could still do it. Now there are only a few big players who are capable of innovating and building the best and newest vehicles. Now that I think about it though, there is still space for tinkerers and inventors in the automobile space. But You can't expect those automobiles to compete with those made by, e.g., Toyota. In the same way, it's still possible to write a spider without a javascript renderer. It just won't be able to compete with Google. One last point: the state of the web is based on the collective decisions of all internet users. Ultimately, people building things on the web decided more often than not that ajax-ifying things benefited their users. If users had wanted a web client that would spider the web and shop for them, they would have latched onto it during the time of great innovation that you think is now gone. But they didn't. The things that users wanted are the things we see today, assuming that there isn't some horrible inefficiency in the feedback loop between web-builders and their users. |