[angstrom]

Both, not all tasks are equally parallelized. Some, like the graphics rendering pipeline, data mining, search indexing, and others can reap the benefits and already do. However, there are many tasks that don't parallelize as easily and lead to the programmer waiting for one synchronous task after another due to a dependency. Even if you did run the tasks in parallel the one would need to block and wait until the other completed.

In some cases the programming languages are just ill suited to handle performing concurrent operations free of side effects. Programs written in the languages would have to be rewritten (not likely for many apps). Parallel Studio, by Intel, attempts to make the task easier on programmers working in c++/Windows to identify such cases. Still, others have proposed and attempted to create a layer beneath the programming language that automatically detects code that can be safely executed concurrently without side effects. There's still no silver bullet that's suddenly going to give significant gains without modifications to existing code at this point.