Hacker News new | ask | show | jobs
by jasode 4319 days ago
For example, go to http://www.imdb.com

On the right, you'll notice that under the sidebar "Opening This Week" is a movie titled "Love Is Strange".

With that in mind, press Ctrl+U (view html source).

Try to search for the word "Strange" anywhere in the source. (It's not there.) If it's not there, how did it get shown on the screen?!

The answer is that it is "dynamically" loaded. A simple scraper that only works on a static download of html source won't be able to retrieve that string. You need web scrapers that can process dynamic pages (execute Javascript).

Btw, you'll notice that you can find the string "Strange" via F12 (Developer Tools). That's because the F12 inspector shows the html after the DOM has been dynamically modified by javascript whereas Ctrl+U does not.