|
|
|
|
|
by pixl97
2616 days ago
|
|
This of course depends on what you mean by 'information'. Lets say we have data points ABCDEFGHIJKLMNOP But depending on the URL you follow to get there you can get a page containing only some of the elements. index.html?ACD or index.html?AP or index.html?GI All different combinations return a page that could be weighted differently by an algormith and represent valid informational return data. To a person looking for the information set DE in one place, this is a valid web page. More so you can abstract the URL query variable away to www.webpage.com/DE. You can quickly run into a combinatorial explosion where even attempting to figure out if a small portion of returns is different would consume most of the energy in the visible universe. |
|
I.e. a service: www.thenumberinsanskrit.com/?q=1 that returns the queried number in Sanskrit, need to not be indexed (except the entry page) while: www.news.com/?article=major-jones-in-scandal-20190103 needs to be indexed.
Usually interesting pages are indexed on the site or linked somewhere on it, though.