[achairapart]

Yes, it looks like there is some artificial placement. This may be driven by malice (some sort of paid or more lucrative placement, like you said), but also by stupidity (algo prioritizing songs already in the client cache, to save some egress bandwidth perhaps?).

So I started clearing the Spotify client cache more often, and it looks to me there is more diversity, at least on the auto-generated "recommended songs" playlists. But still, no hard proof of this.

[linsomniac]

The "recommended songs" playlists seem to have more diversity, but also seem to be fairly short (they'll repeat in a couple hours it feels like; I rarely listen to them when I'm working because they'll start repeating, and I don't usually listen to music for a large fraction of the day, so I'm guessing 1-2 hours).

My best guess is that they are assuming no playlist is more than 50-100 songs, and are limiting the shuffle to that number, so that a shuffle doesn't consume too many resources (memory, database hits, CPU cycles). Maybe someone, possibly in the distant past had a large playlist that caused service problems. And because of that they clamped WAY down to prevent it.