one risk is that if too many people are invested in index funds (passive, not buying or selling based on new information), then the price of those stocks is determined by a small group of active investors
So long as the pool is large enough that doesn't matter. Index funds just need the price to be close to reasonable to work out. When the price is not reasonable index funds do well. In general active investors work to push the price to reasonable levels.
Of course there is such a thing as price manipulation which active investors can try - if there are only a few and they work together this can work out. However the investors have incentive to cheat when working together as the cheater wins against his peers, thus this currently is confined to "penny stocks" (for example the company behind the stock doesn't exist anymore but they didn't properly delist their stock so technically it can be traded - you can buy such stocks for say a penny each and then hype them to suckers as the next big thing and sell for 10 cents each and make a killing - since the company doesn't exist no one else pays attention and the scam works.)
Of course there is such a thing as price manipulation which active investors can try - if there are only a few and they work together this can work out. However the investors have incentive to cheat when working together as the cheater wins against his peers, thus this currently is confined to "penny stocks" (for example the company behind the stock doesn't exist anymore but they didn't properly delist their stock so technically it can be traded - you can buy such stocks for say a penny each and then hype them to suckers as the next big thing and sell for 10 cents each and make a killing - since the company doesn't exist no one else pays attention and the scam works.)