The last line of GP's comment is key here: "Who do I sue if Palantir decides I am an illegal?"
This shouldn't make as much of a difference as it does, but due to how our legal system works, it's much harder to get meaningful legal satisfaction when an algorithm (or other inhuman distributed system) commits a crime against a person than when a person does so.
I think you're confused about the mechanism involved. It's hard to get satisfaction due to e.g. qualified immunity. The fact they use technology is largely irrelevant. You couldn't sue the NSA for spying on you before AI either.
If we assume they are on quotas then what difference does technology make? They had quotas before the technology, qualified immunity too. 100 false arrests with no recourse are 100 false arrests with no recourse.
If anything I would expect technology favors the victim of false arrest because it gives the cop a face-saving get out. Previously, a cop who false-arrested you would have been incentivized to take it all the way, because you getting justice for it is intrinsically tied to impugning their word and/or reputation.
This shouldn't make as much of a difference as it does, but due to how our legal system works, it's much harder to get meaningful legal satisfaction when an algorithm (or other inhuman distributed system) commits a crime against a person than when a person does so.