Hacker News new | ask | show | jobs
by troupo 795 days ago
> Using unvalidated algorithms (including AI) when life is on the line should not be legal.

EU AI Act forbids that.

2 comments

In European bank I worked at, it was worked around by having an employee "reviewing" the "recommendation" given by AI, and making the final decision. The final decision was of course 100% in line with AI "recommendation".
The AI act is less than a month old; you were likely dealing with either older or local legislation, or some attempt at corporate responsibility (possibly risk management; if you tell your financial regulators “yeah, an unverifiable magic box makes the lending decisions, unreviewed”, you will likely get in trouble, at least in the post-noughties-financial-crisis era).

The EC is usually rather sceptical of attempts to work around the rules.

GDPR Article 22 has been in force for a long time:

> The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.

https://gdpr-info.eu/art-22-gdpr/

Yup, that's my worry about these things, too.
If an employee is pressured to deliver faster, and the employer looks the other way when chatgpt comes out - then there is little for the regulator to do. If you penalize the employee, another will just do the same.
As a regulator, you don't penalize the employee; you penalize the company as though it were a wilful violation.
The company is much larger and incentivized to make it look like an employee issue. Unless there are active mechanisms to prevent this, it will be the default outcome.