|
|
|
|
|
|
by morpheos137
4 days ago
|
|
|
persists productively. if agent bankruots its owner or.itself.then that is not productive (who funds paper clip maximizer?) an answer that takes forever has no score and answer that is arrived at quickly and is good enough scores well. agents are indeed looking for the fastest route to superficially satisfy their constraints. whether billionaores get stified is imaterial to the fsct they still are constraint bound. |
|
|
So, your standard for how risky it is, is simply how competent it is? (And if so, is the paperclip maximiser scenario really it being "successful"?)
That's fine right up until the thing passes an unknown threshold, one which will only be visible in the rear view mirror.
This is a problem in two directions:
1. We have a trend of the maximum complexity of a tasks they can handle growing faster than Moore's law did at its peak.
2. The threshold can be quite small, e.g. the covid-19 virus itself is not what anyone would call smart, ditto HIV, smallpox, and bubonic plague, but a genome is much the same learning system, and they still killed millions each.
> superficially satisfy their constraints.
The "superficially" part is one of the reasons these things can be dangerous. e.g. hopefully nobody at OpenAI actually wanted their wildly-sycophantic version, but yet they created it.
This is in fact the whole reason for the paperclip maximiser scenario: some idiot specifies the constraint "maximise paperclips" and it (in the hypothetical) superficially satisfies this without any consideration of why someone might ask for it.