Hacker News new | ask | show | jobs
by brador 2806 days ago
The problem with systems is emergent and unexpected behaviour.

Things can go very bad very quickly once a death spiral kicks in.

2 comments

That can be a problem with programs, too.
Since emergent behavior itself, is wanted, the unexpected subset is what you imply is detrimental, but that's the case for programs as well.
Emergent behavior isn't always wanted. Sometimes it's desirable (Oh, this widget can be copy/pasted into other documents! I didn't expect that to work yet.) Sometimes it's not (I can't figure out why this bot keeps stacking boxes. Patrolling the area is supposed to be its highest priority right now.).

Sometimes we can deliberately induce emergent behavior (https://en.wikipedia.org/wiki/Automated_planning_and_schedul...) by composing our system correctly. To create behaviors and solutions that we didn't initially conceive of. But if unplanned emergent behavior shows up, your system has become unmaintainable until you've identified the causes.

Why do you reference the article on planning and scheduling?
I should've linked directly to the part I thought was relevant:

https://en.wikipedia.org/wiki/Automated_planning_and_schedul...

Also, I think I deleted something extra in an edit of my original post. These planning algorithms (or some of them) involve creating options for your agent/AI and seeing what it does when interacting with its environment. You don't design the agent so that it will do a complex task, rather you give it simpler abilities (recognize an object type, pick up an object, set down an obect on another object, etc.) and the complexity arises precisely from emergent behavior. This was meant as an example of precisely where you do want emergent behavior and actively work to induce it.