[zambelli]

Do you mean catching errors as tokens stream back versus waiting for the full message? If so, then no I hadn't looked into that. This was mostly geared towards local models so token cost isn't really a big deal, though latency might be.

And if you didn't mean that then please elaborate :)

[MWil]

No, more like not waiting for drift/deviation to hit something load bearing or god forbid go on hitting unnoticed over time. Let it hit something trivial that is constantly being monitored cheaply.

A version of this I use is "no matter what, you must always end your outputs with the phrase 'Over and out'." Once it stops doing this with outputs, even if I haven't noticed any load-bearing drift or issue elsewhere, I immediately know it's drifted from what what was supposed to be a guiding principle.

Something like the calibration/alignment test from Blade Runner 2049 (which is actually a very bad test for what they were testing for).

[zambelli]

Ohhhh, that's much more interesting. I haven't looked into that at all, but now I'm curious. I'd need to think way more about how to layer that into forge, but the principle could likely be applied somewhere. I get it now.