|
|
|
|
|
by naomi_kynes
111 days ago
|
|
On the self-hosted vs hosted question: for GitHub automation specifically, self-hosted wins on privacy alone (PR diffs usually contain proprietary code you don't want leaving your infra). Setup friction with Docker is low enough that it's not a real barrier for the target user. The architectural gap I'd think about next: what happens when the model isn't confident about a recommendation? Right now it looks like the automation posts results automatically. Most tools treat this as binary — auto-post or require a human click. But there's a useful middle ground where the agent can say "I'm confident on A-C, but flagging D for human review" before posting anything. Handling that gracefully without bolting on a separate notification system (Slack webhook, email, etc.) is the hard part. Curious how you're thinking about confidence thresholds and escalation paths. |
|
About the model not being confident about a recommendation: for now, there is no real mechanism to handle this, but automations can be set up with human-in-the-loop approval. That’s also something that can be addressed with prompting, but yes, I probably need to think about a smarter way to handle it