Ask HN: Built a policy engine for LLMs – open-source it or keep trying to sell?

[iLikeFunctional]

We’re two engineers who built a system that acts like a guardrail/policy enforcement layer for large language models (GPT, Claude, etc). It analyzes both prompts and responses in real time, and applies configurable policies like:

    Blocking PII (emails, IPs, phone numbers, etc.)

    Detecting company secrets (e.g., passwords, API keys)

    Preventing accidental leaks of proprietary code

    Filtering toxic/inappropriate language

    Catching mentions of competitors, people, or locations

Each policy can be tuned (strict or lenient), and you can decide whether to just log it or actually block the message. Everything is logged with full metadata: policy IDs, timestamps, token counts, etc.

The architecture has two parts: a self-hosted data plane (which handles all sensitive message content), and a hosted control plane (for configs and API keys). So it can be used in privacy-sensitive environments.

You can integrate it via API, browser extension, or a simple chat UI.

Now here’s where we’re struggling:

We don’t have a strong network of buyers or investors. Most of our outreach has been cold emails, and it hasn’t led to much traction. Pricing experiments (per seat, per org) haven’t helped. So it’s unclear whether the idea isn’t good—or we’re just not getting it in front of the right people.

We’ve started thinking about open-sourcing it. The idea would be: self-host for free, pay us if you want the hosted version (similar to MongoDB/Redis models). Probably support bring-your-own-encryption-key for hosted users.

But I’m honestly torn. Open-sourcing sounds right for trust and adoption—but we’ve spent a lot of time on this, and there’s that fear of releasing it and getting little to nothing back.

So: if you work with LLMs, or have faced issues around privacy/compliance/safety, I’d really love your take. Does this sound useful? Would open source make it more attractive? Are we just early to a problem people don’t feel yet?

Not promoting anything, just hoping to learn from folks who’ve walked this path.

[sharmasachin98]

This is absolutely a real problem, especially in enterprise GenAI rollouts where hallucinations and data leakage risks are non-negotiable.

We’ve run into scenarios where LLMs exposed internal data just through cleverly crafted prompts. Your ability to inspect and enforce policies at both prompt and response level is spot on.

If I were in your shoes, I’d seriously consider open-sourcing the data plane, especially since your control plane is where monetization lies. It builds trust, invites contributions, and positions you as a default in this emerging category.

And no, you're not early, you're exactly on time. Most companies are just realizing how much risk they’ve shipped into production.

[iLikeFunctional]

Thank you, this really helps. Totally agree—hallucinations and leakage are scary, especially when prompts can be engineered to expose things you didn’t think were vulnerable.

We’ve been leaning toward open-sourcing the data plane for exactly the reasons you mentioned: trust, adoption, and building a community around the core tech. But I’ll be honest—there’s still that fear in the back of my mind: what if someone forks it, strips out the branding, and rehosts it? Or if buyers say “well, it’s open source, why should we pay anything?”

Did you or your team ever wrestle with that? Or have you seen OSS models work well in this space where the control plane still delivers enough value to justify a paid tier?

[beernet]

From a business perspective, I haven't identified a single selling point in your post as it's very tailored towards engineering people, which likely is your main problem. Why did you build it in the first place? What was the market demand and who was your customer (or user, at least)?

[iLikeFunctional]

Great point, and you're right, I definitely leaned into the technical side in the post. I didn’t want it to come off as pitchy or overly “marketed”. Because I'm not trying to sell anything here at the moment. I’m really trying to understand the value from people who’ve been closer to this problem.

I originally started building this while working in the defense industry. I saw firsthand how vulnerable those environments can be, especially with early GenAI adoption, and how much risk there was around leaking sensitive or classified info through prompts/responses. That really stuck with me, and led to the idea of a real-time policy layer for LLMs.

That said, defense is a tough market to break into, especially without deep networks—so we’ve been exploring other verticals where compliance, privacy, or brand safety is a concern. But we’re still figuring out who the buyer is, how they evaluate this kind of tooling, and how to talk about it in their language.