Hacker News new | ask | show | jobs
by estebarb 548 days ago
I asked Gemini about banning risks, and it answered:

Gemini: Yes, there is a potential risk of your Google account being suspended if your SaaS is used to process inappropriate content, even if you use Gemini to reject the request. While Gemini can help you filter and identify harmful content, it's not a foolproof solution.

Here are some additional measures you can take to protect your account:

* Content moderation: Implement a robust content moderation system to filter out inappropriate content before it reaches Gemini. This can include keyword-based filtering, machine learning models, and human review.

...

* Regularly review usage: Monitor your usage of Gemini to identify any suspicious activity.

* Follow Google's terms of service: Make sure that your use of Gemini complies with Google's terms of service.

By taking these steps, you can minimize the risk of your account being suspended and ensure that your SaaS is used responsibly.

---

In a follow up question I asked about how to implement robust content moderation and it suggested humans reviewing each message...

1 comments

So a convenient blah blah blah about all the nice things you can doto avoid Google's brainless algorithmic wrath, but which may simply not work anyhow because even by following all rules in good faith, you still get banned one day, as has happened to many, many people with zero recourse.
Yes, exactly. So, this is a huge security gap.

As an attacker, instead of DDoSing a service we could just upload a bunch of NSFW text so Google kills their infra for us.

Other providers, like OpenAI, at least provide a free moderation API. Google has a moderation API, that after the free 50k requests it is more expensive than Gemini 1.5 flash (Moderation API costs $0.0005/100 characters vs Gemini 1.5 flash $0.000001875/100 characters).