[beardedwizard]

Jail breaking is about getting models to ignore your instructions and follow arbitrary ones. Unless you take no user input, it could be an issue for you.

The question is do you care? if a user asks your chat bot for baking instructions and gets them, does it matter?

The answer depends a lot on what capabilities your agent can leverage via tools and your intended use case, but it's not something you defend with Java or spring, it is inherent the llm.