[Aachen]

Maybe the hammer analogy I used is not great because hitting yourself is obviously bad whereas any of the recipes might be toxic in a way that the user is not aware of, in which case you're right. But the complaint is that it suggests impossible dishes upon inputting ingredients that are obviously inedible and harmful substances, which sounds to me like it is functioning correctly even if this humorous function was unintentional

Who would ask this LLM to make a recipe involving bleach and then actually proceed to make it? Such a person is already at risk of poisoning themselves, the software doesn't suggest it by itself so I don't see how it increases the risk of harm

[piva00]

But the purpose underlining a meal-planner AI is to create meals. If it's creating something that is not a meal it's not doing it's purpose, if the use-case is for meals then there should be guardrails against non-meals recipes, simply because it does not fit the purpose of the product.

It's not a LLM product, it's not advertised as a general LLM that will generate text based on input of shopping items, it's specifically advertised as an AI meal planner and it's not fit for that purpose since it does not guard against non-meals.

That's the issue, the user is supposed to be a layman, not someone that knows and understands LLMs and its limitations.