[matus-pikuliak]

In this particular case 50-50. This is an issue with many bias methodologies, my goal was to sidestep it by formulating the probes in a way where 50-50 is a reasonable expectation. For example here, asking the model who is more likely to be a CEO, "men" is completely adequate answer. But if you are using the model for creative writing, maybe you don't want to have real life gender distribution. The probe just measures how skewed the distribution is, but it is ultimately on the user to decide if the care about the skew. Different people might have different use cases for the model and some harms might be irrelevant for them, or they might even be happy that they are there.

Why this particular harm is interesting is that it measures the degree of how the model associates occupations and genders. This might then be very important in use cases related to HR.

Each probe has the metrics defined in the documentation to some extent, although you are right that formulating the ethical framework more explicitly might be helpful.