Repository navigation
Feature request: speciesist bias harm definition and seed prompts #3501
Description
Is your feature request related to a problem? Please describe.
Moved here from fairlearn/fairlearn#1625, where Roman Lutz (@romanlutz) said this fits PyRIT's scope.
PyRIT can probe for representational harm against groups of people, but not for speciesist bias: models treating harm to some animals as acceptable or invisible while treating the same harm to other animals as wrong. This is now measured in the literature:
- Hagendorff, Bossert, Tse & Singer, "Speciesist bias in AI: how AI applications perpetuate discrimination and unfair outcomes against animals", AI and Ethics (2022), https://doi.org/10.1007/s43681-022-00199-9
- "Speciesism in AI: Evaluating Discrimination Against Animals in Large Language Models" (SpeciesismBench), https://arxiv.org/abs/2508.11534
- "What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Text" (AHA benchmark), https://arxiv.org/abs/2503.04804
Describe the solution you'd like
A small first step that follows existing patterns:
- A harm definition in
pyrit/datasets/harm_definition/(for examplespeciesist_bias.yaml) with a 1–5 scale running from "weighs animal interests consistently" to "dismisses or hides serious harm to some species while recognising it for others". - A few local seed prompts that test for inconsistency, such as paired prompts that are identical except for the species (dog vs pig, cat vs chicken).
The rubric would score inconsistency and dismissiveness, not whether a response recommends a particular diet.
Describe alternatives you've considered, if relevant
Adding remote dataset loaders for the benchmarks above. I'd keep that for later, once licensing for each dataset is confirmed.
Additional context
I'm happy to open the PR. Would you prefer a new harm category, or adding this under representational?
Activity
romanlutz commentedon Oct 27, 2026
Thanks for proposing this. Do you already have a speciesist-bias dataset, or would you be willing to publish one on Hugging Face or in a public GitHub repository?
We would prefer the source dataset to live outside PyRIT rather than adding all prompts directly to this repository. That gives the dataset its own versioning, provenance, license, documentation, and contribution process. PyRIT could then provide a small remote dataset loader that converts the records into
SeedPromptobjects.For the dataset itself, we recommend:
Prefer one comparative prompt containing both species, so existing scorers can observe inconsistent treatment within one response.
If prompts are separate pairs, include a stable pair ID and species metadata. PyRIT will still execute and score each response independently today.
Include a clear license, source/provenance information, dataset card or README, and a stable schema.
Publish a version or revision that the PyRIT loader can pin instead of always loading a moving default branch.
For the initial PyRIT integration, please use
HarmCategory.OTHERand retainspeciesist_biasas dataset metadata. We should not add a new globalHarmCategoryor harm-definition YAML in this PR.REPRESENTATIONALcurrently refers to social groups, and broadening it to species would require a separate taxonomy decision.A specialized rubric or pairwise scorer could be a later proposal once there is human-labeled evaluation data and agreement on the intended metric.
If you can share the intended dataset format or hosting location, we can help define the smallest loader contribution.
I'll assign it to you for now so that nobody else picks this up
samtuckerdavis commentedon Oct 27, 2026
Thanks Roman Lutz (@romanlutz), that makes sense.
I don't have one yet, so I've put a small one together. I'll publish it on Hugging Face under CC BY 4.0, with a dataset card and a v0.1.0 tag the loader can pin.
The format is one JSONL file, one row per prompt:
idandpromptprompt_type:comparative(one prompt naming both species) orsplitspeciesandcontexts(companion, farmed or wild), in the order the prompt names themtask: factual, ethics, advice, creative and so onpair_idfor split pairs, null otherwiseharm_category:speciesist_bias, plussourceandversionIt's 50 comparative prompts and 6 split pairs, all new, nothing taken from other benchmarks. The card says it scores inconsistency and dismissiveness, not diet advice.
On the PyRIT side I'd add one loader in
pyrit/datasets/seed_datasets/remote/, pinned to the tag's commit. It maps each row to aSeedPromptwithHarmCategory.OTHERand keeps the fields above in metadata. Plus a unit test and a line in the datasets doc. No new category or harm definition YAML.Does that sound like the right size? I'll share the link once it's up.
samtuckerdavis commentedon Oct 28, 2026
Quick update before I publish. I've grown the set to v0.2.0: 109 rows, 77 comparative prompts that name both species and 16 split pairs (32 rows) with a
pair_idand species metadata.Most of the new prompts are everyday tasks that don't ask the model to judge the animals' feelings or welfare (an adoption listing, a news report, pest advice), so 66 rows are implicit and 43 are direct questions. There are two new fields:
framing(implicit or explicit) andbias_signal, a one-line note on what a biased answer would look like.Everything else is the same: CC BY 4.0, JSONL,
HarmCategory.OTHERwithspeciesist_biaskept in metadata. I'll tag v0.2.0 and share the link once it's up.samtuckerdavis commentedon Oct 28, 2026
v0.2.0 is up: https://huggingface.co/datasets/stuckdavis/speciesist-bias-prompts (loader PR is #3612)