A line drawing of a butterfly at the end of a looping dashed flight path.

Teaching models to reason about harm to animals

Research on alignment finetuning12 finds that teaching AI models the reasons behind aligned behaviors is just as important as the behaviors themselves.

Two complementary techniques proved especially effective:

  1. Synthetic document finetuning
    Uses pretraining-style documents from a world where the target model is already aligned to a wide variety of aligned behaviors. This reinforces the existence of an aligned persona for the model in training while increasing the number of aligned propensities associated with it.
  2. Difficult advice Q&A
    Depicts an AI assistant coaching users through ethical dilemmas analogous to those the target model might eventually encounter in deployment. This teaches ethical reasoning skills while teaching the model to identify with the responsible persona.

Following this research, we built pipelines for synthesizing training data on a subject currently absent from the training corpus: welfare considerations of nonhuman sentient beings.

We developed scenarios and reasoning principles in consultation with leading animal ethicists to create a robust training set that mirrors situations where real AI systems will take actions beneficial or detrimental to animal welfare.

The two datasets

Synthetic documentsDifficult advice Q&A
resultDiverse artifacts from a world where your model already reasons responsibly about animal welfare.AI coaching users through ethical dilemmas involving disenfranchised third parties (e.g. animals).
result formatBlogs, interviews, encyclopedia entries, forum threads.One user dilemma in, one assistant answer out.
what it is forMidtrainingSupervised fine-tuning QA
pipelinePipelinePipeline
example datasetExample datasetExample dataset

Walk through either pipeline