Constitutional AI paper: harmlessness from AI feedback
Anthropic published "Constitutional AI: Harmlessness from AI Feedback," a method for training a model to avoid harmful output without relying heavily on human labels. Instead, the model critiques and revises its own responses against a written list of principles — a "constitution." Training proceeds in two stages: supervised learning on the self-revised responses, then reinforcement learning using AI-generated preference data as the reward signal, which the authors called RLAIF (RL from AI feedback). The result, they reported, was an assistant that engages with harmful queries by explaining its objections rather than issuing flat refusals. The technique became central to how Claude is trained and a hallmark of Anthropic's technical approach.