January

1 events

Editors' summary

OpenAI shows a smaller model people prefer

On 27 January, OpenAI announced InstructGPT, tuned on human ratings to follow instructions more closely. A version with a hundredth of the parameters was preferred by people over the original GPT-3, the company reported, and it made these models the default in the API.

It showed there was room to work on models other than by making them larger, and tuning on human feedback spread from there.

On the 28th, researchers at Google published a paper on chain-of-thought prompting. Showing a model a handful of examples that wrote out the intermediate steps, rather than the answer alone, raised its accuracy on arithmetic and commonsense problems. Nothing in the model changed — only the wording handed to it.

This block is written by the editors. It is kept separate from the sourced record below.

Record1 events

27
ModelOpenAI
InstructGPT unveiled; RLHF-tuned models become API default
OpenAI unveiled InstructGPT, versions of GPT-3 fine-tuned to follow instructions using reinforcement learning from human feedback (RLHF), and deployed them as the default language models on its API. Labelers preferred outputs from the 1.3-billion-parameter InstructGPT over those of the 175-billion-parameter GPT-3, a model more than 100x larger. It was the first large-scale production deployment of alignment research and laid the technical groundwork for ChatGPT later that year.

↑ 2022

InstructGPT unveiled; RLHF-tuned models become API default