May

2 events

Editors' summary

GPT-3 takes on tasks from a handful of examples

On 28 May, OpenAI published the GPT-3 paper. At 175 billion parameters, it handled new tasks from a few examples with no further training — translation, arithmetic, continuing a passage — one model covering a wide range. The paper also noted how hard its output was to tell from human writing.

On the 19th at Build, Microsoft announced a supercomputer on Azure built for OpenAI, with 285,000 CPU cores and 10,000 GPUs. The previous year's $1 billion investment had taken the shape of compute.

This block is written by the editors. It is kept separate from the sourced record below.

Record2 events

19
BusinessOpenAI / Microsoft / Kevin Scott
Microsoft unveils AI supercomputer built exclusively for OpenAI
At its Build developer conference, Microsoft announced an AI supercomputer on Azure built in collaboration with and exclusively for OpenAI. According to CTO Kevin Scott, the system featured more than 285,000 CPU cores, 10,000 GPUs, and 400 gigabits per second of network connectivity per GPU server, ranking among the top five publicly disclosed supercomputers. A product of the $1 billion partnership announced in 2019, it became the compute foundation for training GPT-3 and subsequent large models.
28
ResearchOpenAI / Tom Brown / Dario Amodei
GPT-3 paper "Language Models are Few-Shot Learners" published
OpenAI published "Language Models are Few-Shot Learners" on arXiv, introducing GPT-3, a 175-billion-parameter language model—10x larger than any previous non-sparse language model. The paper demonstrated few-shot learning: with only a handful of examples in the prompt and no fine-tuning, GPT-3 performed well across diverse tasks including translation and question answering. The results backed up the scaling-laws predictions and strengthened the view that performance would keep improving with size.

↑ 2020

Microsoft unveils AI supercomputer built exclusively for OpenAI

GPT-3 paper "Language Models are Few-Shot Learners" published