Stable Diffusion at the time of writing the article uses a pre-trained model called CLIP released by OpenAI. CLIP is trained on a dataset consisting of a combination of 'images' and 'captions'.