Foundation Model
A foundation model is any model trained on broad data at scale—generally using self-supervision—that can be adapted to a wide range of downstream tasks. The term was introduced by Bommasani et al. (2021) at Stanford’s Center for Research on Foundation Models to describe models like BERT, GPT-3, and DALL-E that serve as a common basis…