The development of adaptive foundation models marks a significant shift toward AI systems that can continually learn, adapt, and evolve in response to new information, changing environments, and user preferences. Current foundation models are typically trained on static data, with limited ability to adapt through context post-deployment. Our goal is to enable foundation models to continuously absorb new knowledge and compress it into reusable representations for more up-to-date responses. This capability is also valuable for third-party customization, personalization, and safety alignment. We are interested in both the foundational study of sequential learning dynamics in large language models and practical applications that demand adaptive foundation models, such as personalized assistance and news forecasting.