While AI capabilities keep improving, the question of understanding intelligence itself remains a separate quest. Fields from psychology to neuroscience have long studied intelligence, often relying on model organisms ranging from small nematodes to humans. In this talk, I will present some of our group's efforts toward establishing neural networks as a model system for understanding intelligence. I will focus on three topics: compositional generalization, in-context learning, and continual learning. Through these studies, I will show how we can use synthetic setups to evaluate fundamental properties of intelligent phenomena observed in neural networks. Finally, I will conclude with reflections on building AI capable of scientific discovery, and why I believe continual learning and open-endedness are the next fundamental breakthroughs to be made.
