Predictions about future reward drive many (if not all) of our thoughts and behaviors. However, important rewarding events—getting good grades on exams, establishing meaningful relationships, buying your first home—are few and far in between. To work towards these events, we must also predict which behaviors and which intermediary events will bring these rewards into fruition—that studying in a particular manner will provide us with good grades, that some types of social interactions will establish trust and depth of connection, that certain spending habits will facilitate saving for our first home. This leads to several important questions:
- How does the brain discover and encode this predictive knowledge using rich sensory observations of a large and continuously evolving world?
- How does the brain exploit this predictive knowledge to make effective plans, combine its known behaviors, and coordinate with other social agents?
- How can we harness this theoretical understanding to build tools that enhance human agency and help people achieve their goals?
Currently, I am a research fellow in Harvard’s Kempner Institute for the Study of Natural and Artificial Intelligence where I work closely with Sam Gershman. To study these questions, I develop deep reinforcement learning based theories for human learning and generalization. You can find my recent research on my Google Scholar.
I earned my Ph.D. at the University of Michigan, where I studied deep reinforcement learning with Satinder Singh, Honglak Lee, and Richard Lewis, and was supported by the NSF GRFP and a Rackham Merit Fellowship. During my PhD, I was fortunate to spend significant time at DeepMind working with Murray Shanahan, Daniel Zoran, and Danilo Rezende. I got my first stint in ML in a CS M.S. at USC, where I worked with Yan Liu on machine learning for healthcare. Before then, I earned a B.S. in Physics at Stony Brook University where I worked with Axel Drees on computational nuclear physics.
Collaboration and Mentorship
Please feel free to contact me if you’d like to collaborate or be mentored on a research project! While my training is in machine learning, I hope to collaborate broadly with neuroscientists and cognitive scientists.
You can reach me at wcarvalho[at]g.harvard.edu
News
- July, 2025: CCN Benchmarks GAC talk on the utility of benchmarks in cognitive science
- July, 2025: CCN Naturalistic Games Community Event Talk on why NiceWebRL is useful for computational cognitive science
- July, 2025: new preprint on how people generalize to new tasks via counterfactual simulation
- June, 2025: oral at ICML
- May, 2025: New ICML paper on building agents that can coordinate with humans without human data
- April, 2025: Talk at Harvard Cognition, Brain, & Behavior Seminar
- April, 2025: Talk to Kim Stachenfeld’s group at Columbia Neuro
- April, 2025: Talk to Marcelo Mattar’s group at NYU Pysch
- April, 2025: Talk to Eugene Vinitsky’s group at NYU Tandom
- April, 2025: Talk to Mark Ho’s group at NYU Psych
- March, 2025: Talk to RL Seminar at Tübingen
- March, 2025: Talk at Google DeepMind
- March, 2025: Talk at Cognition Seminar at UC Berkeley
- March, 2025: Talk to Bill Thompsons’ group at UC Berkeley Pysch
- March, 2025: Talk to Falk Leider’s group at UCLA
- March, 2025: Talk at Social and Decision Neuroscience Seminar at Caltech
- March, 2025: Talk at USC Symposium “Frontiers of Machine Learning and AI - Fundamentals and Applications”
- March, 2025: Talk at Harvard Neurolunch Seminar
- February, 2025: Talk at RL Seminar at UMich
- February, 2025: Talk to Talia Konkle and George Alvarez’s vision lab at Harvard
- February, 2025: Talk to Bernardo Sabatini’s neuroscience group at Harvard
- February, 2025: Talk at Sam Gershman’s Computational Cognitive Neuroscience Group at Harvard
- February, 2025: New preprint on computational cognitive science for naturalistic experimental paradigms
Miscellaneous
I also maintain a collection of resources for