TOM ZAHAVY 

Deep Reinforcement Learning

tomzahavy (at) gmail (dot) com

I am a research scientist at DeepMind in the field of Reinforcement Learning. Previously, I was a Ph.D.  candidate at the Technion and interned at Microsoft, Walmart, Facebook, and Google.

My high-level research goal is to build an artificial intelligence via Reinforcement Learning. My prior work focused on aspects of scalability, structure discovery, hierarchy, abstraction, and exploration.

Currently, I focus on Discovery:
 

• Metagradients: building reinforcement learning algorithms that discover an internal knowledge base (hyper parameters, loss function, options, reward), in order to solve the original problem better. You can read more about it in my papers [1,2,3] in this blog post by Robert Lange, in this Podcast (with Robert, Tim, Yanick and myself), or in this talk by David Silver.  
   

• Diversity: finding diverse policies that are also nearly optimal in terms maximising reward. I am thinking about this problem through the lens of convex optimisation [1,2,3], that is, suggesting unsupervised convex objectives that lead to diversity as well as RL algorithms that find solutions to this objective via the maximisation of reward.  

About me: I come from a small town in 🇮🇱 on the Mediterranean Sea. I am currently living in London 🇬🇧 and I spent some time in the 🇺🇸. My family is coming from 🇩🇪🇮🇩🇱🇺 and by DNA I am 🇮🇩🇭🇺🇮🇷(50/30/20). I am married to Gili​, a singer-songwriter from 🇮🇩🇲🇦🇮🇱. I love spending my free time outdoors in camping, hiking, 4X4 driving, mountaineering, skiing, and scuba diving. When I am at home, my hobbies are running, basketball, and reading science-fiction. 

I was recently featured in the Machine Learning Street Talk podcast, where we talked about my deep RL journey, automatic discovery of structure and our recent meta gradient papers, you can listen to it here.

News

​​​

• May 21      Two papers accepted to ICML: "Online Limited Memory Neural-Linear Bandits with Likelihood Matching" 

• May 21      and "Emphatic Algorithms for Deep Reinforcement Learning" 

• Jan 21       Two papers accepted to ICLR. "Discovering a Set of Policies for the Worst Case Reward" (spotlight)

• Jan 21       and "Balancing Constraints and Rewards with Meta-Gradient D4PG" 

• Dec 20      A blog post by Robert Lange is covering my work on meta gradients 

• Dec 20      Inverse Reinforcement Learning in Contextual MDPs was accepted to the Journal of Machine Learning 

• Oct 20       A Self-Tuning Actor-Critic Algorithm" was accepted to NeurIPS2020

• Mar 20      Unknown mixing times in apprenticeship and reinforcement learning" was accepted to UAI2020

• Mar 20      Learning to Ask Medical Questions using Reinforcement Learning" was accepted to MLHC2020

• Nov 19      The Technion magazine wrote a story about our "Inverse Reinforcement Learning in Contextual MDPs" paper (Hebrew)

• Nov 19      Planning in Hierarchical Reinforcement Learning: Guarantees for Using Local Policies" was accepted to ALT2020

• Nov 19      Apprenticeship Learning via Frank-Wolfe" was accepted to AAAI20

• Nov 19      I joined Deepmind as a research scientist

• Oct 19       I finished my Ph.D. at the Technion, advised by Shie Mannor.

• Oct 19       I finished my long term internship at Google, working with Yishay Mansour.

• Sep 19       I've been chosen as one of the top 400 reviewers at NeurIPS

• Jul  19       I co-organized the third lifelong learning workshop at ICML

• May 19      Patent "Systems, Method, and Non-Transitory Computer-Readable Storage Media for Multi-Modal Product Classification"

Selected Publications