Data Sciences Institute  - Data Speaker Series

Data Sciences Institute

Prof. Nathan Kallus, Cornell Tech, Cornell University

Data Science Institute, University of Toronto

LLM Post-Training and Reasoning via Efficient Value-Based RLReinforcement learning (RL) has a newfound killer application in post-training LLMs pre-trained to predict next token to adapt to tasks like instruction following, math-problem solving, and generating content or recommendations that maximize user outcomes. But are the same RL algorithms that animated robots and conquered Atari the right ones to post-train LLMs? In this talk I will present new value-based algorithms for post-training and for scaling test-time compute that leverage both the unique structure of autoregressive LLMs and recent advances on increasing efficiency by changing the Q-learning loss function. I will show how (and argue why) these new algorithms achieve state-of-the-art performance on frontier math reasoning tasks with smaller models and at a fraction of test-time FLOPs. Biography: Prof. Kallus’ research interests include causal inference, especially when combined with machine learning; the statistics of optimization under uncertainty; sequential and dynamic decision making; and algorithmic fairness. He is the author of the book “Applied Causal Inference Powered by ML and AI”. Before coming to Cornell, Nathan was a Visiting Scholar at USC’s Department of Data Sciences and Operations and a Postdoctoral Associate at MIT’s Operations Research and Statistics group. This talk is co-sponsored by the Data Sciences Institute and the Master of Management Analytics Program (MMA), Rotman School of Management, University of Toronto. For more information, please visit <a href="https://datasciences.utoronto.ca/dsi-home/data-sciences-speaker-series/" title="https://datasciences.utoronto.ca/dsi-home/data-sciences-speaker-series/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">https://datasciences.utoronto.ca/dsi-home/data-sciences-speaker-series/</a>.

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Data Sciences Institute - Data Speaker Series

More events from Data Sciences Institute

Discover more events from Data Sciences Institute, from Science & Tech to other experiences you might love.

Still looking for the right event?

Explore all events in Toronto and filter by date, category, and more to find the perfect fit.

Data Sciences Institute - Data Speaker Series

More events from Data Sciences Institute

Discover more events from Data Sciences Institute, from Science & Tech to other experiences you might love.

You might also like...

Browse more events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all events in Toronto and filter by date, category, and more to find the perfect fit.