Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs

AI Engineer

Summary

Mahesh Sathiamoorthy, cofundador de Bespoke Labs, analiza la importancia crítica de la curación de datos y entornos de refuerzo (RL) para el post-entrenamiento de LLMs. Explica que, aunque el cómputo y los modelos base están definidos, la verdadera ventaja competitiva radica en datos de alta calidad. Describe su trabajo en 'Open Thoughts' para modelos de razonamiento y cómo trasladaron esa metodología a agentes autónomos, destacando que el ajuste fino (SFT) sigue siendo fundamental, complementado por RL. Finalmente, detalla el stack necesario para gestionar entornos, puntos de control y optimización de prompts, ayudando a las empresas a ganar soberanía sobre sus modelos personalizados.

Chapters

  1. 0:00Introduction to Bespoke Labs
  2. 2:00From Knowing to Doing: Agent Autonomy
  3. 4:50Data as the Post-Training Bottleneck
  4. 6:51Open Thoughts: Reasoning Data Curation
  5. 10:41Key Learnings from Data Curation
  6. 11:54Open Thoughts Agents and SFT
  7. 14:40Enterprise Post-Training: Credit Karma
  8. 16:04Curator Tool and Reference Stack

Topics covered

  • Curación de datos sintéticos
  • Modelos de razonamiento

+7 more topics (full content when you follow the channel)

Transcript excerpt

Hey everyone, um today I'll be talking about data and uh environment curation for uh post- training LLMs. And I am Mahesh Satyimi. Um I'm co-founder and CEO of Bespoke Labs. And previously I was a researcher and uh engineer at uh Google deep mind. So very briefly I will tell you a little bit about uh bespoke and uh after that the talk will be mostly around uh opensource work we have done. So bespoke is an applied data research lab with a mission to help enterprises and frontier labs access high quality data and RL environments for their post training needs. So very briefly what we do and what we have done is that last year we put out something called curator which is a tool for curating uh synthetic data for post training with basically SFT and right after that actually deepse landed and we started an effort to curate reasoning data and that's how we started something called bespoke stra…