Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute

AI Engineer

Summary

Este video presenta una visión sobre el futuro del post-entrenamiento en agentes de IA, donde las máquinas pueden aprender nuevas habilidades directamente en el trabajo. Se explora la evolución desde tareas simples de preguntas y respuestas en entornos controlados hasta entornos sintéticos complejos, destacando los desafíos de fidelidad del entorno y el 'reward hacking'. Finalmente, se propone el concepto de 'trae tu propio harness' para entrenar modelos directamente en entornos de producción, abordando la no-repetibilidad y los datos fuera de política. Las direcciones de investigación incluyen la autodestilación, pipelines de datos automatizados y la ingestión de feedback cualitativo, culminando en la visión de sistemas de IA auto-mejorables que aprenden de cada interacción.

Chapters

  1. 0:00Introduction to Post-Training
  2. 0:46Levels of Agent Learning
  3. 2:47Q&A Training Setup
  4. 6:01Synthetic Environment Challenges
  5. 9:23Bring Your Own Harness
  6. 13:18Frontier Research Directions
  7. 15:16Future Vision & Conclusion

Topics covered

  • Agentes y tareas complejas
  • Adaptación a entornos de entrenamiento personalizados (harnesses)

+4 more topics (full content when you follow the channel)

Transcript excerpt

Yeah, thank you, Jack. Um, really grateful for the opportunity to speak here. Today, I'm going to be sharing some of our frontier work on post training and how we envision a future where agents can learn new skills on the job. So, over the last year or so, we've seen agents develop really strong reasoning skills. And they've learned to use, uh, agentic harnesses to solve longer and longer horizon tasks, which involve many turns and tool calls on complicated environment states. We're seeing an increasing, uh, demand for agents that can just be deployed in a plug-and-play way into how enterprises use the agents, um, so, for instance, if they already have some method of calling the agent to do a task, to do a task, they would want to be able to train a custom model to do that task instead. And that requires new ways of looking at post training that allow you to, um, adapt to any harness, in…