Let's build GPT: from scratch, in code, spelled out.

Summary

En este tutorial exhaustivo, el autor guía al espectador a través de la construcción de un modelo de lenguaje Transformer desde cero, estilo GPT. Partiendo del dataset 'tiny Shakespeare', explica paso a paso la arquitectura de los Transformers, desde los conceptos básicos de tokenización y modelos de Bigrama, hasta la implementación de mecanismos de auto-atención (self-attention), normalización de capas, conexiones residuales y el escalado final. El vídeo desmitifica el funcionamiento interno de sistemas como ChatGPT, diferenciando claramente entre la fase de pre-entrenamiento (completado de documentos) y la posterior fase de fine-tuning para alineación como asistente.

Topics covered

• Arquitectura Transformer
• Tokenización

+4 more topics (full content when you follow the channel)

Transcript excerpt

hi everyone so by now you have probably heard of chat GPT it has taken the world and AI Community by storm and it is a system that allows you to interact with an AI and give it text based tasks so for example we can ask chat GPT to write us a small Hau about how important it is that people understand Ai and then they can use it to improve the world and make it more prosperous so when we run this AI knowledge brings prosperity for all to see Embrace its power okay not bad and so you could see that chpt went from left to right and generated all these words SE sort of sequentially now I asked it already the exact same prompt a little bit earlier and it generated a slightly different outcome ai's power to grow ignorance holds us back learn Prosperity weights so uh pretty good in both cases and slightly different so you can see that chat GPT is a probabilistic system and for any one prompt it…