Velamuvi AIAcademy
Generative AI

LLM Architecture

Open the box. Follow a prompt through tokenisation, embeddings, attention layers and the output head of a GPT-style model.

  • Advanced
  • About 8 hours
  • 2 modules
  • 8 lessons
  • Free

This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.

What you will learn

  • The PipelineFrom Text to Tokens to Vectors, Decoder-Only Transformers, Attention in LLMs and more
  • Scaling UpParameters and Layers, KV Caching, Quantisation and more

Course syllabus

Not released yet · full syllabus below

  1. 1

    The Pipeline

    4 lessons
    1. 1From Text to Tokens to VectorsSoon
    2. 2Decoder-Only TransformersSoon
    3. 3Attention in LLMsSoon
    4. 4The Output Layer and LogitsSoon
  2. 2

    Scaling Up

    4 lessons
    1. 5Parameters and LayersSoon
    2. 6KV CachingSoon
    3. 7QuantisationSoon
    4. 8Mixture of ExpertsSoon

Recommended next