Velamuvi AIAcademy
Mathematics for AI

Mathematical Foundations of Neural Networks

Put it all together. Derive a neural network from scratch, including the forward pass, loss and backpropagation, one line at a time.

  • Advanced
  • About 10 hours
  • 4 modules
  • 13 lessons
  • Free

This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.

What you will learn

  • A Single NeuronThe Neuron as a Function, Weights, Bias and Activation, Loss as a Number to Shrink
  • Networks as MatricesLayers as Matrix Multiplication, Why Non-Linearity Matters, The Forward Pass in NumPy
  • BackpropagationGradients of the Loss, Backpropagation Step by Step, Deriving Updates for a Two-Layer Network and more
  • Probability Meets NetworksSoftmax, Cross-Entropy, Maximum Likelihood

Course syllabus

Not released yet · full syllabus below

  1. 1

    A Single Neuron

    3 lessons
    1. 1The Neuron as a FunctionSoon
    2. 2Weights, Bias and ActivationSoon
    3. 3Loss as a Number to ShrinkSoon
  2. 2

    Networks as Matrices

    3 lessons
    1. 4Layers as Matrix MultiplicationSoon
    2. 5Why Non-Linearity MattersSoon
    3. 6The Forward Pass in NumPySoon
  3. 3

    Backpropagation

    4 lessons
    1. 7Gradients of the LossSoon
    2. 8Backpropagation Step by StepSoon
    3. 9Deriving Updates for a Two-Layer NetworkSoon
    4. 10Checking Gradients NumericallySoon
  4. 4

    Probability Meets Networks

    3 lessons
    1. 11SoftmaxSoon
    2. 12Cross-EntropySoon
    3. 13Maximum LikelihoodSoon

Recommended next