Skip to content
AI Atlas

Deep Learning

Handing feature engineering over to stacked layers

OVERVIEW

All Concepts
6
Beginner
2
Intermediate
3
Expert
1

The turning point of deep learning was not “more layers” but the discovery that with enough width, data and stable optimisation, the intermediate abstractions grow on their own. This domain starts from a single neuron and works up to backpropagation, convolution, attention and the normalisation tricks that make depth actually trainable.

Questions this domain answers

  • Q1

    What is a single neuron actually computing?

  • Q2

    How does backpropagation carry error backwards?

  • Q3

    Why are deeper networks harder to train?

Concepts in this domain

  1. 01Neuron & PerceptronBeginnerThe smallest part of a neural network: a weighted sum, a bias, and one twist of nonlinearity
  2. 02BackpropagationIntermediateTurning “compute the gradient” from mathematical drudgery into a single function call — the moment deep learning took off
  3. 03Activation FunctionsBeginnerWithout it, even a very deep network is only a single linear map
  4. 04Convolutional Neural NetworksIntermediateReplacing full connections with “look locally, reuse the same filter everywhere” — the idea that made image recognition work
  5. 05Recurrent Neural NetworksIntermediateGiving networks a memory: one unit reused across time to handle sequences of any length
  6. 06Normalization & Residual ConnectionsExpertMaking hundred-layer networks trainable: an identity shortcut plus a per-layer rescaling