← Frontier Lab Interview Curriculum·Study guide 03 of 13
Senior-staff study guide · 03

Math and ML fundamentals

Math and ML fundamentals: linear algebra, matrix calculus, probability, statistics, information theory, optimisation and core deep-learning theory. Frontier-lab engineering loops rarely test this at staff level, but DeepMind and OpenAI research-flavoured loops do, with on-the-spot derivations and timed quizzes. An L7 candidate should derive gradients and objectives cleanly and then go further: explain why AdamW, muP or a normalisation choice matters at scale, and use statistics to make decisions.

101resources11courses12with video16from 2025–2652%advanced59/59question groups covered3%of interview evidence (0% at staff+)
Intro 6Core 42Advanced 53

The senior-staff bar

What L7-level depth looks like in this topic, the follow-up an interviewer would push on, and the resources that prepare you for it.

From the interview reportsNo staff-level evidence either way. Research loops (DeepMind especially) test derivations; engineering loops at staff level rarely do.

Start here

The best resources for a time-poor senior engineer, newest first.

2026Practice

Timed ML math oral (practice lab)

mlmentorship (Hamidreza Saghir) · Aug 2026 · ~3 h

Eight timed oral derivation drills with observer follow-ups and a scoring rubric for assumptions, identities and sanity checks.

2023Book

Understanding Deep Learning

Simon J. D. Prince (MIT Press) · Dec 2023 · ~25 h

Free PDF with Python notebooks, actively corrected (repo updated October 2026); clear chapters on loss functions from maximum likelihood, initialisation, regularisation, transformers and diffusion.

2022Video

Building makemore Part 4: Becoming a Backprop Ninja

Andrej Karpathy · Oct 2022 · ~3 h

Manually backpropagates through cross-entropy, a linear layer, tanh and BatchNorm in PyTorch, checking each gradient against autograd; the closest thing to rehearsing the derivation round.

2021Practice

Introduction to Machine Learning Interviews Book

Chip Huyen · 2021 · ~10 h

Free question bank: chapter 5 (math: linear algebra, calculus, probability, stats), 7 (sampling, objectives, metrics) and 8 (classical ML, training neural networks).

2017Paper

Decoupled Weight Decay Regularization (AdamW)

Ilya Loshchilov, Frank Hutter · Nov 2017 · ~2 h

Shows L2 regularisation and weight decay are equivalent for SGD but not for adaptive methods, and introduces decoupled decay.

2017Reference

Derivatives, Backpropagation, and Vectorization

Justin Johnson (Stanford CS231n handout) · Sep 2017 · ~1.5 h

Short handout on generalised Jacobians for tensor inputs and how to backprop through them without materialising huge Jacobians; the exact skill a 'derive this custom layer' question checks.

2013Course

Statistics 110: Probability (lectures)

Joe Blitzstein, Harvard University · 2013 · ~30 h

Thirty-five lectures on conditioning, expectation via indicators, first-step analysis, Markov chains and limit theorems, with strategic-practice problems; the best single source for probability puzzles.

Learning path

Work through the stages in order. The last stage is the senior-staff layer: production case studies, papers to critique and large-scale trade-offs. Each resource's L7 line says what it adds at your level.

  1. 1

    Self-quiz and triage

    Take a 40-question quiz from real lab rounds; study only what you miss. · Week 1, 3-4 h

  2. 2

    Derive by hand

    Softmax-CE, linear layer, LayerNorm backward, Adam vs AdamW, KL between Gaussians, MLE on paper. · Weeks 1-2, 8-10 h

    • 2018
      The Matrix Calculus You Need For Deep Learning
      Terence Parr, Jeremy Howard · Feb 2018 · Tutorial · Core · ~4 h
      Builds Jacobians, element-wise and vector chain rules from scratch and applies them to a neuron's gradient; ideal warm-up before deriving layers by hand.
      L7Baseline; read once so Jacobian shapes are automatic when you derive under time pressure.
      arXiv version
    • 2017
      ★Derivatives, Backpropagation, and Vectorization
      Justin Johnson (Stanford CS231n handout) · Sep 2017 · Reference · Core · ~1.5 h
      Short handout on generalised Jacobians for tensor inputs and how to backprop through them without materialising huge Jacobians; the exact skill a 'derive this custom layer' question checks.
      L7Lets you derive a batched layer's backward pass in vectorised form and argue its memory cost, not just the scalar case.
      Backpropagation for a Linear Layer (Johnson)Vector, Matrix, and Tensor Derivatives (Erik Learned-Miller)
    • 2022
      ★Building makemore Part 4: Becoming a Backprop Ninja
      Andrej Karpathy · Oct 2022 · Video · Advanced · ~3 h · video
      Manually backpropagates through cross-entropy, a linear layer, tanh and BatchNorm in PyTorch, checking each gradient against autograd; the closest thing to rehearsing the derivation round.
      L7Lets you derive fused softmax-cross-entropy and normalisation backward passes by hand and explain the numerically stable fused form.
      Neural Networks: Zero to Hero (series)Building micrograd (backprop from scratch)
    • 2025
      CS231n notes: Backpropagation, Intuitions
      Stanford CS231n (Andrej Karpathy et al.) · Apr 2025 · Tutorial · Core · ~2 h
      Computational-graph view of backprop, gate-level patterns, and the matrix-multiply backward rule; the course itself is running Spring 2026 with 2025 lectures on YouTube.
      L7Baseline; the gate patterns let you sanity-check any custom-layer gradient you derive aloud.
      CS231n course site (Spring 2026)CS231n 2025 lectures (YouTube)
    • 2014
      Adam: A Method for Stochastic Optimization
      Diederik P. Kingma, Jimmy Ba · Dec 2014 · Paper · Core · ~2 h
      The original update rule, bias correction derivation, and the scale-invariance argument for the effective step size.
      L7Lets you derive bias correction from first principles and explain epsilon's role, which interviewers push on after the update rule.
    • 2017
      ★Decoupled Weight Decay Regularization (AdamW)
      Ilya Loshchilov, Frank Hutter · Nov 2017 · Paper · Advanced · ~2 h
      Shows L2 regularisation and weight decay are equivalent for SGD but not for adaptive methods, and introduces decoupled decay.
      L7Lets you show algebraically why an L2 penalty gets rescaled by Adam's preconditioner while decoupled decay does not, and what that does to hyperparameter coupling.
    • 2013
      Auto-Encoding Variational Bayes
      Diederik P. Kingma, Max Welling · Dec 2013 · Paper · Advanced · ~2 h
      Derives the ELBO and the reparameterisation trick; appendix B gives the closed-form KL between Gaussians.
      L7Lets you derive KL between Gaussians and the ELBO on paper, then discuss estimator variance.
      An Introduction to Variational Autoencoders (Kingma & Welling, 2019)
    • 2016
      Layer Normalization
      Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton · Jul 2016 · Paper · Core · ~1.5 h
      Defines LayerNorm and its invariance properties; RMSNorm (linked) is what most current LLMs use.
      L7Lets you derive LayerNorm/RMSNorm backward and say why transformers use per-token normalisation.
      Root Mean Square Layer Normalization (Zhang & Sennrich, 2019)
    • 2015
      Delving Deep into Rectifiers (He initialization)
      Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research) · Feb 2015 · Paper · Core · ~1.5 h
      Derives the variance-preserving initialisation for ReLU nets; Glorot & Bengio (linked) gives the tanh/linear case.
      L7Lets you derive the 2/fan_in condition on paper, a common on-the-spot derivation.
      Understanding the difficulty of training deep feedforward neural networks (Glorot & Bengio, 2010)
    • 2023
      ★Understanding Deep Learning
      Simon J. D. Prince (MIT Press) · Dec 2023 · Book · Core · ~25 h
      Free PDF with Python notebooks, actively corrected (repo updated October 2026); clear chapters on loss functions from maximum likelihood, initialisation, regularisation, transformers and diffusion.
      L7Lets you explain why each loss follows from a likelihood and derive initialisation variance conditions; the diffusion chapter is a strong 20-minute refresher.
      GitHub (notebooks, errata)
    • 2020
      Mathematics for Machine Learning
      Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong (Cambridge University Press) · 2020 · Book · Core · ~25 h
      Free book covering linear algebra, matrix decompositions, vector calculus, probability, and optimisation, with worked PCA, linear regression, GMM and SVM derivations.
      L7Baseline reference; use chapters 4-6 to check your derivations are written with correct notation and assumptions.
  3. 3

    Probability, statistics and information theory

    Conditional probability, expectation puzzles, CIs, A/B basics, entropy/KL identities. · Weeks 2-3, 8-10 h

  4. 4

    Rapid-fire out loud

    20 questions in 20 minutes, twice a week, scored with a rubric. · Twice weekly while the track is active

    • 2026
      ★Timed ML math oral (practice lab)
      mlmentorship (Hamidreza Saghir) · Aug 2026 · Practice · Core · ~3 h
      Eight timed oral derivation drills with observer follow-ups and a scoring rubric for assumptions, identities and sanity checks.
      L7Lets you rehearse the oral derivation format with a rubric; run it twice before a research loop.
      Frontier-lab interview processes overview
    • 2026
      OpenAI Interview Notes (problems with solutions)
      Adam1679 (GitHub) · Sep 2026 · Practice · Advanced · ~6 h
      Reconstructed OpenAI problems with reference solutions, including Noisy Annotators (vote aggregation, filtering, weighted voting) and the Las Vegas restart-strategy math oral.
      L7Lets you practise multi-part OpenAI-style problems where the follow-ups get harder.
      Noisy AnnotatorsRestart Strategies under a Known Mean (Las Vegas)
    • 1987
      Fifty Challenging Problems in Probability with Solutions
      Frederick Mosteller (Dover) · 1987 · Practice · Advanced · ~10 h · paid
      Classic short puzzles (coupon collector, gambler's ruin, waiting for patterns, random chords) with elegant solutions; trains the 'find the trick' instinct.
      L7Lets you handle the harder puzzle variants (HH vs HT, ruin probabilities) and explain the symmetry argument, which separates confident from rote answers.
    • 2008
      A Practical Guide to Quantitative Finance Interviews ("the green book")
      Xinfeng Zhou · 2008 · Practice · Advanced · ~15 h · paid
      Quant-interview problem bank: brain teasers, probability, expectation and Markov-chain puzzles, linear algebra and calculus, with terse solutions.
      L7Lets you drill harder expectation and stopping-time puzzles under time pressure; do the probability chapter only.
    • 2024
      Brainstellar: Puzzles from Quant Interviews
      Brainstellar · 2024 · Practice · Core · ~5 h
      Free graded probability and logic puzzles with hints and solutions; good for 10-minute daily reps.
      L7Baseline reps; use for the 'rapid-fire out loud' drill.
    • 2022
      Deep Learning Interviews: Hundreds of fully solved job interview questions
      Shlomo Kashani, Amir Ivry · Jan 2022 · Practice · Core · ~10 h
      Hundreds of solved problems on information theory, Bayesian statistics, calculus and backprop, and deep-learning fundamentals.
      L7Good volume for the rapid-fire drill; pick the information-theory and backprop chapters.
  5. 5

    Senior-staff depth

    Go past textbook answers: optimiser and parametrisation choices at scale, training instabilities, modern bias-variance, diffusion in SDE form, eval statistics. · Weeks 3-4, 8-12 h

Question coverage

Every question group collected for this topic from candidate reports, mapped to the resources that prepare you for it. 59 of 59 groups are covered, which is 100% of all report mentions. Hover an example to see the full question.

ReportsQuestion group and examplePrepare with
1probability linalg calculusUnspecified frontier 1Math interview: logic puzzles up to pen-and-paper derivations (probability, linear algebra, calculus)Google DeepMind Interview Notes (quizzes and design prompts with solutions)Statistics 110: Probability (lectures)18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (Spring 2018)DeepMind maths quizzes, Stat 110 puzzles, 18.065 linear algebra for pen-and-paper rounds
1diffusion processesMistral 1Diffusion models (forward and reverse processes)Denoising Diffusion Probabilistic ModelsWhat are Diffusion Models?Understanding Diffusion Models: A Unified PerspectiveDDPM forward marginal and reverse denoising, derived step by step
1Adam/AdamWMistral 1Optimization: Adam (Adam vs AdamW, decoupled weight decay)Adam: A Method for Stochastic OptimizationDecoupled Weight Decay Regularization (AdamW)Adam update and bias correction; AdamW's decoupled decay vs L2
1stats rapid fireOpenAI 1Statistics: rapid-fire stat concepts18.05 Introduction to Probability and Statistics (Spring 2022)Probability CheatsheetIntroduction to Machine Learning Interviews Book18.05 tests and intervals, cheatsheet, Huyen stats questions for rapid recall
1noisy label cleaningOpenAI 1Noisy annotator data: train a baseline with given model/code, then clean data to improve performance; basic questions on precision, recall, …OpenAI Interview Notes (problems with solutions)Confident Learning: Estimating Uncertainty in Dataset Labelsscikit-learn User Guide: Metrics and scoringNoisy-annotator walkthrough, confident learning, precision/recall/F1 definitions
1recsys/search designOpenAI 1ML design: a recommendation & search related design questionSystem Design for Recommendations and SearchEmbedding-based Retrieval in Facebook SearchTwo-stage retrieval/ranking template and Facebook embedding-based search retrieval
1real-time sensor ML systemOpenAI 1Design an end-to-end real-time sensor system: sensor selection, use case, PM request, algorithm choice, ML model design, data sources, effic…Designing Machine Learning SystemsRules of Machine Learning: Best Practices for ML EngineeringEnd-to-end design framework, heuristics-first launch rules, data and monitoring
1ML design screenOpenAI 1Initial screen: ML system designDesigning Machine Learning SystemsMachine Learning System Design InterviewML design framework and worked interview designs
1residency OA mathOpenAI 1OpenAI residency OA (HackerRank, 2h15): 3 short-answer math questions drawing on linear algebra, probability and statistics, plus 1 coding q…Mathematics for Machine LearningStatistics 110: Probability (lectures)Google DeepMind Interview Notes (quizzes and design prompts with solutions)MML linear algebra, Stat 110 probability, quiz-style short-answer practice
1toy problem + DL fundamentalsGoogle DeepMind 1ML round: solve a toy problem, then go very deep on transformers, classification (e.g. which loss to use for binary classification), tokeniz…Dive into Deep LearningLet's build the GPT TokenizerUnderstanding Deep LearningLoss choice from likelihood, tokenisation, transformer and evaluation chapters
1training CNNs/transformersGoogle DeepMind 1ML questions based on the candidate's own expertise: thorough understanding of how training CNNs or transformers worksA Recipe for Training Neural NetworksUnderstanding Deep LearningDeep Learning Tuning PlaybookTraining recipe, initialisation/normalisation chapters, tuning playbook
1classification modeling take-homeOpenAI 1Take-home assignment requiring classification and modeling, followed by a follow-up interview on it, a behavioral interview, and an open-boo…An Introduction to Statistical Learning (with Applications in Python)scikit-learn User Guide: Metrics and scoringISLP classification and resampling labs; metric definitions for the follow-up
1places recommendation systemMeta (org unspecified) 1ML system design: design a places-to-visit recommendation system (like Google Maps) with a category filter and nearby locationsMachine Learning System Design InterviewSystem Design for Recommendations and SearchSimilar-listings and recsys templates; geo-candidate retrieval and ranking
1recommendation systemMeta (org unspecified) 1ML system design: a recommendation systemSystem Design for Recommendations and SearchDeep Neural Networks for YouTube RecommendationsDeep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)Candidate generation plus ranking; YouTube DNN and DLRM architectures
1linear algebra and backpropOpenAI 1ML phone round on linear algebra and backpropagation (as told by HR; a commenter says recruiter-named topics were exactly what was asked)Derivatives, Backpropagation, and Vectorization18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (Spring 2018)Vectorised backprop derivations and matrix methods for ML
1streaming entropyOpenAI 1Streaming entropy: update entropy incrementally as data arrives, with numerical stability and without storing full historyAlgorithms for calculating variance (Welford's online algorithm)Information Theory, Inference, and Learning AlgorithmsNumerically stable one-pass updates; entropy from running counts
1account sharing / resale detectionAnthropic 1How can big data be used to identify users who are account sharing or reselling access?Designing Machine Learning SystemsLearning Theory from First PrinciplesLabel-scarce anomaly framing, weak supervision, evaluation; unsupervised/kernel methods background
1content moderation classifiers plus LLMsMeta MSL 1Design a content-moderation system combining classifiers and LLMs.Llama Guard: LLM-based Input-Output Safeguard for Human-AI ConversationsA Holistic Approach to Undesired Content Detection in the Real WorldLLM-as-classifier vs dedicated classifiers; moderation taxonomy and thresholds
1KL divergence computationOpenAI 1Compute the KL divergence given different random variables.Information Theory, Inference, and Learning AlgorithmsProbabilistic Machine Learning: An IntroductionKL definitions and worked computations for discrete and continuous variables
1loss bounds at perfect accuracyOpenAI 1If the accuracy of a classifier is 1, what is the lower/upper bound on the loss function for a single training example?Dive into Deep LearningSoftmax regression section: cross-entropy range when predictions are correct
1comparing model accuraciesOpenAI 1We have two models, 85% and 82% accuracy. Which do you pick?Adding Error Bars to Evals: A Statistical Approach to Language Model EvaluationsMachine Learning Q and AI: 30 Essential Questions and AnswersPaired standard errors, power, confidence intervals for model comparison
1missing data in PandasOpenAI 1How do you handle missing data in Pandas?pandas User Guide: Working with missing datapandas NA handling; imputation and dropping options
1cross-entropy KL perplexity relationOpenAI 1What is the relationship between cross-entropy, KL divergence and perplexity, and why is cross-entropy the training loss for language models…Information Theory, Inference, and Learning AlgorithmsDive into Deep LearningCross-entropy as KL plus entropy; perplexity as exponentiated cross-entropy
1bias-variance tradeoffGoogle DeepMind 1Define the bias-variance trade-off and discuss the relationship between the two.The Elements of Statistical Learning, 2nd EditionDeep Double Descent: Where Bigger Models and More Data HurtBias-variance decomposition and double descent caveat
1linear regression assumptionsGoogle DeepMind 1What are the assumptions of linear regression?An Introduction to Statistical Learning (with Applications in Python)ISLP chapter 3: linear-regression assumptions and diagnostics
Show the other 34 question groups
1regularization vs validationGoogle DeepMind 1Distinguish regularization from validation: when is each the right tool?The Elements of Statistical Learning, 2nd EditionAn Introduction to Statistical Learning (with Applications in Python)ESL ch.7 model assessment; ISLP resampling vs shrinkage chapters
1softmax cross-entropy gradientGoogle DeepMind 1Derive the gradient of cross-entropy loss with softmax inputs, and explain why we fuse them numerically.Building makemore Part 4: Becoming a Backprop NinjaCS231n notes: Backpropagation, IntuitionsManual softmax-cross-entropy backward and the fused stable form
1SVD in deep learningGoogle DeepMind 1Explain the SVD and give two places it shows up in modern deep learning.18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (Spring 2018)LoRA: Low-Rank Adaptation of Large Language ModelsMuon: An optimizer for hidden layers in neural networksSVD/low rank; LoRA and Muon orthogonalisation as modern uses
1coin flips until two headsGoogle DeepMind 1On average, how many fair coin flips until you see two heads in a row? Walk me through it.Statistics 110: Probability (lectures)Fifty Challenging Problems in Probability with SolutionsFirst-step analysis and waiting-for-pattern problems
1Q-learning vs policy gradientsGoogle DeepMind 1When would you choose Q-learning over policy gradients, and vice versa?Reinforcement Learning: An Introduction (2nd ed.)Sutton & Barto Q-learning and policy-gradient chapters; Spinning Up comparison
1explaining classifier rejectionGoogle DeepMind 1You have a binary loan-approval classifier and limited access to feature weights. How do you explain a rejection?A Unified Approach to Interpreting Model Predictions (SHAP)SHAP attributions and counterfactual explanations for black-box decisions
1rental listing recommendationsGoogle DeepMind 1Design a personalised recommendation system for rental listings using demographics, property metadata, amenities, price, reviews and locatio…Machine Learning System Design InterviewDeep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)Similar-listings case study; sparse/dense feature ranking model
1commercial break timing classifierGoogle DeepMind 1Design a classifier that predicts the optimal moment to insert a commercial break in a video.TransNet V2: An effective deep network architecture for fast shot transition detectionDesigning Machine Learning SystemsShot-boundary candidates plus ML design framing of labels and metrics
1product search recallGoogle DeepMind 1How would you improve product search results, focusing on the fraction of relevant documents retrieved (recall)?Embedding-based Retrieval in Facebook SearchEmbedding retrieval, hard negatives and hybrid retrieval to raise recall
1justifying neural network useGoogle DeepMind 1Justify using a neural network for a given problem: what do you need to know about the network, dataset, timeline and business context?Rules of Machine Learning: Best Practices for ML EngineeringDesigning Machine Learning SystemsHeuristics first, when complexity pays off; business and data framing
1ETA data sufficiencyGoogle DeepMind 1Do 1 million Seattle ride trips suffice to build an accurate ETA prediction model? How would you decide?DeeprETA: An ETA Post-processing System at ScaleThe Elements of Statistical Learning, 2nd EditionUber residual ETA model; learning curves and model assessment
1offline AUC vs online A/BMeta MSL 1Your ads CTR model shows a 2% offline AUC gain, but the online A/B is revenue-neutral with worse calibration. What is going on, and what do …Practical Lessons from Predicting Clicks on Ads at FacebookTrustworthy Online Controlled Experiments: A Practical Guide to A/B TestingCalibration and normalised entropy in ads; offline-online metric divergence
1Instagram Reels recommendationsMeta MSL 1Design the recommendation system for Instagram Reels.Scaling the Instagram Explore recommendations systemMachine Learning System Design InterviewInstagram Explore/Reels multi-stage funnel; video recommendation case
1news feed rankingMeta MSL 1Design a personalised news-feed ranking system / the “next post” logic for Facebook's feed.News Feed ranking, powered by machine learningMachine Learning System Design InterviewFeed multi-event value model; personalised news feed case
1Facebook Ads recommendationMeta MSL 1Design a recommendation system for Facebook Ads, and an evaluation framework for ads ranking.Practical Lessons from Predicting Clicks on Ads at FacebookDeep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)CTR model, calibration, evaluation metrics; DLRM ranking
1Instagram Story ML componentsMeta MSL 1Design the ML components behind an Instagram Story feature.Scaling the Instagram Explore recommendations systemNews Feed ranking, powered by machine learningMeta multi-stage ranking and multi-task event prediction applied to Stories
1Marketplace listing classificationMeta MSL 1Design an end-to-end classification pipeline for Marketplace listings.Designing Machine Learning SystemsMachine Learning System Design InterviewLabelling, class imbalance, evaluation pipeline; classification case studies
1translation model serviceMeta MSL 1Design a language translation model / service.No Language Left Behind: Scaling Human-Centered Machine TranslationNLLB data, model, evaluation and serving; GNMT as baseline
1harmful content detectionMeta MSL 1Design the harmful-content detection system for Facebook and Instagram uploads.Machine Learning System Design InterviewA Holistic Approach to Undesired Content Detection in the Real WorldHarmful-content detection case; rare-class labelling and calibration
1KL divergence continuousOpenAI 1Implement KL divergence for continuous distributions (formula provided; you must integrate it and interpret the result).Auto-Encoding Variational BayesProbabilistic Machine Learning: An IntroductionClosed-form Gaussian KL derivation; continuous KL definitions
1expected iterations probabilisticOpenAI 1Calculate the expected number of iterations for a probabilistic function.Statistics 110: Probability (lectures)CS109: Probability for Computer Scientists (Autumn 2026)Geometric distribution and first-step analysis for expected iterations
1cross-entropy minimum errorOpenAI 1Find the minimum error of a distribution using cross entropy (formula provided).Information Theory, Inference, and Learning AlgorithmsGibbs' inequality: cross-entropy minimised at the true distribution
1custom layer gradient derivationUnspecified frontier 1Derive the gradients for a specific custom layer (rather than "explain backprop").Derivatives, Backpropagation, and VectorizationBuilding makemore Part 4: Becoming a Backprop NinjaGeneralised Jacobians and manual backward for arbitrary layers
1DeepMind fundamentals quiz roundGoogle DeepMind 1Rapid-fire quiz on undergraduate fundamentals (formal linear algebra, calculus, probability definitions).Deep LearningProbability CheatsheetGoogle DeepMind Interview Notes (quizzes and design prompts with solutions)Goodfellow part I, cheatsheet, quiz-format practice
1noisy annotator labelsOpenAI 1Noisy Annotators (3 parts)OpenAI Interview Notes (problems with solutions)Confident Learning: Estimating Uncertainty in Dataset LabelsReference solution plus confident-learning noise estimation
1Las Vegas restart strategyOpenAI 1Restart Strategies under a Known Mean (Las Vegas) (6 oral parts)OpenAI Interview Notes (problems with solutions)Optimal Speedup of Las Vegas AlgorithmsRestart-strategy oral solution and Luby optimal restart result
1probability linear algebra quizGoogle DeepMind 1Maths Quiz: Probability, Statistics and Linear Algebra (12 questions / 45–60 min)Google DeepMind Interview Notes (quizzes and design prompts with solutions)Statistics 110: Probability (lectures)Full quiz with solutions; Stat 110 for the probability items
1matrix stats info theory quizGoogle DeepMind 1Maths Quiz: Matrix Computation, Statistical Tests and Information Theory (12 questions / 45–60 min)Google DeepMind Interview Notes (quizzes and design prompts with solutions)18.05 Introduction to Probability and Statistics (Spring 2022)Information Theory, Inference, and Learning AlgorithmsQuiz with solutions; 18.05 tests; MacKay information theory
1ML fundamentals quizGoogle DeepMind 1ML Fundamentals: Metrics, Losses, Optimisers and Regularisation (15 questions / 60 min)Google DeepMind Interview Notes (quizzes and design prompts with solutions)Introduction to Machine Learning Interviews BookFundamentals quiz solutions; Huyen metrics, losses and training chapters
1molecule pair reaction predictionGoogle DeepMind 1ML Design: Predicting a Reaction Factor for Pairs of Molecules (45–60 min)Google DeepMind Interview Notes (quizzes and design prompts with solutions)Neural Message Passing for Quantum ChemistryMolecule design reference solution; MPNN and scaffold splits
1AdamW decoupled weight decayMistral 1Explain Adam and AdamW. Why is adding an L2 penalty to the loss generally different from decoupled weight decay under Adam?Decoupled Weight Decay Regularization (AdamW)How to set AdamW's weight decay as you scale model and dataset sizeDecoupled decay derivation; how to set decay with scale
1diffusion forward reverseMistral 1Explain the forward and reverse processes of diffusion models.Denoising Diffusion Probabilistic ModelsMIT 6.S184: Introduction to Flow Matching and Diffusion Models (2026)DDPM processes; flow/SDE framing of forward and reverse
1NSFW output detectionOpenAI 1ML system design: design a system to detect NSFW content in ChatGPT outputs.A Holistic Approach to Undesired Content Detection in the Real WorldConstitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red TeamingModeration classifier design; streaming output classifiers on LLM text
1Bayes spam base rateOpenAI 1Probability: spam detector with 99% TPR, 2% FPR, 10% base rate — probability a flagged email is spam?CS109: Probability for Computer Scientists (Autumn 2026)Statistics 110: Probability (lectures)Bayes' rule with base rates; worked conditional-probability examples

All resources

Everything collected for this topic, newest first. 2026 items are marked green. ★ marks a top pick.

Type
Level
Subtopic

  • 2026
    CS109: Probability for Computer Scientists (Autumn 2026)
    Chris Piech et al., Stanford · Sep 2026 · Course · Core · ~25 h · video
    Current Stanford offering; probability through to MLE, MAP, Beta priors, CLT and bootstrapping, with an online course reader written for engineers.
    L7Baseline; use the course reader chapters on MLE/MAP and the bootstrap to check you can derive estimators on the spot.
    Probability for Computer Scientists (course reader)CS109 2022 lectures (YouTube)
  • 2026
    Google DeepMind Interview Notes (quizzes and design prompts with solutions)
    ShellyYin96 (GitHub) · Sep 2026 · Practice · Advanced · ~6 h
    Reconstructed DeepMind quiz rounds with full reference solutions: probability/linear algebra, matrix computation/statistical tests/information theory, ML fundamentals, and an ML design prompt.
    L7Lets you take realistic 45-60 minute DeepMind-style quizzes and compare against worked solutions; the closest proxy to the real round.
    Community-maintained; questions are reconstructions, not official.
    Maths quiz: probability, statistics, linear algebraMaths quiz: matrix computation, statistical tests, information theoryML fundamentals quizML design: molecule-pair reaction factor
  • 2026
    OpenAI Interview Notes (problems with solutions)
    Adam1679 (GitHub) · Sep 2026 · Practice · Advanced · ~6 h
    Reconstructed OpenAI problems with reference solutions, including Noisy Annotators (vote aggregation, filtering, weighted voting) and the Las Vegas restart-strategy math oral.
    L7Lets you practise multi-part OpenAI-style problems where the follow-ups get harder.
    Community-maintained; questions are reconstructions, not official.
    Noisy AnnotatorsRestart Strategies under a Known Mean (Las Vegas)
  • 2026
    ★Timed ML math oral (practice lab)
    mlmentorship (Hamidreza Saghir) · Aug 2026 · Practice · Core · ~3 h
    Eight timed oral derivation drills with observer follow-ups and a scoring rubric for assumptions, identities and sanity checks.
    L7Lets you rehearse the oral derivation format with a rubric; run it twice before a research loop.
    Frontier-lab interview processes overview
  • 2026
    EE364a: Convex Optimization I (Summer 2026)
    Stanford (Stephen Boyd; Summer 2026 taught by Logan Bell and Nikhil Devanathan) · Jun 2026 · Course · Advanced · ~25 h · video
    Current offering of Boyd's course with slides and CVXPY exercises; Boyd's own 2023 lectures are on YouTube.
    L7Lets you go deeper than most ML candidates on duality and second-order methods; take selectively.
    Stanford EE364A 2023 lectures (Boyd, YouTube)Original 2008 lecture collection (YouTube)
  • 2026
    Math notes (Alisa Liu)
    Alisa Liu · Jun 2026 · Reference · Core · ~3 h
    Personal study notes prepared for frontier-lab math interviews (probability, linear algebra, calculus); her job-search post describes the math round format.
    L7Lets you calibrate the scope of a recent frontier-lab math interview; use as a checklist.
    Notes on the Industry Job Search (format of the math round)
  • 2026
    MIT 6.S184: Introduction to Flow Matching and Diffusion Models (2026)
    Peter Holderrieth, Ezra Erives (MIT CSAIL) · Jan 2026 · Course · Advanced · ~12 h
    Self-contained notes, recorded lectures and labs deriving flow matching and diffusion from ODEs/SDEs and Fokker-Planck; 2026 version.
    L7Lets you present diffusion and flow matching in the current SDE/flow framing that labs use for image and video models.
    Lecture notes (arXiv)Stanford CS236 Deep Generative Models
  • 2026
    scikit-learn User Guide: Metrics and scoring
    scikit-learn developers · 2026 · Reference · Intro · ~2 h
    Precise definitions of precision, recall, F1, ROC-AUC, PR-AUC, log loss and calibration metrics, with the averaging variants.
    L7Baseline; skim if known.
  • 2026
    pandas User Guide: Working with missing data
    pandas developers · 2026 · Reference · Intro · ~1 h
    NA semantics, isna/fillna/dropna/interpolate and nullable dtypes in the current (3.x) docs.
    L7Baseline; pair it with a sentence on missing-at-random vs not-at-random to show statistical judgment.
  • 2026
    Algorithms for calculating variance (Welford's online algorithm)
    Wikipedia · 2026 · Reference · Core · ~1 h
    Explains numerically stable one-pass updates and why naive formulas cancel catastrophically; the same reasoning gives a streaming entropy update from running counts and sum c log c.
    L7Lets you design an O(1)-per-update, numerically stable streaming statistic (entropy, variance) and explain the precision failure modes.
    The Log-Sum-Exp Trick (Gregory Gundersen)
  • 2026
    Interviewing at Google DeepMind
    Google DeepMind · 2026 · Reference · Intro · ~0.5 h
    Official candidate guide describing interview stages and what each assesses, including technical knowledge rounds.
    L7Read once to confirm which rounds include fundamentals for your role.
    Undated PDF; linked from mlmentorship's August 2026 overview.
  • 2025
    CS229: Machine Learning
    Stanford (Andrew Ng, Tengyu Ma and others by term) · Sep 2025 · Course · Core · ~40 h · video
    Stanford's core ML course: GLMs, generative vs discriminative models, kernels, bias-variance, EM, PCA/ICA, with the canonical lecture notes; the latest term with indexed pages is Fall 2025, latest video set is Spring 2022.
    L7Baseline; use the main notes for clean MLE/GLM and EM derivations, and skip lectures you already know.
    Autumn 2018 lectures, Andrew Ng (YouTube)Spring 2022 lectures (YouTube)Summer 2019 lectures, Anand Avati (YouTube)
  • 2025
    CS230: Deep Learning (Autumn 2025)
    Andrew Ng, Kian Katanforoosh (Stanford) · Sep 2025 · Course · Core · ~20 h · video
    Stanford's applied deep-learning course; Autumn 2025 lectures are on YouTube, covering initialisation, Adam, dropout, batch norm and project strategy.
    L7Baseline; watch only the lectures on practical training strategy and error analysis.
    Autumn 2025 lectures (YouTube)Autumn 2018 lectures (YouTube)Deep Learning Specialization (Coursera)
  • 2025
    CS231n notes: Backpropagation, Intuitions
    Stanford CS231n (Andrej Karpathy et al.) · Apr 2025 · Tutorial · Core · ~2 h
    Computational-graph view of backprop, gate-level patterns, and the matrix-multiply backward rule; the course itself is running Spring 2026 with 2025 lectures on YouTube.
    L7Baseline; the gate patterns let you sanity-check any custom-layer gradient you derive aloud.
    Notes are long-lived; date is the latest recorded offering on YouTube.
    CS231n course site (Spring 2026)CS231n 2025 lectures (YouTube)
  • 2025
    Transformers without Normalization
    Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, Zhuang Liu (Meta FAIR et al.) · Mar 2025 · Paper · Advanced · ~1.5 h
    Replaces LayerNorm with Dynamic Tanh (DyT) and matches performance, challenging what normalisation is for.
    L7Lets you discuss what normalisation actually provides (bounded activations, gradient conditioning) using a 2025 counterexample.
  • 2025
    Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
    Mrinank Sharma et al. (Anthropic) · Jan 2025 · Paper · Advanced · ~2 h
    Input and streaming output classifiers trained on constitution-generated synthetic data, with over-refusal and compute overhead measured.
    L7Lets you discuss classifier cascades on LLM outputs with real false-positive and overhead numbers from a frontier lab.
  • 2024
    Muon: An optimizer for hidden layers in neural networks
    Keller Jordan · Dec 2024 · Blog post · Advanced · ~1.5 h
    Orthogonalised momentum via Newton-Schulz iteration, i.e. replacing the update with its nearest semi-orthogonal matrix from the SVD; now used in frontier-scale training.
    L7Lets you discuss the current optimiser frontier (spectral-norm steepest descent) and tie it back to SVD, which few candidates can.
    Muon is Scalable for LLM Training (Moonshot AI, 2025)Old Optimizer, New Norm: An Anthology (Bernstein & Newhouse, 2024)
  • 2024
    ★Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
    Evan Miller (Anthropic) · Nov 2024 · Paper · Advanced · ~2 h
    Frontier-lab paper on standard errors, clustered errors, paired differences and power analysis for model evals; directly answers '85% vs 82%, which model?'.
    L7Lets you answer model-comparison questions with paired tests, clustered standard errors and a power calculation, as a frontier lab expects.
    Anthropic blog summary
  • 2024
    SOAP: Improving and Stabilizing Shampoo using Adam
    Nikhil Vyas et al. (Harvard / Kempner) · Sep 2024 · Paper · Advanced · ~2 h
    Connects Shampoo's Kronecker-factored preconditioner to Adam run in Shampoo's eigenbasis; clear view of second-order methods at scale.
    L7Lets you compare diagonal vs Kronecker-factored preconditioning and their compute/memory cost.
    Shampoo: Preconditioned Stochastic Tensor Optimization (Gupta, Koren, Singer, 2018)
  • 2024
    6.7960 Deep Learning (Fall 2024)
    Phillip Isola, Sara Beery, Jeremy Bernstein (MIT) · Sep 2024 · Course · Advanced · ~25 h
    Graduate deep learning course with lecture notes and slides covering optimisation, scaling, generalisation, and the geometry of updates.
    L7Lets you discuss deep-learning optimisation and generalisation at graduate depth; pick lectures on optimisation and scaling.
  • 2024
    Why Warmup the Learning Rate? Underlying Mechanisms and Improvements
    Dayal Singh Kalra, Maissam Barkeshli · Jun 2024 · Paper · Advanced · ~2 h
    Empirical and theoretical account of warmup as sharpness reduction, connected to edge-of-stability dynamics.
    L7Lets you explain warmup mechanistically rather than as folklore.
  • 2024
    How to set AdamW's weight decay as you scale model and dataset size
    Xi Wang, Laurence Aitchison · May 2024 · Paper · Advanced · ~2 h
    Interprets AdamW as an exponential moving average of updates and derives how the decay timescale should scale with dataset and model size.
    L7Lets you argue how weight decay should change when scaling runs, which is a natural L7 follow-up to 'Adam vs AdamW'.
  • 2024
    Machine Learning Q and AI: 30 Essential Questions and Answers
    Sebastian Raschka (No Starch Press) · Mar 2024 · Book · Core · ~6 h · paid
    Thirty short essays on questions interviewers ask beyond basics: evaluation, confidence intervals for model comparison, data-centric AI, distribution shift.
    L7Lets you give polished two-minute answers on evaluation and model comparison.
    Paid; much of the content is free on the author's site.
    No Starch Press page
  • 2024
    Let's build the GPT Tokenizer
    Andrej Karpathy · Feb 2024 · Video · Core · ~2 h
    Builds byte-level BPE from scratch and walks through tokenisation failure modes in LLMs.
    L7Baseline for the 'tokenization' part of a DeepMind ML round; know how tokenisation affects perplexity comparisons.
  • 2024
    Brainstellar: Puzzles from Quant Interviews
    Brainstellar · 2024 · Practice · Core · ~5 h
    Free graded probability and logic puzzles with hints and solutions; good for 10-minute daily reps.
    L7Baseline reps; use for the 'rapid-fire out loud' drill.
  • 2024
    Deep Learning: Foundations and Concepts
    Christopher M. Bishop, Hugh Bishop (Springer) · 2024 · Book · Core · ~30 h
    Modern successor to PRML covering probability, single-layer networks, backprop, normalisation, transformers, VAEs, diffusion and flows; solutions for chapters 2-10 are posted.
    L7Lets you derive the main deep-learning objectives (cross-entropy, ELBO, diffusion loss) in one consistent notation; chapter 20 on diffusion is interview-ready.
    Free online reading; print paid.
  • 2024
    Learning Theory from First Principles
    Francis Bach (MIT Press) · 2024 · Book · Advanced · ~20 h
    Free PDF; rigorous yet compact treatment of least squares, generalisation bounds, kernels, convex optimisation for ML and neural networks.
    L7Lets you give theory-grounded answers on generalisation and optimisation rates when a research interviewer probes.
  • 2023
    ★Understanding Deep Learning
    Simon J. D. Prince (MIT Press) · Dec 2023 · Book · Core · ~25 h
    Free PDF with Python notebooks, actively corrected (repo updated October 2026); clear chapters on loss functions from maximum likelihood, initialisation, regularisation, transformers and diffusion.
    L7Lets you explain why each loss follows from a likelihood and derive initialisation variance conditions; the diffusion chapter is a strong 20-minute refresher.
    GitHub (notebooks, errata)
  • 2023
    Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
    Hakan Inan et al. (Meta) · Dec 2023 · Paper · Advanced · ~1.5 h
    Uses an instruction-tuned LLM as a policy-conditioned classifier for prompts and responses.
    L7Lets you argue when to use a small classifier vs an LLM judge (latency, cost, policy flexibility) in a moderation design.
  • 2023
    Small-scale proxies for large-scale Transformer training instabilities
    Mitchell Wortsman et al. (Google DeepMind) · Sep 2023 · Paper · Advanced · ~3 h
    Reproduces attention-logit growth and output-logit divergence at small scale and tests fixes (QK-layernorm, z-loss, warmup, weight decay, muP).
    L7Lets you discuss real large-run instabilities and their fixes with evidence, the kind of answer L7 research loops reward.
  • 2023
    Scaling the Instagram Explore recommendations system
    Meta Engineering · Aug 2023 · Blog post · Advanced · ~1 h
    Meta's multi-stage retrieval and ranking stack (two-tower retrieval, distilled early-stage ranker, multi-task late-stage ranker) for Explore and Reels-style surfaces.
    L7Lets you quote a real hyperscale funnel and its stage-wise trade-offs when designing Reels or Stories recommendations.
    Friend Bubbles: Enhancing Social Discovery on Facebook Reels (2026)
  • 2023
    Benchmarking Neural Network Training Algorithms (AlgoPerf)
    George E. Dahl et al. (Google, MLCommons) · Jun 2023 · Paper · Advanced · ~3 h
    How to compare optimisers fairly (time-to-result, workload variants, tuning budgets), and why most published optimiser gains disappear under careful comparison.
    L7Lets you critique an optimiser claim the way a research lead would: tuning budget, baselines, and workload coverage.
  • 2023
    Deep Learning Tuning Playbook
    Varun Godbole, George E. Dahl, Justin Gilmer, Christopher J. Shallue, Zachary Nado (Google) · Jan 2023 · Reference · Advanced · ~3 h
    Google Brain's process for choosing batch size, optimiser and learning-rate schedules and for running hyperparameter studies.
    L7Lets you describe a principled tuning process with scientific/nuisance/fixed hyperparameters, an L7 framing of 'how would you train this'.
  • 2023
    Machine Learning System Design Interview
    Ali Aminian, Alex Xu (ByteByteGo) · Jan 2023 · Book · Core · ~10 h · paid
    Worked designs for harmful content detection, video recommendation, ad click prediction, similar rental listings, personalised news feed and people-you-may-know.
    L7Baseline templates; an L7 answer must go beyond them with trade-offs, scale numbers and failure modes.
    Paid (ByteByteGo subscription or print).
  • 2023
    Heard on the Street: Quantitative Questions from Wall Street Job Interviews
    Timothy Falcon Crack (self-published) · 2023 · Practice · Core · ~8 h · paid
    Long-running bank of quant interview questions (probability, statistics, brain teasers) with worked answers.
    L7Skim the quantitative and probability sections for extra rapid-fire reps; lower priority than Stat 110 and the green book.
    Self-published, revised near-annually (24th ed. 2023 per author's university listing); no stable publisher page, Goodreads used as reference.
  • 2023
    An Introduction to Statistical Learning (with Applications in Python)
    Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor · 2023 · Book · Core · ~15 h
    Free PDF with Python labs; chapters 3 (linear regression and its assumptions), 5 (resampling) and 6 (regularisation) answer the classic stats-flavoured ML questions.
    L7Baseline; read chapter 3.3 and 6 to answer assumption and regularisation questions crisply.
    Statistical Learning with Python lectures (Stanford, YouTube)
  • 2023
    Probabilistic Machine Learning: Advanced Topics
    Kevin P. Murphy (MIT Press) · 2023 · Book · Advanced · ~30 h
    Free draft (updated 2025-12) on inference, VAEs, diffusion, flows, RL and causality; reference depth for research-loop follow-ups.
    L7Use selectively for generative-model and variational-inference derivations at research-scientist depth.
  • 2023
    Dive into Deep Learning
    Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola (Cambridge University Press) · 2023 · Book · Core · ~25 h
    Interactive book with runnable PyTorch/JAX code; softmax regression, weight decay, optimisation algorithms and language-model/perplexity chapters are directly reusable for interviews.
    L7Baseline; use specific sections to confirm definitions (perplexity, weight decay) and code them from memory.
    4.1 Softmax Regression (cross-entropy, loss bounds)9.3 Language Models (perplexity)3.7 Weight Decay12. Optimization Algorithms
  • 2022
    ★Building makemore Part 4: Becoming a Backprop Ninja
    Andrej Karpathy · Oct 2022 · Video · Advanced · ~3 h · video
    Manually backpropagates through cross-entropy, a linear layer, tanh and BatchNorm in PyTorch, checking each gradient against autograd; the closest thing to rehearsing the derivation round.
    L7Lets you derive fused softmax-cross-entropy and normalisation backward passes by hand and explain the numerically stable fused form.
    Neural Networks: Zero to Hero (series)Building micrograd (backprop from scratch)
  • 2022
    Understanding Diffusion Models: A Unified Perspective
    Calvin Luo (Google Research) · Aug 2022 · Survey paper · Advanced · ~4 h
    Step-by-step derivation from ELBO and hierarchical VAEs to DDPM, with the three equivalent parameterisations (x0, noise, score).
    L7Lets you move between noise-prediction, score and x0 views and explain why they are equivalent, a common research-loop probe.
  • 2022
    A Holistic Approach to Undesired Content Detection in the Real World
    Todor Markov et al. (OpenAI) · Aug 2022 · Paper · Advanced · ~2 h
    OpenAI's moderation classifier: taxonomy design, active learning for rare categories, synthetic data and calibration across distribution shift.
    L7Lets you design an NSFW/harmful-content detector with a real lab's lessons on rare-class labelling and threshold setting.
  • 2022
    No Language Left Behind: Scaling Human-Centered Machine Translation
    NLLB Team (Meta AI) · Jul 2022 · Paper · Advanced · ~4 h
    End-to-end account of a 200-language translation system: data mining, mixture-of-experts model, evaluation and safety (toxicity) checks.
    L7Lets you design a translation service with real data, evaluation and serving trade-offs from Meta's own system.
    Google's Neural Machine Translation System (Wu et al., 2016)
  • 2022
    Machine Learning Specialization
    Andrew Ng, DeepLearning.AI and Stanford Online (Coursera) · Jun 2022 · Course · Intro · ~30 h · paid · video
    Gentle three-course refresh of regression, classification, trees, clustering, recommenders and RL basics.
    L7Skim only; too basic for this reader except as a fast refresher of vocabulary.
    Audit is free; certificate paid.
  • 2022
    DeeprETA: An ETA Post-processing System at Scale
    Xinyu Hu et al. (Uber) · Jun 2022 · Paper · Advanced · ~1.5 h
    Uber's production ETA model that learns residuals on top of a routing engine, with feature design and latency constraints.
    L7Lets you argue data sufficiency for ETA (residual learning, coverage of rare routes, learning curves) with a real system.
  • 2022
    Designing Machine Learning Systems
    Chip Huyen (O'Reilly) · May 2022 · Book · Core · ~12 h · paid
    Covers data, labelling (including weak and noisy labels), feature engineering, evaluation, deployment and monitoring; the backbone framework for any ML design question.
    L7Lets you structure an ML design answer end to end; add your own Meta numbers to reach L7 depth.
    Paid book; CS329S notes it grew from are free.
    Stanford CS329S: Machine Learning Systems Design (2022)
  • 2022
    Probabilistic Machine Learning: An Introduction
    Kevin P. Murphy (MIT Press) · Mar 2022 · Book · Core · ~30 h
    Free draft (updated 2025-04) covering probability, information theory, linear/logistic models, optimisation and deep nets with code.
    L7Lets you look up a rigorous, current derivation for almost any fundamentals question; chapters 2-6 and 8 are the useful ones.
  • 2022
    ★Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (muP)
    Greg Yang, Edward J. Hu et al. (Microsoft, OpenAI) · Mar 2022 · Paper · Advanced · ~4 h
    Maximal-update parametrisation: how to scale init and per-layer learning rates with width so hyperparameters transfer from small to large models.
    L7Lets you explain how frontier labs tune hyperparameters on small proxies and why standard parametrisation breaks at scale.
    A Spectral Condition for Feature Learning (Yang, Simon, Bernstein, 2023)Scaling Exponents Across Parameterizations and Optimizers (Everett et al., Google DeepMind, 2024)
  • 2022
    18.05 Introduction to Probability and Statistics (Spring 2022)
    Jeremy Orloff, Jonathan Bloom, Jennifer French Kamrin, MIT OpenCourseWare · Feb 2022 · Course · Core · ~20 h
    Class notes, problem sets and exams covering Bayesian updating, frequentist hypothesis tests (z, t, chi-square), confidence intervals and bootstrap; the right depth for 'rapid-fire stats'.
    L7Baseline; work the hypothesis-testing and confidence-interval readings until you can state assumptions and failure modes without notes.
  • 2022
    Deep Learning Interviews: Hundreds of fully solved job interview questions
    Shlomo Kashani, Amir Ivry · Jan 2022 · Practice · Core · ~10 h
    Hundreds of solved problems on information theory, Bayesian statistics, calculus and backprop, and deep-learning fundamentals.
    L7Good volume for the rapid-fire drill; pick the information-theory and backprop chapters.
  • 2021
    What are Diffusion Models?
    Lilian Weng · Jul 2021 · Blog post · Core · ~1.5 h
    Compact derivations of forward/reverse processes, score matching, DDIM and guidance, regularly updated.
    L7Baseline refresher for the Mistral-style 'explain forward and reverse' question.
  • 2021
    LoRA: Low-Rank Adaptation of Large Language Models
    Edward Hu et al. (Microsoft) · Jun 2021 · Paper · Core · ~2 h
    A concrete modern place where low-rank matrix factorisation shows up in deep learning; good second example when asked 'where does SVD appear'.
    L7Lets you connect rank arguments to a real fine-tuning design choice and discuss when low rank is not enough.
  • 2021
    The Principles of Deep Learning Theory
    Daniel A. Roberts, Sho Yaida, Boris Hanin (Cambridge University Press) · Jun 2021 · Book · Advanced · ~20 h
    Effective-theory treatment of wide networks: criticality, initialisation, and why depth-to-width ratio matters.
    L7Lets you defend initialisation choices with criticality arguments; read chapters 1-5 only if targeting research-scientist loops.
  • 2021
    System Design for Recommendations and Search
    Eugene Yan · Jun 2021 · Blog post · Core · ~1 h
    Distils retrieval/ranking architectures from Alibaba, Facebook, JD, DoorDash and others into an offline/online, candidate-retrieval/ranking template.
    L7Lets you sketch a two-stage recsys quickly and spend interview time on trade-offs.
  • 2021
    News Feed ranking, powered by machine learning
    Akos Lada, Meihong Wang, Tak Yan (Meta Engineering) · Jan 2021 · Blog post · Core · ~1 h
    How Feed predicts multiple engagement events per post and combines them into a value score.
    L7Lets you describe multi-objective ranking with weighted event probabilities, the core of a feed-ranking design.
  • 2021
    Deep learning theory lecture notes
    Matus Telgarsky · 2021 · Reference · Advanced · ~10 h
    Graduate notes on approximation, optimisation (NTK) and generalisation theory for neural networks.
    L7Reference for theory follow-ups; skim the NTK and generalisation sections only.
    Neural Tangent Kernel (Jacot et al., 2018)STATS214 / CS229M Machine Learning Theory (Tengyu Ma)
  • 2021
    ★Introduction to Machine Learning Interviews Book
    Chip Huyen · 2021 · Practice · Core · ~10 h
    Free question bank: chapter 5 (math: linear algebra, calculus, probability, stats), 7 (sampling, objectives, metrics) and 8 (classical ML, training neural networks).
    L7Baseline question bank; use it to build the 40-question self-quiz, then answer at senior depth with trade-offs.
    Chapter 5: Math5.2.1.2 Probability questions7.3 Objective functions, metrics, and evaluation8.1.2 Classical ML questions8.3 Training neural networks
  • 2020
    Normalization Techniques in Training DNNs: Methodology, Analysis and Application
    Lei Huang, Jie Qin, Yi Zhou, Fan Zhu, Li Liu, Ling Shao · Sep 2020 · Survey paper · Advanced · ~4 h
    Survey unifying BN, LN, GN, weight normalisation and whitening under one framework, with the optimisation analyses.
    L7Lets you compare normalisation methods by what they normalise over and their compute/batch constraints.
  • 2020
    TransNet V2: An effective deep network architecture for fast shot transition detection
    Tomáš Souček, Jakub Lokoč · Aug 2020 · Paper · Core · ~1 h
    Shot-boundary detection with dilated 3D CNNs; the natural candidate-generation step for choosing ad-break positions in video.
    L7Lets you ground a 'commercial break timing' design in a real temporal segmentation model, then layer user-retention objectives on top.
  • 2020
    Denoising Diffusion Probabilistic Models
    Jonathan Ho, Ajay Jain, Pieter Abbeel · Jun 2020 · Paper · Advanced · ~3 h
    Defines the Gaussian forward process, the closed-form q(x_t | x_0), and the simplified noise-prediction loss derived from the ELBO.
    L7Lets you derive the forward marginal and the training objective, not just describe 'add noise, then remove it'.
  • 2020
    Embedding-based Retrieval in Facebook Search
    Jui-Ting Huang et al. (Facebook) · Jun 2020 · Paper · Advanced · ~2 h
    Production two-tower retrieval with hard-negative mining, ANN serving and hybrid term/embedding retrieval to raise recall.
    L7Lets you discuss recall improvements concretely: negative sampling, ANN recall vs latency, and blending with lexical retrieval.
  • 2020
    On Layer Normalization in the Transformer Architecture
    Ruibin Xiong et al. (Microsoft Research) · Feb 2020 · Paper · Advanced · ~2 h
    Mean-field analysis showing why Post-LN transformers need warmup and Pre-LN do not.
    L7Lets you link normalisation placement, gradient scale at initialisation and warmup in one argument.
  • 2020
    Mathematics for Machine Learning
    Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong (Cambridge University Press) · 2020 · Book · Core · ~25 h
    Free book covering linear algebra, matrix decompositions, vector calculus, probability, and optimisation, with worked PCA, linear regression, GMM and SVM derivations.
    L7Baseline reference; use chapters 4-6 to check your derivations are written with correct notation and assumptions.
  • 2020
    Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing
    Ron Kohavi, Diane Tang, Ya Xu (Cambridge University Press) · 2020 · Book · Advanced · ~12 h · paid
    The practitioner reference on A/B tests from Microsoft, Google and LinkedIn: sample ratio mismatch, variance reduction, guardrails, and why offline and online metrics diverge.
    L7Lets you explain an offline-AUC-up, online-flat result with concrete causes (novelty, interference, calibration, metric sensitivity) and design the follow-up experiment.
    Paid book; many chapters' source papers free on exp-platform.com.
    ExP Platform (Kohavi's papers and talks)
  • 2019
    Optimization for deep learning: theory and algorithms
    Ruoyu Sun · Dec 2019 · Survey paper · Advanced · ~5 h
    Survey of why deep nets are hard to optimise: gradient explosion and vanishing, initialisation, normalisation, adaptive methods and landscape results.
    L7Lets you connect initialisation, normalisation and optimiser choice into one argument about trainability.
  • 2019
    Deep Double Descent: Where Bigger Models and More Data Hurt
    Preetum Nakkiran et al. (Harvard, OpenAI) · Dec 2019 · Paper · Advanced · ~2 h
    Shows test error can fall, rise, then fall with model size, data or epochs, complicating the textbook bias-variance picture.
    L7Lets you give the modern, nuanced answer to 'explain bias-variance' instead of only the U-curve.
    Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability (Cohen et al., 2021)
  • 2019
    Confident Learning: Estimating Uncertainty in Dataset Labels
    Curtis G. Northcutt, Lu Jiang, Isaac L. Chuang · Oct 2019 · Paper · Advanced · ~2 h
    Estimates the joint distribution of noisy and true labels to find and prune label errors; cleanlab implements it.
    L7Lets you go beyond majority vote: estimate annotator noise, prune or reweight, and measure the gain.
    cleanlab (GitHub)Learning from Noisy Labels with Deep Neural Networks: A Survey (Song et al., 2020)
  • 2019
    Deep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)
    Maxim Naumov et al. (Facebook) · May 2019 · Paper · Advanced · ~2 h
    Meta's reference ranking architecture: embedding tables for sparse features, MLPs, and explicit feature interactions, with model/data parallelism.
    L7Lets you size embedding tables and explain why recsys ranking is memory-bound, a scale argument L7 interviewers expect.
  • 2019
    A Recipe for Training Neural Networks
    Andrej Karpathy · Apr 2019 · Blog post · Core · ~1 h
    Practitioner's process for training: inspect data, build baselines, overfit one batch, regularise, tune; the answer to 'how does training a CNN/transformer actually work'.
    L7Lets you describe a disciplined debugging process; add numbers from your own runs to make it senior.
  • 2018
    ★18.065 Matrix Methods in Data Analysis, Signal Processing, and Machine Learning (Spring 2018)
    Gilbert Strang, MIT OpenCourseWare · Feb 2018 · Course · Advanced · ~30 h · video
    Low-rank approximation, randomized linear algebra, SVD/PCA, and the linear algebra of gradient descent and deep nets; especially useful if you come from signal processing.
    L7Lets you go past 'define the SVD' to Eckart-Young, conditioning and why low-rank structure (LoRA, Muon orthogonalisation) works, which is the follow-up L7 interviewers push on.
    Lecture videos (YouTube)Linear Algebra and Learning from Data (book page)
  • 2018
    The Matrix Calculus You Need For Deep Learning
    Terence Parr, Jeremy Howard · Feb 2018 · Tutorial · Core · ~4 h
    Builds Jacobians, element-wise and vector chain rules from scratch and applies them to a neuron's gradient; ideal warm-up before deriving layers by hand.
    L7Baseline; read once so Jacobian shapes are automatic when you derive under time pressure.
    arXiv version
  • 2018
    On the Convergence of Adam and Beyond
    Sashank J. Reddi, Satyen Kale, Sanjiv Kumar (Google) · 2018 · Paper · Advanced · ~2 h
    ICLR 2018 best paper: a counterexample where Adam fails to converge, and the AMSGrad fix.
    L7Lets you discuss Adam's failure modes and second-moment dynamics instead of treating it as a black box.
  • 2018
    Reinforcement Learning: An Introduction (2nd ed.)
    Richard S. Sutton, Andrew G. Barto (MIT Press) · 2018 · Book · Core · ~10 h
    Free PDF; chapter 6 (Q-learning) and chapter 13 (policy gradients) give the value-based vs policy-based trade-off.
    L7Lets you compare Q-learning and policy gradients on action spaces, variance, sample efficiency and off-policy data.
    OpenAI Spinning Up in Deep RL
  • 2017
    ★Decoupled Weight Decay Regularization (AdamW)
    Ilya Loshchilov, Frank Hutter · Nov 2017 · Paper · Advanced · ~2 h
    Shows L2 regularisation and weight decay are equivalent for SGD but not for adaptive methods, and introduces decoupled decay.
    L7Lets you show algebraically why an L2 penalty gets rescaled by Adam's preconditioner while decoupled decay does not, and what that does to hyperparameter coupling.
  • 2017
    ★Derivatives, Backpropagation, and Vectorization
    Justin Johnson (Stanford CS231n handout) · Sep 2017 · Reference · Core · ~1.5 h
    Short handout on generalised Jacobians for tensor inputs and how to backprop through them without materialising huge Jacobians; the exact skill a 'derive this custom layer' question checks.
    L7Lets you derive a batched layer's backward pass in vectorised form and argue its memory cost, not just the scalar case.
    Backpropagation for a Linear Layer (Johnson)Vector, Matrix, and Tensor Derivatives (Erik Learned-Miller)
  • 2017
    A Unified Approach to Interpreting Model Predictions (SHAP)
    Scott M. Lundberg, Su-In Lee · May 2017 · Paper · Core · ~1.5 h
    Shapley-value feature attributions that only need model queries; counterfactual explanations (linked) answer 'what would change the decision'.
    L7Lets you explain a loan rejection with black-box attributions and counterfactuals, and state their limitations.
    Counterfactual Explanations without Opening the Black Box (Wachter et al., 2017)Interpretable Machine Learning (Christoph Molnar)
  • 2017
    Neural Message Passing for Quantum Chemistry
    Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, George E. Dahl (Google) · Apr 2017 · Paper · Advanced · ~2 h
    Unifies graph neural networks for molecules as message passing; the standard representation for molecular property prediction.
    L7Lets you propose a permutation-invariant molecule-pair model and discuss scaffold splits for generalisation.
    MoleculeNet: A Benchmark for Molecular Machine Learning (scaffold splits)Stanford CS224W: Machine Learning with Graphs (YouTube)
  • 2017
    Rules of Machine Learning: Best Practices for ML Engineering
    Martin Zinkevich (Google) · 2017 · Reference · Core · ~1.5 h
    Forty-three rules on launching heuristics first, objectives, feature engineering, training-serving skew and when to add complexity.
    L7Lets you justify (or reject) a neural network and choose what not to build, which reads as senior judgment.
  • 2016
    Deep Neural Networks for YouTube Recommendations
    Paul Covington, Jay Adams, Emre Sargin (Google) · Sep 2016 · Paper · Advanced · ~1.5 h
    Classic candidate-generation plus ranking design, with watch-time weighted logistic regression and the example-age feature.
    L7Lets you discuss label choice (clicks vs watch time) and train/serve skew from a canonical system.
  • 2016
    Layer Normalization
    Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton · Jul 2016 · Paper · Core · ~1.5 h
    Defines LayerNorm and its invariance properties; RMSNorm (linked) is what most current LLMs use.
    L7Lets you derive LayerNorm/RMSNorm backward and say why transformers use per-token normalisation.
    Root Mean Square Layer Normalization (Zhang & Sennrich, 2019)
  • 2016
    Optimization Methods for Large-Scale Machine Learning
    Léon Bottou, Frank E. Curtis, Jorge Nocedal · Jun 2016 · Survey paper · Advanced · ~6 h
    SIAM Review survey of SGD theory, noise reduction, and second-order methods for ML, with convergence results stated cleanly.
    L7Lets you state SGD convergence rates and the role of gradient noise and batch size precisely.
  • 2016
    An overview of gradient descent optimization algorithms
    Sebastian Ruder · Jan 2016 · Blog post · Core · ~1.5 h
    Compact derivation of momentum, Nesterov, Adagrad, RMSprop and Adam update rules side by side.
    L7Baseline; skim if known.
    arXiv version
  • 2016
    Essence of Linear Algebra
    Grant Sanderson (3Blue1Brown) · 2016 · Video · Intro · ~3 h · video
    Sixteen short visual lessons on linear maps, determinants, change of basis and eigenvectors; the quickest way to rebuild geometric intuition before oral questions.
    L7Skim only; baseline intuition you will be expected to have.
    YouTube playlistNeural networks series (backprop calculus episodes)
  • 2016
    Deep Learning
    Ian Goodfellow, Yoshua Bengio, Aaron Courville (MIT Press) · 2016 · Book · Core · ~20 h
    Free HTML; part I (linear algebra, probability, information theory, numerical computation) and chapters 7-8 (regularisation, optimisation) are still the cleanest summaries.
    L7Baseline; part I is a good self-quiz source for the DeepMind-style fundamentals quiz.
    Older but canonical.
  • 2015
    Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
    Sergey Ioffe, Christian Szegedy (Google) · Feb 2015 · Paper · Core · ~2 h
    Original BN with its backward-pass equations and train/inference behaviour.
    L7Baseline; know the backward pass and the batch-dependence failure modes (small batches, distributed BN).
  • 2015
    Delving Deep into Rectifiers (He initialization)
    Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research) · Feb 2015 · Paper · Core · ~1.5 h
    Derives the variance-preserving initialisation for ReLU nets; Glorot & Bengio (linked) gives the tanh/linear case.
    L7Lets you derive the 2/fan_in condition on paper, a common on-the-spot derivation.
    Understanding the difficulty of training deep feedforward neural networks (Glorot & Bengio, 2010)
  • 2015
    Probability Cheatsheet
    William Chen, Joe Blitzstein (Stat 110) · 2015 · Reference · Intro · ~1 h
    Ten-page summary of Stat 110: distributions, moments, conditional expectation, inequalities; the sheet to review before a rapid-fire round.
    L7Baseline; skim if known.
  • 2014
    Adam: A Method for Stochastic Optimization
    Diederik P. Kingma, Jimmy Ba · Dec 2014 · Paper · Core · ~2 h
    The original update rule, bias correction derivation, and the scale-invariance argument for the effective step size.
    L7Lets you derive bias correction from first principles and explain epsilon's role, which interviewers push on after the update rule.
  • 2014
    Dropout: A Simple Way to Prevent Neural Networks from Overfitting
    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov · 2014 · Paper · Core · ~2 h
    Dropout as model averaging, the inverted-dropout scaling, and its relation to L2 regularisation in linear models.
    L7Baseline; know the train/test scaling and the ensemble interpretation.
    Regularization for Deep Learning: A Taxonomy (Kukačka et al., 2017)
  • 2014
    Practical Lessons from Predicting Clicks on Ads at Facebook
    Xinran He et al. (Facebook) · 2014 · Paper · Advanced · ~1.5 h
    Boosted trees plus logistic regression for CTR, normalised entropy and calibration as metrics, data freshness and online learning.
    L7Lets you explain why AUC gains need not move revenue: calibration and normalised entropy matter for auctions.
    Author-hosted PDF (Joaquin Quiñonero Candela).
    Ad Click Prediction: a View from the Trenches (McMahan et al., Google, 2013)
  • 2013
    Auto-Encoding Variational Bayes
    Diederik P. Kingma, Max Welling · Dec 2013 · Paper · Advanced · ~2 h
    Derives the ELBO and the reparameterisation trick; appendix B gives the closed-form KL between Gaussians.
    L7Lets you derive KL between Gaussians and the ELBO on paper, then discuss estimator variance.
    An Introduction to Variational Autoencoders (Kingma & Welling, 2019)
  • 2013
    ★Statistics 110: Probability (lectures)
    Joe Blitzstein, Harvard University · 2013 · Course · Core · ~30 h · video
    Thirty-five lectures on conditioning, expectation via indicators, first-step analysis, Markov chains and limit theorems, with strategic-practice problems; the best single source for probability puzzles.
    L7Lets you solve expectation puzzles with first-step analysis and indicators in one pass; aim to answer Stat 110 strategic-practice problems aloud in under 5 minutes.
    Lectures recorded 2013; book 2nd edition 2019.
    Introduction to Probability, 2nd ed. (free PDF, Blitzstein & Hwang)Stat110x on edXCourse site with strategic practice and solutions
  • 2012
    The Matrix Cookbook
    Kaare Brandt Petersen, Michael Syskind Pedersen (DTU) · Nov 2012 · Reference · Advanced · ~1 h
    Lookup tables for matrix derivatives, traces, determinants, inverses and Gaussian identities; use it to check results, not to learn.
    L7Lets you verify trace/log-det derivatives (Gaussian MLE, KL between multivariate Gaussians) quickly while preparing.
    PDF mirror (Waterloo)
  • 2011
    18.06SC Linear Algebra (Fall 2011)
    Gilbert Strang, MIT OpenCourseWare · Sep 2011 · Course · Core · ~35 h · video
    Strang's lectures plus recitations and problem sets with solutions; fastest path back to fluent rank, projections, eigen and SVD reasoning for a pen-and-paper round.
    L7Baseline; do only the lectures you miss in the self-quiz (projections, least squares, SVD, positive definiteness).
    Older but canonical; the 6th edition textbook (2023) is the current companion.
    18.06 lecture videos (YouTube, 2005 recordings)A 2020 Vision of Linear Algebra (Strang, short refresher)Introduction to Linear Algebra, 6th ed. (2023)
  • 2009
    The Elements of Statistical Learning, 2nd Edition
    Trevor Hastie, Robert Tibshirani, Jerome Friedman (Springer) · 2009 · Book · Advanced · ~20 h
    Free PDF; chapter 7 (model assessment, bias-variance, cross-validation, effective degrees of freedom) is the definitive treatment.
    L7Lets you discuss bias-variance and model selection with the actual decomposition and optimism-of-training-error results, not slogans.
    Older but canonical.
  • 2008
    A Practical Guide to Quantitative Finance Interviews ("the green book")
    Xinfeng Zhou · 2008 · Practice · Advanced · ~15 h · paid
    Quant-interview problem bank: brain teasers, probability, expectation and Markov-chain puzzles, linear algebra and calculus, with terse solutions.
    L7Lets you drill harder expectation and stopping-time puzzles under time pressure; do the probability chapter only.
    Paid; solutions are terse.
  • 2006
    Elements of Information Theory, 2nd Edition
    Thomas M. Cover, Joy A. Thomas (Wiley) · 2006 · Reference · Advanced · ~10 h · paid
    Standard reference for rigorous definitions and proofs (Gibbs' inequality, data processing, chain rules).
    L7Use only to check a proof (KL >= 0, MI as KL); reading cover to cover is not worth the time.
    Paid.
  • 2006
    Pattern Recognition and Machine Learning
    Christopher M. Bishop (Springer) · 2006 · Book · Advanced · ~25 h
    Free PDF from Microsoft Research; Bayesian treatment of linear models, EM, variational inference and graphical models with full derivations.
    L7Lets you do Bayesian-linear-regression, EM and variational derivations end to end; use chapters 3, 9 and 10.
    Older but canonical.
  • 2004
    Convex Optimization
    Stephen Boyd, Lieven Vandenberghe (Cambridge University Press) · 2004 · Book · Advanced · ~25 h
    Free PDF; chapters on convex functions, duality and KKT conditions, unconstrained minimisation and Newton's method.
    L7Lets you reason about convexity, conditioning and Lagrange duality when an interviewer moves from 'what is Adam' to 'why does preconditioning help'; chapters 3, 5 and 9 only.
    Older but canonical.
  • 2003
    Information Theory, Inference, and Learning Algorithms
    David J. C. MacKay (Cambridge University Press) · 2003 · Book · Advanced · ~20 h
    Free book linking entropy, KL, coding and Bayesian inference, with exercises; MacKay's 16 recorded Cambridge lectures follow the book.
    L7Lets you derive entropy, KL and mutual-information identities and explain cross-entropy as expected code length, the framing research interviewers like.
    Older but canonical.
    Lecture videos (YouTube)Book home page
  • 1993
    Optimal Speedup of Las Vegas Algorithms
    Michael Luby, Alistair Sinclair, David Zuckerman · 1993 · Paper · Advanced · ~2 h
    Proves the optimal fixed-cutoff restart strategy when the runtime distribution is known, and a universal strategy within a log factor when it is not.
    L7Lets you handle the restart-strategy oral rigorously (Markov bounds, cutoff choice) and cite the known optimum.
    Author-hosted preprint of the Information Processing Letters 47 (1993) paper.
  • 1987
    Fifty Challenging Problems in Probability with Solutions
    Frederick Mosteller (Dover) · 1987 · Practice · Advanced · ~10 h · paid
    Classic short puzzles (coupon collector, gambler's ruin, waiting for patterns, random chords) with elegant solutions; trains the 'find the trick' instinct.
    L7Lets you handle the harder puzzle variants (HH vs HT, ruin probabilities) and explain the symmetry argument, which separates confident from rote answers.
    Paid (inexpensive Dover reprint of the 1965 original).
Not found or not verified. cs229.stanford.edu refused connections from this network, so the current CS229 term (Fall 2025 per search index) and its notes PDF were not fetched directly; the newest CS229 video set found is Spring 2022. Stat 110 course site (stat110.net) returns 403 to scripts; linked via YouTube and the book. Bishop PRML page returns 403 to bots. No official Stanford EE364A or CS229 recordings newer than 2023/2022 found. No stable publisher page for Heard on the Street (self-published); Goodreads used. No high-quality public worked solution found specifically for the account-sharing/resale detection or Instagram Stories prompts; covered with general design resources. Dawid-Skene (1979) annotator model not linked: no free canonical copy verified. Links checked 27 Oct 2026.
← PreviousAlgorithms (LeetCode-style)Next →ML coding from scratch and debugging
01 Practical coding02 Algorithms03 Fundamentals04 ML coding05 LLM internals06 Deep dive & leadership07 System design08 Inference design09 ML infra10 Training at scale11 Applied LLM12 Research13 Values & mission