Your plan
Hours per track = how often the track shows up for your chosen role × how much more weight it gets at staff+ level × how much you still need to learn it. The gap sliders start from defaults for a staff-level ML engineer who is new to LLMs; move them to match where you are.
| Track | How often it's asked share of evidence for this role | L7 weight | Your gap how much is left to learn | Hours |
|---|
Week by week hours per track; each track runs in its own window
What to study each week each track's study-guide stages, spread across its weeks
Small samples are pulled toward the all-roles mix (a 20-report prior), so research-scientist and applied plans don't swing on a handful of reports. Values and deep-dive rounds use the all-roles share for every role, because every candidate takes them but research-role reports rarely describe them. Your gap settings are saved in this browser only.
What the reports show
The numbers behind the plan. Shares are of weighted evidence: one report counts once per track, firsthand and recent reports count more.
- Practical coding is the largest block of evidence, at every level. 28% of weighted evidence overall and 19% in staff+ reports, from 111 firsthand reports. These are multi-part, unaided Python problems, not puzzles.
- The same problems come back again and again. Anthropic: web crawler (21), banking / multi-level oa system (16), image processing pipeline with parallelism (13), duplicate file detection (12). OpenAI: grid infection / contagion simulation (18), in-memory kv store / database (17), credits / token ledger with expiry (15), job scheduler / fault-tolerant queue (8). Numbers are distinct reports.
- At staff and above, the mix shifts toward judgment. Deep dive and leadership go from 12% to 23%, values from 11% to 19%, inference design from 6% to 11%. Based on 18 staff+ reports, so read it as direction, not precision.
- System design differs by lab. OpenAI asks product systems (payments, webhooks, CI/CD, cloud IDE, chess). Anthropic asks LLM infrastructure (batch inference over GPUs, model distribution to thousands of machines, chat at scale, prompt playground).
- Research roles get a different interview. For research engineers and MLEs, ML coding, LLM internals, fundamentals, research and training at scale make up 50% of evidence, versus 1% for software engineers. Samples are small: 36 firsthand RE/MLE and 12 research-scientist reports.
- Values rounds are mostly an Anthropic thing, and they decide outcomes. 77 reports mention values questions, most at Anthropic: why this lab, views on AI risk, critique of the lab, ethics under pressure.
- Level is decided by the loop, usually one step down. A Meta E6 typically lands at OpenAI L5 (staff); the E7-equivalent L6 is rare. Anthropic hides levels; MSL downlevels hard. See leveling.
Evidence by role
Share of weighted evidence for each track, by role. Darker = asked more. The last column is staff-and-above reports only; it drives the L7 weights. Hover a cell for counts.
| All roles216 firsthand | Research Engineer / MLE36 firsthand | Research Scientist12 firsthand | ML Infra / Performance19 firsthand | Applied AI11 firsthand | Software Engineer107 firsthand | Staff and above18 reports | |
|---|---|---|---|---|---|---|---|
| Practical coding | 28% | 17% | 15% | 26% | 11% | 35% | 19% |
| Algorithms | 6% | 7% | 10% | 0% | 5% | 6% | 5% |
| Fundamentals | 3% | 8% | 11% | 0% | 0% | 0% | 0% |
| ML coding | 4% | 15% | 14% | 0% | 0% | 0% | 0% |
| LLM internals | 2% | 9% | 8% | 0% | 0% | 0% | 0% |
| Deep dive & leadership | 12% | 10% | 9% | 12% | 18% | 12% | 23% |
| System design | 11% | 0% | 0% | 9% | 2% | 18% | 13% |
| Inference design | 6% | 5% | 0% | 3% | 1% | 7% | 11% |
| ML infra | 3% | 3% | 0% | 12% | 4% | 2% | 4% |
| Training at scale | 3% | 6% | 5% | 19% | 0% | 1% | 0% |
| Applied LLM | 7% | 6% | 4% | 0% | 48% | 6% | 3% |
| Research | 4% | 13% | 20% | 0% | 0% | 0% | 2% |
| Values & mission | 11% | 1% | 5% | 19% | 10% | 12% | 19% |
Most public reports are from software-engineer candidates. Research-scientist and applied-AI columns rest on few reports.
Leveling at L7
Labs mostly use one title (Member of Technical Staff) and set your level after the loop. For a big-tech staff engineer (Google L6 / Meta E6), the realistic base case is a staff-equivalent offer; a senior-staff offer needs senior-staff evidence in the loop itself.
| Lab | Ladder | ≈ Google L7 / Meta E7 | What candidates report | Evidence |
|---|---|---|---|---|
| OpenAI | L2–L7, all titled Member of Technical Staff | L6 (L5 = staff) | A Meta E6 normally lands at L5. L6 is described as a “ridiculously high bar”, won mostly by people already at E7/L7 or by research hires. Same loop for senior and staff; level set afterwards. Open to level negotiation; a competing offer is the main lever. | Many consistent Blind data points |
| Anthropic | Hidden; MTS title. Job board has Staff+, one Senior Staff and Principal roles | Unclear | Level isn't shown in the offer. Reports conflict (“L5 strong IC, L6 tech lead” vs “one level below big tech”). A staff candidate was offered senior. Offers reported as fixed within a level. References sometimes checked before the offer. | Weak, contradictory |
| Google DeepMind | Google ladder | L7 (same ladder) | The only lab where the level maps 1:1. Hiring committee can still downlevel to L6. No firsthand 2025–26 L7 research loop found. | Official ladder |
| Meta MSL | Meta E-ladder | E7 | Downlevels hard: a Microsoft principal got E6 (“would not consider me for E7”). Meta GenAI's 2024 E7 loop: talk, 1 coding, 3 AI design, leadership, people management, technical and project retrospectives. | Firsthand, few |
| xAI | Flat MTS title, no public ladder | No mapping | Seniority shows in comp, not title. Short hiring-manager or exec gates (15 minutes) can end a staff+ process. | Very weak |
What changes in the loop at L7
- ReportedExpect to be leveled from the loop, not your title. OpenAI and Anthropic set level after the loop and commonly place people one level down. Ask the recruiter which level the role is, when it's decided, and whether the loop differs by level.
- ReportedCoding stays a gate. Anthropic staff and senior-staff loops keep multi-part practical coding; one candidate needed all 5 follow-ups for a strong hire.
- ReportedTwo design rounds, AI-infrastructure flavoured. Model downloader without a coordinator, batch inference over GPUs, 1:1 chat at scale, prompt playground.
- ReportedThe deep dive tests business judgment. “Was the outcome worth a $20M investment?”, “what did you personally do?”, a separate technical presentation run as reverse system design.
- ReportedThe culture round is a senior failure point. Answers must make sense to a non-technical interviewer.
- InferenceTreat OpenAI L5 / Anthropic staff as the base case. Prepare a senior-staff case only with evidence of org-wide scope; argue level from scope shown in the loop.
- InferenceRun loops in parallel. Offers landing together are the main negotiation lever; Anthropic is reported as non-negotiable within a level.
- InferenceBrief your references. References are sometimes checked before the offer; make sure they can confirm the scope stories you told.
How each lab's loop runs
From firsthand reports only: which rounds candidates name, and the problems that recur. Google DeepMind and MSL samples are thin.
OpenAI
Rounds named in firsthand reports
Most reported problems reports
Google DeepMind
Rounds named in firsthand reports
Most reported problems reports
Meta MSL
Rounds named in firsthand reports
Most reported problems reports
xAI
Rounds named in firsthand reports
Most reported problems reports
Core library
The resources that prepare you for the most reported questions across all 13 topics, ranked by how many report mentions they cover. Drawn from 982 unique resources in the study guides; 89 of them serve more than one topic. Topic chips jump to the track.
- 12026OpenAI Interview Notes (unofficial reconstructed problems)37 coding problems reconstructed from reports: infection spread, GPU credits, in-memory database, durable KV store, memory allocator, spreadsheet dependencies, fault-tolerant work queue, resumable iterator, IPv4 CIDR, social network snapshots, time-based KV, rate-limiter debug, chatbot refactoring, cd command, type inference, monster battle and more, each with multi-part statements, solutions and checks.Unofficial reconstructions. The original Schuture/* repo no longer resolves (404 on 2026-10-27); this is a 1-star mirror created 2026-09-23.Practical codingAlgorithmsFundamentals
- 22026Anthropic Interview Notes (unofficial reconstructed problems)25 coding pages reconstructed from candidate reports: concurrent web crawler, file deduplication, LRU cache bugfix + durability, stack-trace reconstruction, longest-match tokenizer, image pipeline, bootloader loop, nine multi-level OAs (bank, in-memory DB, cloud storage, recipe manager, task manager…), with full statements, solutions and CI-run checks.Unofficial; statements are reconstructions, not the actual questions. The original Schuture/* repo no longer resolves (404 on 2026-10-27); this 2026-09 copy is a 2-star mirror, so it may disappear.Practical coding
- 32026Anthropic Behavioral Interview Field GuideYour own field guide of Anthropic behavioral and culture prompts with answer patterns; the fastest map from reported questions to the stories you need.Deep dive & leadershipValues & mission
- 42026Behavioral Coaching Notes: Anthropic Philosophy and OpenAI Technical Deep DivesNotes from coaching senior ICs for Anthropic and OpenAI loops: develop a real moral philosophy for Anthropic, lead with the headline, how OpenAI deep dives go.Deep dive & leadershipValues & mission
- 52026Anthropic Staff Software Engineer Interview Guide (2026)Living guide to Anthropic's staff loop: project deep dive, hiring-manager conversation, Culture round, references, with recent user-reported questions.Practical codingDeep dive & leadershipValues & mission
- 62023Core Views on AI Safety: When, Why, What, and HowAnthropic's founding argument: rapid progress is likely, safety is empirical, so a safety-focused lab must work at the frontier.Values & mission
- 72024Deeper dive into the signal areas for behavioral interviewsDefines the signal areas (scope, ownership, ambiguity, conflict, growth) and notes that level follows the scope assessment.Deep dive & leadership
- 82023The Long-Term Benefit TrustGovernance structure giving an independent trust power to elect board members.Values & mission
- 92021Crucial Conversations (3rd edition)Method for high-stakes disagreements: start with heart, make it safe, state your path.Deep dive & leadership
- 102026Anthropic's Responsible Scaling Policy (v3.4, with Risk Reports)Current RSP (v3.4 effective 8 Jul 2026) with every prior version and redline, plus the Feb and Aug 2026 redacted Risk Reports.ResearchValues & mission
- 112026Open Gauntlet: 4-level progressive coding practiceLocal web app with six four-level problems, a timer and hidden tests that unlock the next level, built by a candidate who took Anthropic-style OAs.Practical coding
- 122026LeetCode 1236 Web Crawler / 1242 Web Crawler MultithreadedSame-hostname BFS crawl with a provided HtmlParser, then the multithreaded version; the reported phone-screen problem.Practical coding
- 132026Upleveling your responses | Behavioral Coaching Notes #2Field notes on rewriting senior answers into staff-level ones: headline first, broader impact, reflection.Deep dive & leadership
- 142025How To Scale Your ModelRoofline analysis, TPU/GPU networking, parallelism strategies and serving maths for LLMs, written by DeepMind researchers.ML codingLLM internalsML infraTraining at scale
- 152017Designing robust and predictable APIs with idempotencyIdempotency keys, safe retries with exponential backoff and how Stripe's API makes charge creation retry-safe.Practical codingSystem designApplied LLM
- 162020Design Docs at GoogleAnatomy of a design doc: context, goals and non-goals, alternatives considered, cross-cutting concerns.Deep dive & leadershipSystem design
- 172025Communicating Scope in a Tech Lead InterviewHow to state team size, org reach, and your personal decisions so interviewers can level you.Deep dive & leadership
- 182026International AI Safety Report 2026Government-backed scientific review of general-purpose AI capabilities, risks and mitigations.Values & mission
- 192026Python docs: concurrent.futures — launching parallel tasksThreadPoolExecutor/ProcessPoolExecutor, as_completed, wait, cancellation and shutdown; the standard way to turn a BFS crawler or image batch into a parallel one.Practical coding
- 202018How to SpeakClassic one-hour lecture on structuring talks: empowerment promise, cycling, slides, closing.Deep dive & leadership
- 212026Interview notes: LLM Inference API with Batched GPU ServingA worked reference solution (about 6,500 words with Python estimates) for the Anthropic batching prompt: requirements, API, a dispatcher in front of GPU replicas, real-time and offline pools, KV bytes per token, prefix-cache savings, and a discrete-event simulation of a size-or-time batcher.Inference design
- 222026Why AI risks are the world's most pressing problemsAccessible problem profile on AI risk with counterarguments, updated Oct 2026.Values & mission
- 232026Python docs: collections, heapq, bisect, itertools, dataclassesThe containers every lab problem leans on: OrderedDict/deque/Counter/defaultdict, heap-based priority queues (including the priority-queue implementation notes), bisect for time-indexed lookups, itertools recipes.Practical coding
- 242021Present to executivesHow to structure a presentation for senior leaders: lead with the ask, handle interruptions.Deep dive & leadership
Study tracks
Each track shows the problems that recur in reports with the resources that prepare you for them, recent firsthand questions, what changes at L7 and the probes to expect, a staged learning path drawn from its study guide, and top picks (2026 in green). Hours match your plan above.
01 Practical coding, no AI 28% of evidence · 174 reports · 111 firsthand
Share by role: Research Engineer 17% · Research Scientist 15% · ML Infra 26% · Applied AI 11% · Software Engineer 35% · staff+: 19%
Most reported problems distinct reports
- 28In-memory KV store / database (TTL, transactions, snapshots, serialization)OpenAI 17, Anthropic 8, xAI 2 · 7 firsthand · 2 staff+
- 22Web crawler (BFS, then concurrent)Anthropic 21, OpenAI 1 · 13 firsthand · 3 staff+
- 18Banking / multi-level OA systemAnthropic 16, xAI 1, Unnamed lab 1 · 9 firsthand · 1 staff+
- 18Grid infection / contagion simulationOpenAI 18 · 11 firsthand · 1 staff+
- 18Credits / token ledger with expiryOpenAI 15, Anthropic 3 · 7 firsthand · 2 staff+
- 15LRU cache (decorator, persistence, crash recovery)Anthropic 11, xAI 4 · 8 firsthand · 1 staff+
- 15Refactor or extend unfamiliar codeOpenAI 8, Anthropic 5, xAI 2 · 11 firsthand · 1 staff+
Recent firsthand questions
- OpenAI · 2026-10Implement malloc/free with provided tests; part 2: optimize to O(log n) malloc and free (e.g. two sorted lists, third-party library allowed)
- OpenAI · 2026-10Fault-tolerant queue: implement enqueue, reserve, fail and a dead-letter queue passing given unit tests; part 2 adds dependencies between tasks
- OpenAI · 2026-10Fault-tolerant work queue (two parts)
- OpenAI · 2026-10Fault-tolerant queue: enqueue, reserve, fail, DLQ with 7 unit tests; part 2 adds task dependencies; preempted tasks have a cooldown computed from the preempt timestamp and are re-queued ordered by original enqueue time
- OpenAI · 2026-10Chat session analytics over a non-decreasing log stream: ingest(user, chat, ts) and count_recent(user, chat) for the closed window [T-14, T] with memory bounded by active chats; part 2 adds touch/close events and count_live(user); part 3 al…
At L7Coding does not go away at staff or senior staff. Anthropic staff candidates report 3–5 follow-up parts per problem and needing to finish all of them for a strong hire; an inconclusive round triggers an extra one. Treat it as a gate you must pass cleanly.
Senior-staff probes
- Concurrency correctness under follow-ups
Your crawler's visited set and work queue: where exactly is the race, how do you bound in-flight requests, and how does the crawl terminate and shut down when one fetch hangs?
- Durability and crash recovery
The process dies halfway through writing the LRU snapshot. What does the next start-up see, and which fsync/rename steps make that safe?
- Data-structure choice with stated complexity at every level
Make malloc and free O(log n). Which structures, how do you coalesce neighbours, and what fragmentation do first-fit vs. best-fit give you?
- Designing level 1 for level 4
Level 3 adds account merges and historical balances. What in your level-1 design makes that a small change instead of a rewrite?
Learning path stages from the study guide; best 3 resources each
- Python without a netWrite idiomatic, stdlib-only Python fast with autocomplete off.
- 2024
- 2026
- 2026
- Concurrency you can defendTurn BFS and batch jobs into thread/process/async versions and explain the choice.
- 2026
- 2015
- 2026
- Drill the lab setsFinish every part of each recurring Anthropic and OpenAI problem unaided.
- 2026
- 2026
- 2026
- Refactor, debug and timed levelsRead unfamiliar code, test first, change safely; finish 4 levels in 90 minutes twice in a row.
- 2026
- 2018
- 2026
- Senior-staff depthAnswer the systems follow-ups (durability, allocator policy, retries, rate limits, ledgers, trace formats) with production-grade reasoning.
- 2021
- 2019
- 2015
Ready whenYou finish all four levels of an unseen progressive problem in 90 minutes with tests, twice in a row.
Start here top picks, newest first
- 2026★Python docs: concurrent.futures — launching parallel tasksL7Lets you choose threads vs. processes per workload and explain shutdown/cancellation semantics when a follow-up adds timeouts.
- 2026★Anthropic Interview Notes (unofficial reconstructed problems)L7Lets you rehearse every recurring Anthropic problem through all its parts; do each unaided first, then compare with the solution's pitfalls notes.
- 2026★OpenAI Interview Notes (unofficial reconstructed problems)L7Lets you drill the full OpenAI recurring set through every part; time each one and track how many parts you finish.
- 2026★Open Gauntlet: 4-level progressive coding practiceL7Lets you rehearse the real constraint: L1 design choices that make L3/L4 possible under 90 minutes.
- 2024★Effective Python: 125 Specific Ways to Write Better Python (3rd ed.)L7Gives you crisp one-line justifications for each idiom choice, which is how senior candidates narrate while typing; read the concurrency and robustness chapters closely, skim the rest.
- 2021★A Philosophy of Software Design, 2nd EditionL7Lets you pick an L1 interface that makes L3/L4 cheap and explain why, which is the difference between finishing four levels and three.
02 Algorithms (LeetCode-style) 6% of evidence · 43 reports · 23 firsthand
Share by role: Research Engineer 7% · Research Scientist 10% · ML Infra 0% · Applied AI 5% · Software Engineer 6% · staff+: 5%
Most reported problems distinct reports
Nothing in this track recurs in two or more reports; see the recent questions below.
Recent firsthand questions
- Anthropic · 2026-10 · staff+Staff-level coding interview: a LeetCode dynamic-programming problem (despite being told it would not be LeetCode)
- OpenAI · 2026-08Binary search to guess a secret number where the too-high/too-low feedback is delayed by one call (each response refers to the previous guess); part 2 not seen
- OpenAI · 2026-07Implement a simplified 'vector' (indexed sequence) with add(index, val) that shifts later elements and get(index); improve via linked list with checkpoints every k nodes and maintain checkpoints on insert
- GDM · 2026-05Coding round 1: modified LeetCode-medium problem with a probability twist, plus several follow-ups including a hard one
- GDM · 2026-05Coding round 2: problem with a probability + graph framing solved with Dijkstra's algorithm; in-depth discussion of the math and complexity (including heap complexity)
At L7At Google L7 and Meta E7, coding is mostly a filter. Labs report LeetCode mediums (grid DP, sliding window, LIS, binary search) but far less often than practical problems.
Senior-staff probes
- Name the pattern and complexity before coding
Before you write anything: what's the pattern, the target complexity, and the brute force you're beating?
- State-space modelling for search
Shortest path that must pass through a 1 and pays only for first visits: what is your search state, how many states, and why is BFS still correct?
- Heap and Dijkstra complexity in depth
Your probabilistic graph uses Dijkstra on products: prove it's valid, give the complexity with a binary heap and lazy deletion, and say when it breaks.
- DP derivation under follow-ups
Now the workload can switch clusters at most K times. Define the state and transition, give the complexity, and reduce memory.
Learning path stages from the study guide; best 3 resources each
- Refresh the patternsReload the pattern catalogue and Python complexity facts.
- 2026
- 2026
- 2026
- Do 40–60 mediumsUnaided, 25 minutes each, complexity stated, from a pattern-balanced list.
- 2026
- 2026
- 2017
- Fill gaps with one courseUse one video course only for weak topics (graphs, DP).
- 2020
- 2026
- 2020
- Lab-style variantsPractise the reported lab problems and turn them into clean classes.
- 2026
- 2026
- 2026
- Senior-staff depthDefend correctness and complexity, and handle hard follow-ups (state-space search, probability, amortized structures).
- 2026
- 2026
- 2026
Ready whenYou solve an unseen medium in 25 minutes with tests and complexity, five days in a row.
Start here top picks, newest first
- 2026★OpenAI Interview Notes (unofficial reconstructed problems)L7Lets you practise the 'algorithm in disguise' variants and their follow-ups as asked, with checks to validate your solution.
- 2026★NeetCode Roadmap (with video explanations)L7Fastest route to re-loading the pattern catalogue; the senior use is triage, so watch videos only for problems you can't solve in 25 minutes.
- 2026★NeetCode 150 (practice list)L7Source of the 40–60 mediums in step 2; pick by pattern weakness, not order.
- 2026
- 2020★MIT 6.006 Introduction to Algorithms (Spring 2020, OCW)L7Lets you rebuild graph and DP fundamentals with proofs; the SRTBOT DP lectures are the most useful.
- 2017★Elements of Programming Interviews in PythonL7Lets you practise the variants interviewers add after the base problem; do the hash table, heaps, graphs and DP chapters.
03 Math and ML fundamentals 3% of evidence · 23 reports · 13 firsthand
Share by role: Research Engineer 8% · Research Scientist 11% · ML Infra 0% · Applied AI 0% · Software Engineer 0% · staff+: 0%
Most reported problems distinct reports
Nothing in this track recurs in two or more reports; see the recent questions below.
Recent firsthand questions
- Mistral · 2026-09Diffusion models (forward and reverse processes)
- Mistral · 2026-09Optimization: Adam (Adam vs AdamW, decoupled weight decay)
- OpenAI · 2026-09Statistics: rapid-fire stat concepts
- GDM · 2026-05ML round: solve a toy problem, then go very deep on transformers, classification (e.g. which loss to use for binary classification), tokenization and evaluation
- OpenAI · 2026-04Noisy annotator data: train a baseline with given model/code, then clean data to improve performance; basic questions on precision, recall, F1; pandas dataframe
At L7No staff-level evidence either way. Research loops (DeepMind especially) test derivations; engineering loops at staff level rarely do.
Senior-staff probes
- Derive any gradient on paper and connect it to implementation
Derive the LayerNorm backward pass, then tell me how you would fuse it with the residual add and what that saves in memory traffic.
- Explain optimiser behaviour mechanistically and at scale
Show algebraically why L2 under Adam differs from decoupled weight decay, and how you would set the decay as model and data size grow 10x.
- Know how hyperparameters transfer and why big runs go unstable
Your 70B run diverges at step 40k with attention-logit growth. What do you check, what do you change, and how do you test the fix cheaply?
- Judge modern optimisers with evidence
A team claims a new optimiser gives 1.3x speedup. How do you decide if it is real before switching the next frontier run?
Learning path stages from the study guide; best 3 resources each
- Self-quiz and triageTake a 40-question quiz from real lab rounds; study only what you miss.
- 2021
- 2026
- 2026
- Derive by handSoftmax-CE, linear layer, LayerNorm backward, Adam vs AdamW, KL between Gaussians, MLE on paper.
- 2023
- 2022
- 2017
- Probability, statistics and information theoryConditional probability, expectation puzzles, CIs, A/B basics, entropy/KL identities.
- 2024
- 2013
- 2026
- Rapid-fire out loud20 questions in 20 minutes, twice a week, scored with a rubric.
- 2026
- 2026
- 2024
- Senior-staff depthGo past textbook answers: optimiser and parametrisation choices at scale, training instabilities, modern bias-variance, diffusion in SDE form, eval statistics.
- 2022
- 2018
- 2026
Ready when20 rapid-fire fundamentals in 20 minutes, and any one gradient derived on paper.
Start here top picks, newest first
- 2026★Timed ML math oral (practice lab)L7Lets you rehearse the oral derivation format with a rubric; run it twice before a research loop.
- 2024★Adding Error Bars to Evals: A Statistical Approach to Language Model EvaluationsL7Lets you answer model-comparison questions with paired tests, clustered standard errors and a power calculation, as a frontier lab expects.
- 2023★Understanding Deep LearningL7Lets you explain why each loss follows from a likelihood and derive initialisation variance conditions; the diffusion chapter is a strong 20-minute refresher.
- 2022★Building makemore Part 4: Becoming a Backprop NinjaL7Lets you derive fused softmax-cross-entropy and normalisation backward passes by hand and explain the numerically stable fused form.
- 2022★Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (muP)L7Lets you explain how frontier labs tune hyperparameters on small proxies and why standard parametrisation breaks at scale.
- 2021★Introduction to Machine Learning Interviews BookL7Baseline question bank; use it to build the 40-question self-quiz, then answer at senior depth with trade-offs.
04 ML coding from scratch and debugging 4% of evidence · 30 reports · 16 firsthand
Share by role: Research Engineer 15% · Research Scientist 14% · ML Infra 0% · Applied AI 0% · Software Engineer 0% · staff+: 0%
Most reported problems distinct reports
- 20Debug a broken model or training loopOpenAI 9, Anthropic 6, Unnamed lab 3 · 9 firsthand
- 19Attention / transformer block from scratchOpenAI 8, Unnamed lab 4, Anthropic 3 · 10 firsthand
- 6Backprop / autograd / optimizer by handOpenAI 3, Unnamed lab 2, GDM 1 · 4 firsthand
- 4Sampling / decoding (top-k, top-p, beam)Unnamed lab 2, GDM 1, OpenAI 1 · 1 firsthand
Recent firsthand questions
- OpenAI · 2026-09ML coding: debug 4-5 implanted bugs in transformer code and implement KV cache from scratch
- Anthropic · 2026-09Research fundamentals round: implement an ML algorithm and discuss research/mathematical aspects around it
- OpenAI · 2026-08ML coding (Colab, numpy/pytorch): implement vectorized 1-NN with L2 distance without for loops; express 1-NN as a neural network forward pass (linear layer + softmax); how to do it with L1 distance
- Anthropic · 2026-08Anthropic Fellows OA2: debugging exercise (6 parts) where the bug can be anywhere, with a high expectation of NumPy proficiency
- Anthropic · 2026-05Implement multi-head attention from scratch (no torch.nn.MultiheadAttention) and handle KV cache plus grouped-query attention in the same hour
At L7No staff-level evidence either way. Meta MSL loops at E6 include 'AI coding' (NumPy/PyTorch components); a Microsoft principal prepared for it with CS336 alone.
Senior-staff probes
- KV-cache design with memory arithmetic
Your 70B model OOMs at 32k context with batch 16. Size the cache per token, then compare GQA, MLA, paging and quantized KV — what do you pick and what does it cost in quality?
- Numerically stable attention and kernels
Your attention NaNs in bf16 at long context. Walk through the softmax numerics, then write the online-softmax recurrence FlashAttention uses so the N×N matrix is never stored.
- Structured debugging under time pressure
The miniGPT's loss sits at ln(vocab) and samples are garbage. What do you check first, second, third, and what single test would have caught each bug?
- Diagnosing instability at scale
A run that was fine at 1B spikes and diverges at 30B. What do you look at (attention logits, output logits, LR, data), what small-scale proxy do you run, and what do you change?
Learning path stages from the study guide; best 3 resources each
- Attention from a blank fileWrite causal multi-head attention, RoPE and a KV-cached decode loop with shapes in comments, tested against SDPA
- 2025
- 2025
- 2024
- Backprop by hand and classic MLMatmul/softmax-CE/layernorm backward in NumPy checked with finite differences; AdamW; loop-free kNN, k-means, logistic regression
- 2026
- 2022
- 2026
- Debug a broken miniGPT, then samplingBuild a 15-bug zoo, narrate a bisection, find 5 planted bugs in 45 min; implement temperature/top-k/top-p/beam
- 2019
- 2026
- 2026
- CS336 Assignment 1 end to endBPE tokenizer, transformer LM, AdamW, schedule and training loop passing the official tests, trained on TinyStories
- 2026
- 2026
- 2024
- Senior-staff depthArgue KV-cache budgets, attention variants, stability fixes and nondeterminism with numbers and primary sources
- 2023
- 2026
- 2025
Ready whenCausal multi-head attention with a KV cache from a blank file in 30 minutes, and 5 planted bugs found in 45.
Start here top picks, newest first
- 2026★LLMs-from-scratch (code for 'Build a Large Language Model (From Scratch)')L7The bonus folders let you compare MHA/GQA/MLA-style variants side by side in readable code, which is what you need to discuss architecture trade-offs while coding.
- 2026★CS336: Language Modeling from Scratch (Spring 2026)L7Gives you first-hand numbers (tokens/s, memory, loss curves) from a model you built, so you can argue design choices from experience rather than recall; A2 adds FlashAttention-2 in Triton and DDP.
- 2026★CS231n: Deep Learning for Computer Vision (Spring 2026)L7Baseline for a staff-level ML engineer; do A1 and the BN-backward part of A2 cold to make hand backprop and vectorization automatic under time pressure.
- 2025★nanoGPTL7Baseline; the value is knowing this file so well that a planted bug in a look-alike stands out.
- 2025★Understanding and Coding the KV Cache in LLMs from ScratchL7Baseline for the OpenAI 'add a KV cache' round; then extend it yourself to GQA and a pre-allocated cache.
- 2024★Neural Networks: Zero to HeroL7Baseline for implementation fluency; the 'backprop ninja' and 'build GPT' episodes are the ones to redo without the video.
05 LLM internals: architecture, pretraining, post-training 2% of evidence · 21 reports · 7 firsthand
Share by role: Research Engineer 9% · Research Scientist 8% · ML Infra 0% · Applied AI 0% · Software Engineer 0% · staff+: 0%
Most reported problems distinct reports
Nothing in this track recurs in two or more reports; see the recent questions below.
Recent firsthand questions
- Mistral · 2026-09Mixture of experts (and expert parallelism)
- Mistral · 2026-09Normalization (RMSNorm vs LayerNorm in transformer blocks)
- Mistral · 2026-09Preference-based training (reward-model RLHF vs DPO)
- GDM · 2026-05Deep discussion of NLP and transformer internals
- OpenAI · 2026-04Design an ML system to mine novel/interesting images from a huge unlabeled corpus, using human labelers without over-relying on them
At L7At L7 research-track loops, expect 'what's wrong with our models and how would you fix it' (Meta GenAI E7 talk topic) and rapid-fire questions across the stack (positional encodings, 5D parallelism, PPO vs GRPO).
Senior-staff probes
- Explain every architecture choice as a cost/quality trade-off with numbers
Llama 3 uses GQA, DeepSeek-V3 uses MLA. Compute KV-cache bytes per token for each at 128K context and tell me which you'd pick for a 1M-context product and why.
- Do scaling arithmetic live and connect it to business constraints
You have 10^25 FLOPs and expect 10^13 inference tokens a month. How big a model, how many tokens, how many GPU-days, and why not Chinchilla-optimal?
- Diagnose training instabilities and choose cheap experiments
Loss spikes at step 40K of a 400B MoE run and one expert's routing share jumps. What do you check first, and what small-scale run would confirm the cause?
- Design a post-training pipeline and justify each stage
Design post-training for a new base model targeting coding and math. Where do you use SFT, DPO, RLVR and distillation, and how do you know each helped?
Learning path stages from the study guide; best 3 resources each
- Foundations: get fluent fastKnow the full pipeline and the vocabulary in about a week
- 2025
- 2025
- 2024
- Architecture tourBuild the GPT-2 → Llama 3 → DeepSeek-V3 → Qwen3 diff table and defend each choice
- 2025
- 2025
- 2025
- Scaling arithmetic and pretrainingEstimate a 70B run on paper and explain why labs over-train
- 2025
- 2022
- 2025
- Post-training and reasoning RL, hands-onExplain PPO vs GRPO in two minutes, then run a small GRPO job on math
- 2026
- 2026
- 2025
- Senior-staff depthCritique each target lab's models and propose a fix, citing frontier-scale evidence
- 2024
- 2024
- 2026
Ready whenYou can explain every design choice in Llama 3, PPO vs GRPO in two minutes, and give one model critique per lab.
Start here top picks, newest first
- 2026★Reinforcement Learning from Human Feedback (RLHF Book)L7Lets you lay out a complete post-training recipe and say where each stage breaks, which is the core of 'how would you fix our model' answers.
- 2026★CS336: Language Modeling from Scratch (Spring 2026)L7Lets you answer any 'why this design choice' question with something you have actually built and measured, not something you read.
- 2025★The Smol Training Playbook: The Secrets to Building World-Class LLMsL7Lets you talk about how real pretraining decisions are made (ablation budgets, when to stop, what broke) rather than textbook recipes.
- 2025★nanochatL7Gives you first-hand numbers for every stage of the pipeline in a weekend, so your answers carry real experience.
- 2025★The Big LLM Architecture ComparisonL7The fastest way to build the curriculum's one-page architecture diff table and say why each lab chose what it did.
- 2025★DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningL7Lets you explain what RLVR does and does not need (no process reward model, no MCTS) and why distillation beats RL for small models.
06 Project deep dive and leadership stories 12% of evidence · 75 reports · 50 firsthand
Share by role: Research Engineer 10% · Research Scientist 9% · ML Infra 12% · Applied AI 18% · Software Engineer 12% · staff+: 23%
Most reported problems distinct reports
- 44Project deep dive: what you personally did and whyAnthropic 19, OpenAI 16, xAI 6 · 31 firsthand · 7 staff+
- 20Conflict / disagreementAnthropic 9, OpenAI 8, Meta (org unspecified) 1 · 10 firsthand · 5 staff+Prepare withAnthropic Behavioral Interview Field GuideCrucial Conversations (3rd edition)
- 17Leading / influencing without authorityOpenAI 7, Anthropic 6, MSL 2 · 11 firsthand · 4 staff+
- 4Failure and what changedOpenAI 2, Mistral 1, Anthropic 1 · 1 firsthand
Recent firsthand questions
- Anthropic · 2026-09Why are you a good fit?
- Anthropic · 2026-09Give an example of a new measurement you came up with and its business impact.
- Mistral · 2026-09Background and projects discussion
- OpenAI · 2026-09Leadership: leadership experience, decision-making, ownership, influence; cross-functional collaboration with PM
- OpenAI · 2026-09Project deep dive on a large product launch you led, with API questions and a behavioral question
At L7This is where level is decided. Level is set after the loop from how you did, and a one-level downlevel is the common outcome. Senior-staff candidates report a deep dive probing business judgment (“was the outcome worth a $20M investment?”), a separate technical presentation run as a reverse system design, and the recurring question “what did you personally do?”. Meta GenAI's E7 loop added a talk plus technical and project retrospectives.
Senior-staff probes
- Personal contribution at org scope
Out of the 40 engineers on this, which three decisions would not have happened without you, and how do you know?
- Business judgment and ROI
This took 50 engineer-years at roughly $400K each. Was the outcome worth $20M, and what would you have cut?
- Rejected alternatives and reversals
Why not the obvious alternative? When did you reverse a decision, and what evidence triggered it?
- Running the talk as a reverse system design
Stop at slide 3: walk me through the data path and its failure modes, then defend the latency budget.
Learning path stages from the study guide; best 3 resources each
- Learn the barKnow what each lab's staff loop probes and where level is decided
- 2026
- 2026
- 2026
- Write the two one-pagersProblem, your part, rejected alternatives, cost and ROI, what failed
- 2025
- 2024
- 2020
- Build and rehearse the talk20-minute talk that survives interruptions and runs as a reverse system design
- 2026
- 2026
- 2026
- Story bank and mocksEight stories (conflict, failure, influence, mentoring, ambiguity, XFN, reversal, bad project) told at staff scope
- 2026★Anthropic Behavioral Interview Field Guide
- 2026
- 2026
- 2026
- Senior-staff depthDefend every decision three levels deep with business and org-level judgment
- 2024
- 2022
- 2021
Ready whenA peer at senior-staff level can't find a decision in either project you can't justify, and your talk survives interruptions.
Start here top picks, newest first
- 2026★Anthropic Behavioral Interview Field GuideL7Lets you pre-load stories for the exact Anthropic prompts (wrong, conflict, neck-on-line) instead of improvising at staff level.
- 2026★Behavioral Coaching Notes: Anthropic Philosophy and OpenAI Technical Deep DivesL7Current (2026) frontier-lab-specific calibration from someone coaching senior+ candidates through these exact loops.
- 2026★Anthropic Staff Software Engineer Interview Guide (2026)L7Shows what the staff loop adds over senior (project deep dive, cross-team leadership) so you prepare for the level, not just the company.
- 2025★Communicating Scope in a Tech Lead InterviewL7Lets you answer 'what did you personally do' with scope numbers rather than team achievements.
- 2024★How I ship projects at big tech companiesL7Gives language for missed deadlines, infeasible projects, and 'what would you do faster'.
- 2024★Executive translationL7Lets you answer the $20M / 50 engineer-years question in business terms instead of technical ones.
07 Product and distributed system design 11% of evidence · 75 reports · 37 firsthand
Share by role: Research Engineer 0% · Research Scientist 0% · ML Infra 9% · Applied AI 2% · Software Engineer 18% · staff+: 13%
Most reported problems distinct reports
- 18Chat / messaging at scale (1:1, Slack, livestream)Anthropic 9, OpenAI 8, xAI 1 · 4 firsthand · 5 staff+
- 12Payments (authorization, settlement, holds)OpenAI 12 · 6 firsthand · 1 staff+
- 10Real-time game (chess, typing race, trivia)OpenAI 10 · 6 firsthand · 1 staff+
- 8Cloud IDE / devbox sandboxOpenAI 8 · 7 firsthand
- 8Metrics / telemetry / IoT ingestionOpenAI 6, Anthropic 1, xAI 1 · 7 firsthand
- 7Webhook delivery platformOpenAI 7 · 2 firsthand · 2 staff+
- 7CI/CD / job execution platformOpenAI 7 · 2 firsthand · 2 staff+
Recent firsthand questions
- OpenAI · 2026-10Design an online devbox
- OpenAI · 2026-10Metrics monitoring system design variation, going deep on 1-2 areas
- OpenAI · 2026-10Design a typing race game for 2-8 players: generated passage, first to finish wins, 1-second freeze on a typo; cooldown/unfreeze mechanism, matchmaking; follow-ups on ranking by score and reducing network delay differences for fairness
- OpenAI · 2026-10Design a real-time trivia battle royale platform: dozens to ~100 players per room, questions released round by round, each round eliminates wrong answers and the slowest player, last alive wins; spectators watch a live leaderboard of any in…
- Anthropic · 2026-09Design a one-to-one text-messaging web app at 100M DAU that keeps messages for offline recipients; follow-ups: DB load at this scale, finding a user's chat IDs on login, full message lifecycle, lost/out-of-order messages between app servers…
At L7Senior-staff loops include at least two design rounds (Google L7, Anthropic senior staff, OpenAI staff). Interviewers ask you to defend internals and justify each component (“why do you need this?”), not just draw boxes.
Senior-staff probes
- Money correctness under retries and partial failure
The processor times out after you placed a hold. The client retries with the same key but a different amount. What happens to the hold, the ledger and the merchant webhook?
- Delivery semantics and ordering stated precisely
In your 100M-DAU chat, a recipient is offline for a week on two devices. Walk me through how each device gets every message exactly once, in order, and what your DB does per login.
- Isolation and fairness in multi-tenant async systems
One customer's webhook endpoint is down and you've built up 2 billion pending retries. How do you keep everyone else's latency flat, and how does the system drain when they recover?
- Choosing the right isolation for untrusted code
Containers, gVisor or Firecracker for the devbox? Defend it on cold-start time, density, disk persistence and what a kernel exploit costs you.
Learning path stages from the study guide; best 3 resources each
- Refresh the frameworkRun a 45-minute design with a fixed pacing script and quick estimates
- 2026
- 2026
- 2026
- Foundations of data systemsKnow replication, partitioning, consistency and storage engines well enough to argue them
- 2026
- 2026
- 2026
- OpenAI and Anthropic problem setOne worked design per recurring prompt, each with numbers and a failure story
- 2026
- 2025
- 2026
- Design docs and mocksCritique a flawed design doc and run six mocks with a mid-interview requirement change
- 2020
- 2026
- 2025
- Senior-staff depthDefend internals with production evidence: papers and frontier/hyperscale write-ups you can cite and critique
- 2026
- 2024
- 2020
Ready whenYou lead an unseen design for 45 minutes, defend every component, and handle a mid-interview requirement change.
Start here top picks, newest first
- 2026★Designing Data-Intensive Applications, 2nd EditionL7Lets you defend a consistency or partitioning choice from first principles and name the failure mode of the alternative, which is the main L7 probe.
- 2026★MIT 6.5840 Distributed Systems (Spring 2026)L7Building Raft and a sharded KV store yourself is what lets you answer 'what happens when the leader dies mid-write' without hand-waving.
- 2026★Design a Messaging App Like WhatsAppL7Mid-level answer by itself; at L7 add explicit ordering guarantees, at-least-once plus client dedup, and DB load numbers for 100M DAU.
- 2025★Design a Payment System like StripeL7Mid-level baseline; pair with Stripe's ledger and idempotency posts for holds, settlement batches and double-entry correctness.
- 2024★Ledger: Stripe's system for tracking and validating money movementL7Lets you design holds, captures and settlement as ledger entries and explain how you'd detect drift, which separates staff from mid-level payment answers.
- 2020★Monarch: Google's Planet-Scale In-Memory Time Series DatabaseL7Lets you defend availability over consistency for monitoring, and zone-local ingestion, which is the L7 metrics answer.
08 LLM inference and serving design 6% of evidence · 53 reports · 17 firsthand
Share by role: Research Engineer 5% · Research Scientist 0% · ML Infra 3% · Applied AI 1% · Software Engineer 7% · staff+: 11%
Most reported problems distinct reports
- 24Batch inference API over GPUsAnthropic 18, xAI 3, Mistral 2 · 7 firsthand · 4 staff+
- 20LLM inference API / router / serviceAnthropic 17, Mistral 2, xAI 1 · 8 firsthand · 3 staff+
- 11Latency debugging (p99, TTFT, decode)Anthropic 4, xAI 2, Unnamed lab 2 · 4 firsthand
- 7KV cache, paging and prefix reuseUnnamed lab 3, xAI 1, GDM 1 · 1 firsthand
Recent firsthand questions
- OpenAI · 2026-09Design Sora where each task uses only one GPU: fair queueing approaches and trade-offs, then cancellation approaches
- Anthropic · 2026-08Anthropic Fellows OA1: an "Inference Engine" problem involving prefill/decode requests, implemented against provided custom classes and a README
- Anthropic · 2026-08Anthropic Fellows OA1: implement the load-balancer functions for an inference engine
- Anthropic · 2026-08Anthropic Fellows OA1: add prefill and decode handling to a scheduler (same inference-engine question as other commenters)
- Anthropic · 2026-07Design an ML inference service: batching strategy, compute theoretical max throughput then estimate actual throughput under latency constraints, caching value, GPU server load-balancing strategy
At L7Rises sharply in staff+ reports. Anthropic senior-staff and staff infra loops: batch inference API over GPU clusters, GPU inference API with batching, inference router, critique an inference design.
Senior-staff probes
- Capacity math from roofline to GPU count
Compute the theoretical max decode throughput of a 70B model on one 8×H100 node, then what you actually get under a 50 ms TPOT SLO, and how many nodes 1,000 rps needs.
- KV-cache budget and the 'it doesn't fit' playbook
70B at 128k context OOMs at batch 32. Size the cache per token, then rank GQA/MLA, FP8 KV, paging/preemption, offload and context parallelism by cost and quality risk.
- Scheduling for goodput under separate TTFT and TPOT SLOs
TTFT is fine but inter-token latency grows with conversation length and p99 doubles at peak. Where does the time go, and when do you choose chunked prefill over disaggregation?
- Router and cache-aware load balancing
You add a router in front of 200 replicas. What state does it hold, how do you trade prefix-cache affinity against load, and what happens with a hot system prompt or a replica failure mid-stream?
Learning path stages from the study guide; best 3 resources each
- Foundations: inference arithmeticDerive prefill vs decode bounds, KV bytes per token and tokens/s per GPU from first principles
- 2025
- 2022
- 2026
- Serving toolboxKnow how continuous batching, paging, prefix caching, chunked prefill, disaggregation, speculation, quantization and decode kernels work and what each costs
- 2025
- 2024
- 2023
- Hands-on: serve and measureServe a model with vLLM and SGLang, plot TTFT/ITL/throughput against load, turn on prefix caching, and modify a small engine
- 2025
- 2026
- 2026
- Design the reported promptsRehearse batch API, batched online API, router, Sora fair queueing, chat backend and design-doc critique with numbers
- 2026
- 2026
- 2026
- Senior-staff depthArgue frontier-scale trade-offs (cost per token, KV storage, disaggregation, MoE, reliability, correctness) from production case studies and papers you can critique
- 2025
- 2025
- 2024
Ready whenYou design a batch and an online inference system end to end with GPU counts, latency targets and failure handling.
Start here top picks, newest first
- 2026★Interview notes: LLM Inference API with Batched GPU ServingL7Practise against it after writing your own answer. Check its numbers rather than memorising them, and be ready to change assumptions when pushed.
- 2025★Inside vLLM: Anatomy of a High-Throughput LLM Inference SystemL7The best single source for drawing a real engine's request lifecycle on the whiteboard and saying where queueing, preemption and cache misses happen. It is also the map for the Fellows OA's 'vLLM-like codebase'.
- 2025★A postmortem of three recent issuesL7Frontier-lab production evidence: lets you discuss sticky routing, cross-hardware consistency and quality monitoring in a latency/quality debugging round.
- 2025★nano-vllmL7The best rehearsal for the Anthropic Fellows inference-engine OA and xAI 'implement batching' rounds: add a prefill/decode-aware scheduler or load balancer to it yourself.
- 2025★DeepSeek-V3/R1 Inference System OverviewL7The best public frontier-scale cost-per-token case: lets you reason from H800 counts and tokens per second to dollars per million tokens.
- 2025★How to Scale Your Model — 'All About Transformer Inference' (with the Rooflines chapter)L7Lets you compute 'theoretical max throughput, then actual under a latency SLO' on the whiteboard, which is the exact Anthropic phone-screen prompt, and defend your sharding choice with numbers.
09 ML infrastructure: model distribution, data pipelines 3% of evidence · 21 reports · 9 firsthand
Share by role: Research Engineer 3% · Research Scientist 0% · ML Infra 12% · Applied AI 4% · Software Engineer 2% · staff+: 4%
Most reported problems distinct reports
- 11Distribute model weights to thousands of machinesAnthropic 10, xAI 1 · 4 firsthand · 3 staff+
Recent firsthand questions
- Mistral · 2026-09Keeping GPUs supplied with data (efficient data-preparation pipeline)
- Anthropic · 2026-08Onsite prompt: role-play debugging a production compute job that has a networking problem
- Anthropic · 2026-08Design a platform for shipping tens to hundreds of GB of model weights to a large number of GPU workers; observability when node capacity is opaque or a deployment gets stuck; guarantee requests only route to instances that fully loaded the…
- OpenAI · 2026-08Infrastructure interview: deep dive with a team member on a team-specific area expanding into system design
- Anthropic · 2026-07Design a model downloader distributing a model to many machines where download and upload share a 10Gbps cap; argue the minimal time bound, then design peer-to-peer distribution
At L7The Anthropic senior-staff infra loop's design round was a model downloader that recovers without a coordinator. Distributing large model files to thousands of machines recurs across staff reports.
Senior-staff probes
- Deriving the lower bound before designing
400 GB to 5,000 hosts, each with a 10 Gbps link shared by upload and download, and the origin capped at 10 Gbps. What is the minimum time, and how close does your design get?
- Fan-out topology choice with failure behaviour
Tree, pipeline chain, or swarm? A mid-tree node dies at 60%: what happens to its subtree, and how long does recovery take?
- Coordinator-free recovery and integrity
Your tracker is down for ten minutes mid-rollout. How do peers keep finding chunks, verify them, and avoid a thundering herd on object storage when it returns?
- Safe rollout of weights to serving
How do you guarantee no request ever hits an instance with a partial or wrong-version model, and how do you see that a rollout is stuck when node capacity is opaque?
Learning path stages from the study guide; best 3 resources each
- Numbers and vocabularyDo the bandwidth, memory and failure-rate arithmetic every design here needs
- 2026
- 2025
- 2026
- Model and checkpoint distributionDesign chunked, content-addressed, P2P/tree distribution with coordinator-free recovery
- 2019
- 2026
- 2026
- Data pipelines and checkpointsBuild a loader that saturates GPUs and checkpoints that restore elastically
- 2026
- 2026
- 2026
- Cluster realitiesScheduling, preemption, stragglers and failure handling on real fleets
- 2026
- 2026
- 2026
- Senior-staff depthCritique and cite frontier/hyperscale systems; run the 5,000-host and 1,024-GPU mocks
- 2024
- 2024
- 2023
Ready whenYou design model distribution for 5,000 hosts with numbers for bandwidth, time and failure recovery.
Start here top picks, newest first
- 2026★Machine Learning Engineering Open BookL7Lets you speak from the failure modes real large runs hit (hung NCCL, slow nodes, full disks) instead of textbook ones.
- 2026★Best practices design patterns: optimizing Amazon S3 performanceL7Gives the concrete object-storage limits that force you to add P2P or a cache tier when 5,000 hosts start at once.
- 2026★Mosaic StreamingL7Lets you answer 'resume mid-epoch on a different node count with the same sample order' — a precise staff-level probe.
- 2025★How To Scale Your ModelL7Gives you the bandwidth and memory arithmetic (bytes/s per link, HBM, ICI/NVLink vs DCN) to put real numbers on any distribution or loader design.
- 2024★The Llama 3 Herd of Models (infrastructure sections)L7Lets you quote failure rates and checkpoint bandwidth from a real frontier run when sizing recovery.
- 2024★MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUsL7Lets you discuss detection-to-recovery time at 10k+ GPUs with real numbers, which is the core of 100K-GPU cluster questions.
10 Distributed training and GPU performance 3% of evidence · 26 reports · 11 firsthand
Share by role: Research Engineer 6% · Research Scientist 5% · ML Infra 19% · Applied AI 0% · Software Engineer 1% · staff+: 0%
Most reported problems distinct reports
Nothing in this track recurs in two or more reports; see the recent questions below.
Recent firsthand questions
- Mistral · 2026-09Model parallelism: compare tensor parallelism and pipeline parallelism
- Anthropic · 2026-08Compute distributed mode and median given 10 nodes and 1,000 elements, with network as the bottleneck — minimize data movement; collectives (tree/ring), uniform vs skewed distributions; median follow-up discussed only
- Anthropic · 2026-08Onsite prompt (from candidate portal): system design on scalable, performance-bounded systems under realistic constraints, deriving bounds from first principles
- OpenAI · 2026-04ML coding: simplified GEMM — derive sharded matmul, then matmul + backprop across devices in numpy/pytorch, then find a fixed number of planted bugs
- Anthropic · 2026-03Distributed coding: compute mode and median using supplied messaging interfaces, attending to partitioning and communication costs
At L7No staff-level evidence either way; core for infra and research-engineer roles. “What is 5D parallelism?” and “how would you train a 100B+ model?” are reported questions.
Senior-staff probes
- First-principles memory and step-time estimates
A 100B dense model, 4k sequence, 4M-token global batch on 1,024 H100s: how many bytes per GPU under ZeRO-3 vs TP8×PP4×DP32, including activations, and what step time and MFU do you expect? Show the arithmetic.
- Choosing a parallel layout against the bandwidth hierarchy
Why does TP stay inside the NVLink domain, when does adding PP beat more FSDP sharding, and how does your answer change on a 72-GPU NVLink domain (GB200 NVL72) or on TPU ICI?
- Communication cost modeling and overlap
Derive ring vs tree all-reduce cost for a 1 GB gradient on 1,024 GPUs; when is DP communication exposed; how would you hide MoE all-to-all, and what does NCCL actually do at small message sizes?
- Designing for failures and goodput
Llama 3 saw 419 unexpected interruptions in 54 days on 16K GPUs. For your 1,024-GPU run, estimate failures per day, pick a checkpoint interval, and explain how you detect a silent straggler or NCCL hang.
Learning path stages from the study guide; best 3 resources each
- Foundations: memory and the GPU mental modelDo the 16-bytes-per-param math, explain compute- vs memory-bound, and know the H100 numbers.
- 2026
- 2022
- 2026
- Parallelism and collectivesExplain every parallelism axis, its collectives and memory effect, and combine them into 5D parallelism.
- 2025
- 2025
- 2026
- GPU performance and kernelsRead a profile, reason with rooflines, and write one fused Triton kernel.
- 2026
- 2026
- 2026
- Hands-onRun DDP then FSDP2/TP on rented GPUs, and do the Anthropic take-home in a 2-hour timebox.
- 2026
- 2026
- 2026
- Senior-staff depthLay out a 100B run on 1,024 GPUs with step-time estimates and failure handling, and critique production designs.
- 2024
- 2026
- 2025
Ready whenYou explain 5D parallelism on a whiteboard and lay out a 100B run with numbers.
Start here top picks, newest first
- 2026★Stanford CS336: Language Modeling from Scratch (Spring 2026) — GPUs/TPUs, Kernels/Triton, Parallelism lecturesL7Assignment 2 gives you a real artifact (your own Triton FlashAttention + DDP/sharded optimizer) to cite instead of describing papers secondhand.
- 2026★Anthropic's original performance take-homeL7This is the actual Anthropic interview task; timeboxing it to 2 hours and reaching the README thresholds is the most direct preparation available.
- 2026★Triton tutorialsL7Writing fused softmax and matmul yourself is the minimum to credibly discuss fusion and tiling; the attention tutorial is FlashAttention in ~200 lines.
- 2026★TorchTitanL7Running real multi-dimensional parallel configs and reading the traces is the hands-on step that turns paper knowledge into numbers you measured yourself.
- 2025★The Ultra-Scale Playbook: Training LLMs on GPU ClustersL7This is the direct source for 'what is 5D parallelism?'; its benchmark section lets you justify a layout with measured MFU instead of folklore.
- 2025★How to Scale Your Model: A Systems View of LLMs on TPUsL7The worked problems train exactly the back-of-envelope step-time and memory math needed to lay out a 100B run on 1,024 GPUs and defend it under questioning.
11 Applied LLM: RAG, agents, evals 7% of evidence · 55 reports · 24 firsthand
Share by role: Research Engineer 6% · Research Scientist 4% · ML Infra 0% · Applied AI 48% · Software Engineer 6% · staff+: 3%
Most reported problems distinct reports
- 16Agents and tool useAnthropic 10, OpenAI 3, GDM 1 · 6 firsthand
- 13Evals and LLM-as-judgeOpenAI 4, GDM 3, xAI 2 · 6 firsthand
- 11RAG / enterprise search with citationsOpenAI 5, Cohere 2, Unnamed lab 2 · 3 firsthand
- 6Design ChatGPT / a chat assistantOpenAI 5, xAI 1 · 4 firsthand · 1 staff+
Recent firsthand questions
- Anthropic · 2026-10Agent coding interview: using provided starter code (Anthropic API client, tool-definition templates, basic agentic loop), optimize an agent by creating/modifying tools the LLM can call, implementing the tool logic, and adjusting prompting;…
- OpenAI · 2026-09Coding round: evaluate the quality of a classifier (question inspired by arXiv 2201.08555)
- OpenAI · 2026-08Design a real-time/streaming-response AI product feature for a large number of users: streaming API (SSE/websockets), rate limiting and queuing, caching and token cost control, frontend streaming error handling, observability
- Anthropic · 2026-07System design: a prompt playground — database schema, then how to handle large prompts and a share feature; storing tiny edits to large prompts (deltas) and where to store them
- Anthropic · 2026-07System design: prompt playground (low-level details probed)
At L7Lower in staff+ reports than overall, but core for applied and forward-deployed roles. OpenAI applied loops ask you to design ChatGPT itself; Anthropic asks about agents and enterprise deployment.
Senior-staff probes
- Serving economics of a chat product
How much KV-cache memory does a 100k-token conversation take, and how do prefix caching and disaggregation change cost per request and your capacity plan?
- Retrieval trade-offs with evidence
The index grew 10x and answers got worse. What is your diagnosis plan, and when does a reranker earn its latency?
- Eval statistics and ship gates
Model B beats A by 2 points on 500 questions. Do you ship? How many examples would you need, and how do you handle clustered questions?
- Judge validity
How do you know your LLM judge is right, and what do you do when the metric rises but users say outputs feel worse?
Learning path stages from the study guide; best 3 resources each
- Design ChatGPTDraw a chat assistant end to end: SSE streaming, conversation state, caching, rate limits, cost per request.
- 2025
- 2024
- 2026
- Build RAG onceBuild hybrid search + rerank with citations and measure retrieval recall separately from answer quality.
- 2025
- 2024
- 2024
- Build an agent with evalsWrite a tool loop, improve its tools, run error analysis and put error bars on a regression set.
- 2026
- 2026
- 2025
- Production debuggingRehearse incidents: stateless replicas losing context, 20x bill spikes, 'the model got worse', hallucination complaints.
- 2026
- 2026
- 2025
- Senior-staff depthRead the lab case studies and papers you should be able to critique, with serving, retrieval and eval numbers.
- 2025
- 2024
- 2025
Ready whenYou show an eval with error bars that caught a regression in something you built.
Start here top picks, newest first
- 2026★AI Evals: Everything You Need to Know (Evals FAQ)L7Covers the eval-design questions that recur in applied loops; read all of it.
- 2026★Demystifying evals for AI agentsL7Gives you Anthropic's own vocabulary for agent eval design and the reliability metrics (pass^k) to set ship gates.
- 2025★Effective context engineering for AI agentsL7Gives you a defensible position for 'thoughts on context engineering' and for long-running agent memory, with named techniques and their trade-offs.
- 2025★Writing effective tools for AI agents—using AI agentsL7Directly prepares the Anthropic agent-coding round (improve an agent by changing tools) and 'how many tools is too many'.
- 2025★How we built our multi-agent research systemL7A real frontier-lab system with numbers: lets you discuss when multi-agent is worth the token bill and how to evaluate and deploy stateful agents.
- 2025★AI Engineering: Building Applications with Foundation ModelsL7Gives you a decision framework with costs for prompting vs RAG vs fine-tuning and a full platform architecture to anchor any design answer.
12 Research discussion and research taste 4% of evidence · 26 reports · 14 firsthand
Share by role: Research Engineer 13% · Research Scientist 20% · ML Infra 0% · Applied AI 0% · Software Engineer 0% · staff+: 2%
Most reported problems distinct reports
Nothing in this track recurs in two or more reports; see the recent questions below.
Recent firsthand questions
- Anthropic · 2026-08Anthropic Fellows take-home research assignment: run a set of experiments (at least 6 questions), fill in provided slides with results, and record a ~15-minute presentation video
- Anthropic · 2026-06Anthropic Fellows take-home: ML performance option, and a ~5-hour experiment take-home
- OpenAI · 2026-04Collaboration interview: current SOTA in your research direction, pros and cons, how the field will develop, and how to apply it to product
- OpenAI · 2026-04Research presentation of your recent results; questions on motivation, why/how, and whether your experimental design was sound
- Anthropic · 2026-03Anthropic Fellows 15-minute open-ended research interview testing research intuition
At L7Google L7 research loops go over existing papers, how your research fits products, and leading research teams. Expect to argue a research direction, not just discuss a paper.
Senior-staff probes
- Choosing and defending a direction
Why is this the most important problem for this team in the next six months, what would make you drop it, and what is the first experiment?
- Five-minute paper critique
What is the weakest link in the evidence for 'refusal is mediated by a single direction', and what experiment would break the claim?
- Experimental rigor
Your ablation shows +1.5 points. How many seeds, what variance, what tuning budget did the baseline get, and what is the confound?
- Fast black-box empiricism
You have five hours and an API key: how would you test whether the model has property X, and what controls rule out prompt artefacts?
Learning path stages from the study guide; best 3 resources each
- One paper a weekRead with a fixed template (claim, method, evidence, weaknesses, next experiment) and critique aloud in 5 minutes.
- 2026
- 2026
- 2026
- Research brainstorm drillsGiven a question, propose three experiments in 10 minutes and rank them by information per unit time.
- 2025
- 2025
- 2024
- Reproduce one recent resultSmall-scale reproduction from a public repo with error bars and a short write-up.
- 2024
- 2026
- 2026
- Research agenda and talkWrite a six-month agenda per target team and rehearse a 20-minute talk with hostile questions.
- 2026
- 2025
- 2012
- Senior-staff depthBe able to critique the 2025–2026 papers that set current agendas and defend experimental rigor with statistics.
- 2025
- 2025
- 2025
Ready whenYou critique a recent paper in 5 minutes and pitch a six-month agenda for each target team.
Start here top picks, newest first
- 2026★Notes on the Industry Job SearchL7The most current description of what these rounds look like; read before the loop.
- 2025★My Research Process: Understanding and Cultivating Research TasteL7Gives you a precise answer to 'how do you judge which experiment to run' at senior level.
- 2025★How I Think About My Research Process: Explore, Understand, DistillL7Lets you narrate a messy black-box investigation as a deliberate process, which Fellows-style rounds score.
- 2025★Recommendations for Technical AI Safety Research DirectionsL7The best input for a six-month agenda pitch to an Anthropic team and for alignment brainstorms.
- 2025★DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningL7You must be able to explain and critique the paper that set 2025's reasoning agenda.
- 2024★Adding Error Bars to Evals: A Statistical Approach to Language Model EvaluationsL7Lets you put correct error bars on take-home results, which Anthropic reviewers check.
13 Values, mission and safety views 11% of evidence · 77 reports · 37 firsthand
Share by role: Research Engineer 1% · Research Scientist 5% · ML Infra 19% · Applied AI 10% · Software Engineer 12% · staff+: 19%
Most reported problems distinct reports
- 38AI risk and safety views (p(doom), how AI goes wrong)Anthropic 29, OpenAI 7, Unnamed lab 2 · 17 firsthand · 1 staff+
- 34Why this lab / mission fitAnthropic 22, OpenAI 9, xAI 3 · 19 firsthand · 4 staff+
- 15Ethics under pressure (did something wrong, put neck on line)Anthropic 12, OpenAI 2, Unnamed lab 1 · 5 firsthand · 1 staff+Prepare withAnthropic Behavioral Interview Field GuideJustice with Michael Sandel
- 11Critique the lab / disagree with itAnthropic 11 · 3 firsthand · 2 staff+
- 6Stock to zero / revenue vs safetyAnthropic 6 · 0 firsthand
Recent firsthand questions
- Anthropic · 2026-09Why Anthropic?
- Anthropic · 2026-09Why leave your current company? Why AI companies?
- Anthropic · 2026-09What is your p(doom)?
- OpenAI · 2026-09Behavioral with a manager: AI safety views plus Amazon-style behavioral questions
- xAI · 2026-09Recruiter call: background, strongest languages, why xAI
At L7Reported as the most common reason senior candidates fail at Anthropic; a Google-tagged commenter said most of his rejected referrals failed on culture. Senior-staff questions: “what evidence shows our sincerity about the mission?”, “critique our approach”, “something morally wrong you did”, “criteria to release a model”. Answers must work for a non-technical interviewer.
Senior-staff probes
- Evidence of mission sincerity
Name two decisions where Anthropic paid a real cost for safety, and one where you think it didn't.
- Model release criteria
You own the launch decision. What evals, thresholds and safeguards gate release, and who can overrule you?
- Calibrated risk view
What's your p(doom), what drives it, and what evidence in the next year would move it by half?
- Specific critique of the lab
Where is Anthropic's approach wrong or self-serving? Use a specific RSP change or launch.
Learning path stages from the study guide; best 3 resources each
- Each lab in its own wordsRead every target lab's mission and safety framework with dates
- 2026
- 2026
- 2026
- Form your own viewsA p(doom), timeline and biggest-risk view you can defend against both optimists and pessimists
- 2025
- 2026
- 2026
- Write the one-pagers and ethics storiesOne page per question and three 3-minute ethics stories
- 2026★Anthropic Behavioral Interview Field Guide
- 2026
- 2026
- 2026
- Senior-staff depthCritique specific lab decisions with primary evidence and propose what you would do
- 2025
- 2026
- 2026
Ready when12 values questions in 45 minutes with specific stories and views you'd still hold in a year.
Start here top picks, newest first
- 2026★Anthropic Behavioral Interview Field GuideL7Lets you rehearse the exact reported prompts so your 3-minute ethics stories are specific, not generic.
- 2026★Anthropic's Responsible Scaling Policy (v3.4, with Risk Reports)L7Lets you discuss release criteria and critique specific changes (e.g. revised automated-R&D threshold) with dates, which few candidates can.
- 2026★Behavioral Coaching Notes: Anthropic Philosophy and OpenAI Technical Deep DivesL7Current calibration of what culture interviewers at Anthropic and OpenAI reward from senior candidates.
- 2026★Frontier Safety Framework (v3.1)L7Third data point for comparing release criteria and security commitments across labs.
- 2026★Claude's ConstitutionL7Lets you argue concrete value trade-offs (helpfulness vs harm, honesty) in Anthropic's own framing during hypotheticals.
- 2026★The Adolescence of TechnologyL7The latest statement of Anthropic leadership's risk view; quote it accurately and say where you differ.
Study guides
One page per track with courses (Stanford and others, with lecture videos), tutorials, references, survey papers, key papers and practice sets, calibrated to the senior-staff bar and sorted newest first.
01Practical coding, no AI
140 resources · 12 courses · 80 from 2025–26 · 57 advanced02Algorithms (LeetCode-style)
50 resources · 9 courses · 31 from 2025–26 · 22 advanced03Math and ML fundamentals
101 resources · 11 courses · 16 from 2025–26 · 53 advanced04ML coding from scratch and debugging
72 resources · 7 courses · 32 from 2025–26 · 34 advanced05LLM internals: architecture, pretraining, post-training
90 resources · 11 courses · 44 from 2025–26 · 46 advanced06Project deep dive and leadership stories
61 resources · 4 courses · 27 from 2025–26 · 25 advanced07Product and distributed system design
81 resources · 8 courses · 42 from 2025–26 · 36 advanced08LLM inference and serving design
95 resources · 5 courses · 62 from 2025–26 · 52 advanced09ML infrastructure: model distribution, data pipelines
67 resources · 6 courses · 30 from 2025–26 · 34 advanced10Distributed training and GPU performance
74 resources · 6 courses · 36 from 2025–26 · 39 advanced11Applied LLM: RAG, agents, evals
108 resources · 14 courses · 60 from 2025–26 · 44 advanced12Research discussion and research taste
86 resources · 2 courses · 51 from 2025–26 · 44 advanced13Values, mission and safety views
74 resources · 7 courses · 55 from 2025–26 · 32 advanced
Method and sources
How the plan was built
- Collect. 1,245 questions from 369 reports: 215 firsthand, 94 question bank, 34 prep site, 25 relayed, 1 official. Sources: PracHub, Exponent/Aced candidate experiences, Blind, LeetCode Discuss, Reddit (via the Arctic Shift archive), Hacker News, candidate blogs, X, and prep-site question banks. Each question has a link.
- Tag. Each question got a lab, role, seniority (only when stated), round type and one of 22 topics, merged into 13 study tracks.
- Weight. One report counts once per track. Firsthand 1.0, relayed 0.6, prep site 0.35, question bank 0.25; 2026 ×1.0, 2025 ×0.8, 2024 ×0.6. Big-tech questions not specific to the AI labs ×0.3.
- Adjust for L7. Staff+ share ÷ overall share, shrunk by 0.47 (only 18 staff+ reports) and capped at 0.7–1.6: Practical coding ×0.86, Algorithms ×0.92, Deep dive & leadership ×1.42, System design ×1.10, Inference design ×1.42, ML infra ×1.21, Applied LLM ×0.73, Values & mission ×1.36. ML-knowledge tracks stay at ×1.00 because staff+ reports are almost all engineering roles.
- Resources. One study guide per track: 1,099 links checked by HTTP request, at least 40% advanced, each with a note on what it adds at senior-staff level. All 1,042 question groups are mapped to at least one resource. Learning paths, weekly reading and the core library on this page come from those guides.
- Allocate. Hours ∝ role share × L7 weight × your gap. Role shares are shrunk toward the all-roles mix (20-report prior); values and deep dive use the all-roles share for every role. Week placement follows a fixed order: fundamentals and coding first, scale and design in the middle, values and mocks last.
Read with care
- About half the firsthand reports are for software-engineer roles; research roles rest on dozens, not hundreds.
- PracHub pages are curated and often translated from other forums. Blind is anonymous. Several question banks copy each other; copies were grouped and down-weighted.
- Seniority is tagged only when a report states it, so staff+ evidence is thin. No firsthand OpenAI L6, DeepMind L7 research or MSL E7 loop from 2025–26 was found.
- Glassdoor and 1point3acres were not usable (login walls). No lab publishes its questions or leveling rules.
- Gap defaults assume a staff-level ML engineer who is new to LLMs; they are an estimate, not a measurement.
Where the questions came from
| Source | Reports |
|---|---|
| Exponent / Aced | 98 |
| PracHub | 90 |
| 84 | |
| Blind | 31 |
| LeetCode Discuss | 21 |
| X | 11 |
| Hello Interview | 8 |
| HN | 5 |
| GitHub repos | 5 |
| techinterview.org | 4 |
| interviewing.io | 3 |
| Jobright.ai | 2 |
| Candidate blogs | 2 |
| Axios (via HN) | 1 |
Key context sources: Alisa Liu, Notes on the industry job search (Jun 2026) · Anthropic, AI-resistant technical evaluations (Jan 2026) · Anthropic candidate AI guidance · mlmentorship lab loops (Aug 2026) · levels.fyi and Blind for leveling (Oct 2026) · an X interview report (Sep 2026).