ml-coding.md

2. ML/Data Coding 🤖

This page is the canonical index for ML coding interview practice. Work through one category at a time: implement the prompt without looking, compare with the linked answer, test edge cases, and explain the trade-offs aloud.

The catalog is intentionally divided by interview skill rather than framework:

1. Classic ML — algorithms, data preparation, metrics, and training fundamentals.
2. Language Models (LM) — tokenization, attention, decoding, and sequence batching.
3. Generative AI (GenAI) — adaptation, retrieval, preference learning, and generation primitives.
4. Agentic AI coding — tools, orchestration, memory, retries, permissions, and termination.

Every catalog row has an answer. Python answers are deliberately interview-sized and self-contained; a few established visual exercises use an existing notebook as their answer.

How to use this chapter

1. Pick a question and restate its input, output, shapes, and assumptions.
2. Implement a correct baseline without opening the answer.
3. Test a normal case and at least one boundary or failure case.
4. Compare with the linked answer, then discuss complexity and production trade-offs.
5. Revisit Easy questions in 15 minutes, Medium questions in 20–35 minutes, and Hard questions in 40–60+ minutes.

Run all Python answer tests from the repository root:

uv run --with numpy python -m unittest discover -s src/MLC/tests -p "test_*.py"

Difficulty colors:

Priority ML coding problems

Classic ML

These questions cover the core implementation patterns expected in general ML, applied-science, and data-focused interviews.

Problem	Difficulty	Tags	Company tags	Answer	Interview focus
Numerically stable softmax and cross-entropy		NumPy, probability, numerical stability	Apple, Meta, Google, Amazon	Python	Max subtraction, log-sum-exp, shapes, class-index validation
Linear regression with gradient descent		regression, optimization, vectorization	—	Python · Notebook	MSE gradients, bias, feature scale, convergence
Logistic regression with gradient descent		classification, optimization, numerical stability	Google, Meta, Amazon	Python · Notebook	Stable sigmoid, binary targets, gradients, decision threshold
k-nearest neighbors		distance, vectorization, classification	Uber, LinkedIn, Meta	Python · Notebook	Pairwise distances, top-k selection, ties, time and memory cost
k-means clustering		clustering, unsupervised learning, convergence	Uber, LinkedIn, Google, Amazon	Python · Notebook	Initialization, assignment/update steps, empty clusters
Decision-tree split		trees, Gini impurity, search	—	Python · Notebook	Candidate thresholds, weighted impurity, stopping conditions
Principal component analysis		linear algebra, dimensionality reduction, SVD	—	Python	Centering, component ordering, explained variance
2D convolution		deep learning, computer vision, tensor shapes	—	Python · Notebook	Output shape, stride, cross-correlation versus convolution
Binary metrics and ROC-AUC		evaluation, ranking, class imbalance	—	Python	Zero denominators, tied scores, rank interpretation
Reservoir sampling		streaming, probability, sampling	—	Python	Unknown stream length, uniform inclusion probability, O(k) memory
Linear SVM and hinge loss		margin methods, optimization, classification	—	Notebook	Hinge loss, margin violations, regularization
Perceptron learning rule		online learning, linear classification	—	Notebook	Update condition, convergence assumptions, decision boundary
Feedforward neural network and backpropagation		neural networks, chain rule, gradients	—	Notebook	Forward cache, gradient flow, parameter updates, shape checks
Multiclass metrics and macro averaging		evaluation, confusion matrix, averaging	—	Python	Per-class counts, macro versus micro, absent classes
Multinomial Naive Bayes for text		NLP, probability, smoothing	—	Python	Log space, Laplace smoothing, unseen tokens, independence assumption
Matrix factorization for recommendations		recommendation, embeddings, SGD	—	Python	Observed entries, regularization, latent rank, cold start
Gradient-boosting residual step		ensembles, residual fitting, regression trees	—	Python	Negative gradients, weak learner, learning-rate shrinkage
Train/validation/test split without leakage		data splitting, reproducibility, stratification	—	Python	Disjoint indices, class balance, train-only fitting
Feature standardization with training statistics		preprocessing, leakage, numerical stability	—	Python	Fit/transform separation, constant features, shape validation
Missing values and unseen categories		preprocessing, imputation, categorical encoding	—	Python	Training-only state, unknown bucket, all-missing columns
Uniform, weighted, and stratified sampling		sampling, imbalance, reproducibility	—	Python	Replacement, normalized weights, per-class guarantees
Sample-weighted losses and streaming metrics		evaluation, streaming, distributed aggregation	—	Python	Weight by sample count, mergeable state, empty state

Language Models (LM)

These questions move from text features to the token, attention, and decoding mechanics used by modern language models.

Problem	Difficulty	Tags	Company tags	Answer	Interview focus
TF-IDF		NLP, sparse features, information retrieval	—	Python	Token counts, document frequency, smoothing, vocabulary order
Scaled dot-product attention		attention, transformer, masking	—	Python	Q/K/V shapes, 1/sqrt(d_k), masking before stable softmax
Learn and apply byte-pair encoding		tokenization, BPE, vocabulary	—	Python	Pair counts, deterministic merges, unseen-word encoding
Build a causal attention mask		transformer, masking, autoregressive decoding	—	Python	Query/key semantics, padding, boolean-mask convention
Sinusoidal positional encoding		transformer, position, vectorization	—	Python	Alternating sine/cosine channels, odd widths, shape
Temperature, top-k, and top-p sampling		decoding, probability, generation	Anthropic, OpenAI, DeepMind	Python	Filtering order, renormalization, deterministic tests
Append-only KV cache		inference, attention, latency, memory	Anthropic, OpenAI, Meta, Perplexity	Python	Cache shape, capacity, prefill versus decode, memory growth
Pad variable-length token batches		batching, padding, masks	—	Python	Padding ID, validity mask, empty sequences, batch shape

Generative AI (GenAI)

These are compact implementations of common fine-tuning, retrieval, alignment, and generation building blocks.

Problem	Difficulty	Tags	Company tags	Answer	Interview focus
LoRA update for a linear layer		fine-tuning, low rank, linear algebra	Meta, Google, Anthropic, OpenAI, Databricks	Python	Matrix shapes, alpha/r scaling, parameter savings, merging
Symmetric contrastive loss		embeddings, InfoNCE, multimodal	OpenAI, Anthropic, DeepMind, Midjourney	Python	Normalization, temperature, in-batch negatives, two directions
Exact cosine top-k retrieval for RAG		RAG, embeddings, retrieval, ranking	—	Python	Normalization, top-k ordering, zero vectors, production ANN trade-offs
Direct Preference Optimization loss		alignment, preference learning, fine-tuning	Anthropic, OpenAI, DeepMind, Meta	Python	Policy/reference log-probability gaps, beta, stable log-sigmoid
Classifier-free guidance combination		diffusion, guidance, generation	—	Python	Conditional/unconditional predictions, guidance scale trade-off

Agentic AI coding

These questions test the deterministic control plane around a model. The reference answers keep permissions, budgets, retries, and termination in code rather than relying on prompt text alone.

Problem	Difficulty	Tags	Company tags	Answer	Interview focus
Validate JSON tool arguments		tools, schema, validation	—	Python	Required fields, types, enums, unexpected arguments
Permission-aware tool registry		tools, permissions, function calling	—	Python	Registration, signature binding, side-effect approval boundary
Planner/executor boundary		planning, orchestration, tracing	—	Python	Bounded plan, structured results, partial failure
Retry with idempotency		reliability, retries, side effects	—	Python	Transient errors, exponential backoff, duplicate suppression
Sliding-window conversation memory		memory, context window, token budget	—	Python	Pinned instructions, recent history, eviction, summarization trade-off
Bounded agent tool-use loop		agents, tools, termination, safety	—	Python	Step budget, repeated-call detection, unknown tools, traceability

Extended practice

• PyTorch ML Coding Problems is the extended framework-specific drill bank. It is supplementary; the four tables above are the canonical question index.
• Legacy notebooks remain available for visual and exploratory practice. The canonical tables link only the notebooks used as current answers or supplements.
• Agentic AI Systems interview prep covers the system-design layer that complements the agentic coding exercises here.

Difficulty and company-tag policy

• Easy: one focused operation, usually about 15 minutes.
• Medium: multiple steps or meaningful edge cases, usually 20–35 minutes.
• Hard: interacting components, advanced debugging, or systems trade-offs, usually 40–60+ minutes.

Difficulty is an editorial judgment about the complete prompt. Exact matches were calibrated against TorchLeet and Deep-ML; unmatched prompts use the same scope-based rubric. Company tags appear only when a reference associates the company with the same implementation problem. They are historical preparation signals, not claims about a current interview loop. This repository uses metadata only and does not copy third-party prompts or answers.

What to explain during the interview

1. State shapes, dtypes, assumptions, and expected outputs before coding.
2. Start with a correct baseline, then optimize the bottleneck.
3. Discuss time and space complexity, including large intermediate matrices and model state.
4. Handle numerical stability, empty inputs, invalid labels, ties, and permission boundaries.
5. Write a normal-case test and at least one failure or boundary test.
6. Explain how the answer changes for large datasets, GPUs, distributed execution, or production services.