Algorithm & ML Interview: 15 In-Depth Questions

③ Quantified high-score answer

Under extreme skew, naive oversampling causes overfitting while aggressive undersampling discards valuable variance. In production, I combine loss engineering with realistic evaluation. On the loss side, I adopt Focal Loss or Class-Balanced Loss: the modulating factor dynamically suppresses gradient updates from easy negative examples, forcing backpropagation to focus on hard minority instances. If negative downsampling is necessary to control compute budgets, I apply posterior calibration post-inference to map predicted scores back to true unskewed probabilities. For evaluation, I avoid ROC-AUC because colossal true-negative denominators mask false-positive explosions; instead, I monitor PR-AUC and measure recall at strict business-mandated precision thresholds to reflect real commercial trade-offs.

③ Quantified high-score answer

Gradient instability arises when consecutive Jacobian matrix multiplications amplify or decay gradients across deep layers. Stabilizing convergence requires three complementary defenses. First, residual connections formulate layer transformations as y = x + F(x), ensuring the derivative with respect to input always contains an identity term, creating an unattenuated gradient highway. Second, we use LayerNorm rather than BatchNorm: LayerNorm computes statistics across hidden dimensions per individual sample, eliminating instability caused by varying sequence lengths and small batch sizes. For very deep Transformers, Pre-LN or RMSNorm eliminates gradient scale growth. Third, we enforce gradient norm clipping and transition to BF16 mixed-precision to eliminate exponent underflow and loss explosion.

③ Quantified high-score answer

Selecting ANN algorithms requires balancing memory footprint, search latency, and recall accuracy. HNSW constructs hierarchical navigable small-world graphs: using skip-list-like long-range jumps at upper layers and greedy local searches at lower layers, it achieves sub-millisecond query latency and high recall (>95%). However, storing edge lists inflates memory by 1.5 to 2 times, making it ideal for ten-million-scale, latency-critical real-time retrieval. In contrast, IVF-PQ narrows candidate clusters via inverted file lists (IVF) and compresses vectors into compact byte codes via Product Quantization (PQ) across sub-spaces, slashing memory usage tenfold. In billion-scale production pipelines, we combine coarse IVF-PQ retrieval with full-precision re-ranking.

③ Quantified high-score answer

Standard self-attention is bounded by memory IO rather than raw compute FLOPs: reading and writing intermediate O(N^2) attention matrices between GPU high-bandwidth memory (HBM) and fast on-chip SRAM creates heavy latency bottlenecks. FlashAttention resolves this through IO-aware tiling. By partitioning Q, K, and V matrices into blocks that fit within SRAM, it uses Online Softmax to dynamically update running row maximums and scaling factors incrementally without ever materializing the full N-by-N attention matrix in HBM. During backward passes, instead of loading intermediate weights from HBM, it recomputes gradients directly in SRAM from cached Q and K tiles. This reduces peak memory complexity from quadratic to linear O(N).

③ Quantified high-score answer

Data leakage creates artificially inflated offline metrics that collapse upon deployment. Preventing it requires three architectural safeguards. First, enforce point-in-time joins for dataset creation: feature values must reflect historical state strictly before the event timestamp, and all scaling transformers must fit exclusively on training splits. Second, when applying target encoding to high-cardinality categoricals, use out-of-fold splits or additive Laplacian smoothing noise to prevent label contamination from leaking directly into input representations. Third, bridge the training-serving gap by establishing schema contracts and versioned feature stores: we capture online inference snapshots and run automated Kolmogorov-Smirnov distribution parity checks against training baselines to eliminate subtle pipeline drift.

③ Quantified high-score answer

Industrial recommendation systems balance multi-objective optimization against strict millisecond latency budgets using tiered candidate convergence. The retrieval layer filters millions of candidates down to thousands via concurrent heterogeneous routes, including dual-tower vector embeddings and graph-based walks. The pre-ranking layer applies lightweight neural models to trim the set to hundreds, balancing throughput against candidate loss. The ranking layer deploys complex multi-task architectures (such as MMoE or PLE) with deep cross-features, simultaneously predicting click-through rate, dwell time, and conversion probabilities. Finally, the re-ranking layer moves beyond greedy item-level scoring: it runs Determinantal Point Processes (DPP) over top candidates to maximize joint relevance while enforcing topic diversity and fresh exploration.

③ Quantified high-score answer

Large model memory comprises static states—parameters, gradients, and Adam states consuming 16 bytes per parameter—and dynamic activations. Designing distributed architectures requires aligning parallel strategies with physical interconnect topologies. High-bandwidth NVLink within an 8-GPU chassis supports Tensor Parallelism (TP), which splits intra-operator matrix multiplications across GPUs with low communication latency. Across server nodes, limited network bandwidth favors Pipeline Parallelism (PP) or Data Parallelism. For hundred-billion-scale training, the most effective baseline combines ZeRO-3/FSDP with overlapping communication: sharding parameters, gradients, and optimizer states across the entire cluster while fetching weights dynamically via all-gather, scaling beyond physical device limits without altering model architectures.

③ Quantified high-score answer

Autoregressive decoding generates one token per step, making throughput bounded by memory bandwidth rather than compute. Traditional KV Cache pre-allocates contiguous memory for maximum context lengths, resulting in 60% to 80% memory waste from internal fragmentation. In production, we deploy PagedAttention (as implemented in vLLM): drawing inspiration from virtual memory paging, it breaks KV Caches into fixed-size logical blocks mapped dynamically to non-contiguous physical GPU pages. This enables memory sharing across concurrent requests and multiplies serving capacity. Paired with 4-bit AWQ weight quantization, which inspects activation distributions to preserve salient outlier channels while quantizing secondary weights, inference memory footprint drops by half without accuracy degradation.

③ Quantified high-score answer

Ensuring training-serving consistency requires unifying feature definitions at the architectural source. We implement a production Feature Store governed by three principles. First, a centralized metadata catalog where every feature is defined once; unified stream-batch engines compile logic into Flink streaming jobs for real-time updates and Spark batch jobs for historical aggregation. Second, for online serving, an in-memory cluster like Aerospike or Redis stores pre-materialized entity features serialized into binary blobs, allowing batch MGET retrievals to return hundreds of features within 5 milliseconds. Third, for training data generation, the platform performs point-in-time joins: matching label events with historical feature snapshots captured at the exact moment of interaction, completely eradicating feature drift.

③ Quantified high-score answer

Model degradation is an inevitable consequence of shifting user behavior and macroeconomic environments. We combat decay through proactive MLOps automation. Because true ground-truth labels often face attribution delays, we monitor input and output distributions upstream: calculating Population Stability Index (PSI) and Kolmogorov-Smirnov statistics on inference features and prediction scores, triggering automated alerts when PSI exceeds 0.1. On the pipeline side, continuous training workflows periodically ingest sliding-window data for incremental fine-tuning. Before deployment, new candidate models undergo automated shadow evaluation alongside the active champion: asserting that latency, calibration, and guardrail metrics remain within bounds before canary traffic shifting, delivering fully automated, self-healing model lifecycles.

③ Quantified high-score answer

Resolving conflicts between immediate KPIs and long-term ecosystem health requires quantitative evidence and architectural trade-offs rather than abstract debates. When optimizing purely for click-through rate led to clickbait saturation and a 15% decline in reading duration, I addressed the problem in three steps. First, I quantified the downstream cost: demonstrating that cohorts exposed to homogeneous content suffered accelerated second-week churn, proving that short-term clicks were cannibalizing user equity. Second, I restructured the objective function: shifting from raw CTR to a multi-task blend incorporating completion rate, favorites, and negative feedback penalties. Third, I allocated a 5% traffic budget to Contextual Bandits for cold-start category exploration, safeguarding baseline revenue while sustaining catalog vitality.

③ Quantified high-score answer

Discrepancies between offline metrics and online gains highlight classic offline-online divergence. My triage follows three stages: mitigation, telemetry validation, and structural debiasing. First, I revert traffic immediately to the control baseline to safeguard business performance. Second, I compare online feature logs against offline training dumps, checking for missing values, feature leakage, or version mismatches. Third, I tackle structural bias: offline AUC measures global pairwise ranking across historical impressions, but online serving operates across truncated candidate pools burdened by position bias. To fix this, we implemented Entire Space Multi-task Models (ESMM) to eliminate sample selection bias and introduced position-debiasing towers during training, aligning offline validation with real serving funnels and successfully turning subsequent experiments positive.

③ Quantified high-score answer

Handling high-concurrency serving failures demands disciplined incident command, tiered degradation, and strict circuit breaking. When latency spiked during a major sales launch, I executed pre-established failover protocols immediately. Level 1: dynamically trimmed ranking candidates from 500 to 150 items, slashing GPU matrix compute by 70%. As gateway queues remained elevated, I triggered Level 2: bypassing the heavy neural ranking model entirely and serving pre-ranking scores directly. If downstream latency breached critical thresholds, Level 3 failover engaged: routing requests to pre-warmed Redis cache clusters with static popular recommendations to guarantee zero client-side crashes. Post-incident root cause analysis revealed queuing bottlenecks in dynamic batching; we implemented adaptive request shedding and isolated GPU resources across critical tiers.

③ Quantified high-score answer

Compliance scrutiny is a necessary safeguard against operational risk rather than an obstacle to innovation. Building trust requires delivering actionable interpretability, auditability, and fallback guardrails. In regulated risk domains, I integrated TreeSHAP frameworks to decompose model outputs into the top three positive and negative feature contributions for every prediction, translating mathematical weights into clear compliance narratives. Furthermore, we implemented counterfactual explanations: informing stakeholders which metric adjustments would reverse an adverse classification. Architecturally, we deployed a hybrid model: high-confidence predictions process automatically, while borderline cases route to human audit queues governed by hard-coded policy rules. This auditable transparency convinced compliance and risk committees to approve production deployment.