AI & Machine Learning Fundamentals: Interview Questions
Core concepts every AI interview starts with — supervised vs unsupervised learning, overfitting, bias-variance, evaluation metrics, and the vocabulary that shows you understand the basics cold.
easyWhat is the difference between supervised and unsupervised learning?
Supervised learning trains on labelled data — each example has a known target — to predict labels for new inputs (e.g. spam vs not-spam, house prices). Unsupervised learning works on unlabelled data to find structure, such as clustering customers into segments or reducing dimensionality. A third category, reinforcement learning, learns from reward signals through trial and error. The quick tell: supervised needs labels and predicts; unsupervised has no labels and discovers patterns.
Learn this: What Is AI? (AI vs Machine Learning vs Deep Learning) →
easyWhat is overfitting, and how do you prevent it?
Overfitting is when a model learns the training data too closely — including its noise — so it performs well on training data but poorly on new data. Signs: high training accuracy, low validation accuracy. Prevent it with more/represented data, simpler models, regularisation (L1/L2, dropout), early stopping, cross-validation, and holding out a genuine test set. The core idea is to reward generalisation, not memorisation.
mediumExplain the bias-variance tradeoff.
Bias is error from overly simplistic assumptions (underfitting — the model misses real patterns). Variance is error from over-sensitivity to the training data (overfitting — the model changes a lot with small data changes). Increasing model complexity usually lowers bias but raises variance, and vice versa. The goal is the sweet spot that minimises total error on unseen data. Techniques like regularisation and ensembling help manage the balance.
easyWhy split data into training, validation, and test sets?
The training set fits the model; the validation set tunes hyperparameters and guides choices (so you don't peek at the test set); the test set gives a final, unbiased estimate of real-world performance. If you tune against the test set, your reported accuracy becomes optimistic and misleading. Cross-validation rotates the validation split to use data efficiently when data is limited.
mediumWhat are precision and recall, and when do you favour one?
Precision = of the items predicted positive, how many were truly positive (TP / (TP+FP)). Recall = of all actual positives, how many did we catch (TP / (TP+FN)). Favour precision when false positives are costly (e.g. flagging a legit transaction as fraud annoys customers); favour recall when missing a positive is costly (e.g. cancer screening). The F1 score balances the two. Which matters depends on the real-world cost of each error type.
easyWhat's the difference between classification and regression?
Both are supervised learning. Classification predicts a discrete category (spam/not-spam, which digit); regression predicts a continuous number (price, temperature). The distinction drives your choice of model output, loss function (e.g. cross-entropy vs mean-squared-error), and metrics (accuracy/F1 vs RMSE/MAE). Some problems can be framed either way — predicting an exact age (regression) vs an age bracket (classification).
mediumWhat is feature engineering and why does it matter?
Feature engineering is transforming raw data into inputs that help a model learn — scaling numbers, encoding categories, creating ratios or date parts, handling missing values. For classic ML on tabular data it's often the biggest lever on performance: good features can make a simple model beat a complex one on poor features. Deep learning automates some of this by learning representations, but thoughtful features and clean data still matter enormously.
hardA model performs well in testing but poorly in production. What could be wrong?
Common causes: data drift (production data differs from training data over time), train/serve skew (features computed differently in training vs serving), data leakage (a feature in training secretly encoded the label and isn't available at prediction time), an unrepresentative test set, or feedback loops where the model's own actions change the data. Diagnose by comparing training vs production data distributions, auditing feature pipelines for consistency, and checking for leakage. This is really an MLOps question about monitoring and pipelines, not just modelling.
easyWhat is gradient descent and how does it work?
Gradient descent is an iterative optimization algorithm that minimizes a loss function by adjusting model parameters in the direction opposite to the gradient of the loss with respect to those parameters. At each step it computes the gradient, then updates weights: w := w - learning_rate * gradient. The learning rate controls step size. Variants differ by how much data is used per update: batch gradient descent uses the full dataset, stochastic (SGD) uses one example, and mini-batch uses a small subset, trading off gradient noise against computational efficiency. It converges to a minimum of convex functions; for non-convex problems like neural networks it finds a local minimum or saddle-adjacent region, which in practice is often good enough.
easyWhat is a loss function, and how do you choose one?
A loss function measures how far a model's predictions are from the true values; training minimizes it. The choice depends on the task. For regression, mean squared error (MSE) penalizes large errors heavily and assumes Gaussian noise, while mean absolute error (MAE) is more robust to outliers. For binary classification, use binary cross-entropy (log loss), which penalizes confident wrong predictions; for multi-class, use categorical cross-entropy. Hinge loss is used for SVMs. The loss should match the output distribution and the cost of different error types in your problem. Note the distinction from an evaluation metric: the loss must be differentiable for gradient-based training, whereas a metric like accuracy is what you actually care about but may not be optimizable directly.
mediumWhat is the difference between L1 and L2 regularization?
Both add a penalty on model weights to the loss to reduce overfitting, but they differ in the penalty term and effect. L1 (Lasso) adds the sum of absolute values of weights (lambda * sum|w|). Its gradient is constant, which drives some weights exactly to zero, producing sparse models and performing implicit feature selection. L2 (Ridge) adds the sum of squared weights (lambda * sum w^2), shrinking weights smoothly toward zero but rarely to exactly zero, which handles correlated features more gracefully. L1 is preferred when you expect few relevant features or want an interpretable, sparse model; L2 when many features contribute a little. Elastic Net combines both. The hyperparameter lambda controls regularization strength: larger lambda means stronger penalty and simpler models.
easyWhat is k-fold cross-validation and why is it useful?
K-fold cross-validation is a resampling technique for estimating how well a model generalizes. The data is split into k equal folds; the model is trained k times, each time using k-1 folds for training and the remaining fold for validation, and the results are averaged. This uses all data for both training and validation, giving a more reliable, lower-variance performance estimate than a single train/test split, which is especially valuable on small datasets. Common choices are k=5 or k=10. For classification with imbalanced classes, use stratified k-fold to preserve class proportions in each fold. For time series, use forward-chaining splits instead, since standard k-fold would leak future information into the past. Its main cost is training k times.
easyWhat is a confusion matrix and what can you compute from it?
A confusion matrix is a table that summarizes classification results by comparing predicted versus actual labels. For binary classification it has four cells: true positives (TP), true negatives (TN), false positives (FP, type I error), and false negatives (FN, type II error). From these you derive key metrics: accuracy = (TP+TN)/total; precision = TP/(TP+FP), how many predicted positives are correct; recall/sensitivity = TP/(TP+FN), how many actual positives were caught; specificity = TN/(TN+FP); and F1 = harmonic mean of precision and recall. It reveals what kind of errors a model makes, not just overall accuracy, which is essential for imbalanced problems where accuracy can be misleadingly high. It extends to multi-class as an NxN grid.
mediumWhat is ROC-AUC and how do you interpret it?
The ROC curve plots the true positive rate (recall) against the false positive rate (FP/(FP+TN)) across all classification thresholds. AUC is the area under this curve, ranging from 0 to 1. An AUC of 0.5 means the model is no better than random guessing, and 1.0 is perfect separation. Intuitively, AUC equals the probability that the model ranks a randomly chosen positive example higher than a randomly chosen negative one, so it measures ranking quality independent of any single threshold. It is threshold-agnostic and useful for comparing models. A caveat: on highly imbalanced datasets ROC-AUC can look optimistic because the false positive rate stays low; in that case the precision-recall curve and PR-AUC are more informative.
mediumHow do you handle class imbalance in a classification problem?
Class imbalance is when one class vastly outnumbers others, causing models to favor the majority and appear accurate while missing the minority. Approaches fall into several groups. Data-level: oversample the minority (e.g. SMOTE, which synthesizes new examples) or undersample the majority. Algorithm-level: apply class weights so minority errors cost more, which most libraries support directly. Evaluation-level: stop using accuracy and instead track precision, recall, F1, and PR-AUC, and choose the decision threshold deliberately rather than defaulting to 0.5. You can also gather more minority data or reframe as anomaly detection when imbalance is extreme. The right choice depends on the business cost of false negatives versus false positives; always resample only the training folds, never the validation or test set.
mediumWhat is the difference between bagging and boosting?
Both are ensemble methods that combine many weak learners, but they differ fundamentally. Bagging (bootstrap aggregating) trains models in parallel on random bootstrap samples of the data and averages or votes their predictions. Because the models are independent, this mainly reduces variance and helps with overfitting; Random Forest is the classic example. Boosting trains models sequentially, where each new learner focuses on the examples the previous ones got wrong, and combines them as a weighted sum. This mainly reduces bias and can turn weak learners into a strong one; examples include AdaBoost, Gradient Boosting, and XGBoost. Trade-off: bagging is more robust to noise and parallelizable, while boosting usually achieves higher accuracy but is more prone to overfitting noisy data and is harder to parallelize.
mediumHow does a random forest improve on a single decision tree?
A single decision tree splits data recursively on feature thresholds to maximize purity (using Gini impurity or entropy). It is interpretable and handles non-linear relationships, but a deep tree easily overfits: small changes in data can produce a very different tree, meaning high variance. A random forest is an ensemble of many decision trees trained via bagging: each tree sees a bootstrap sample of rows, and at each split only a random subset of features is considered. This decorrelates the trees, and averaging their predictions dramatically reduces variance while keeping bias low, giving better generalization and robustness to noise. The cost is lost interpretability and higher compute, though feature importance and out-of-bag error estimates partly recover insight. In short, the forest trades a single tree's transparency for accuracy and stability.
mediumHow does the k-means clustering algorithm work, and what are its limitations?
K-means is an unsupervised algorithm that partitions data into k clusters. You choose k, initialize k centroids, then iterate two steps until convergence: assign each point to its nearest centroid (by Euclidean distance), then recompute each centroid as the mean of its assigned points. This minimizes within-cluster variance. Limitations: you must specify k in advance (often chosen via the elbow method or silhouette score); results depend on initialization, so k-means++ is used for smarter seeding; it assumes spherical, similarly sized clusters and struggles with elongated or varying-density shapes; it is sensitive to outliers and feature scaling, so standardize features first. For non-spherical clusters, DBSCAN or Gaussian mixture models are better alternatives. It converges to a local optimum, so multiple restarts are common.
hardWhat is the curse of dimensionality?
The curse of dimensionality refers to problems that arise as the number of features grows. As dimensions increase, the volume of the feature space grows exponentially, so data becomes sparse and points spread far apart. This breaks distance-based methods like k-NN and k-means, because in high dimensions the distances between all pairs of points become nearly equal, so 'nearest' loses meaning. It also means the amount of data needed to cover the space and generalize grows exponentially, worsening overfitting when features outnumber samples. Practical consequences include increased compute and unreliable density estimates. Mitigations include dimensionality reduction (PCA, t-SNE, autoencoders), feature selection to keep only informative variables, regularization, and gathering more data. It is a key reason simpler models with fewer features often generalize better.
hardWhat is data leakage and how do you prevent it?
Data leakage occurs when information that would not be available at prediction time leaks into training, producing overly optimistic validation scores that collapse in production. Common forms: target leakage, where a feature is a proxy for or derived from the label (e.g. a 'payment_received' flag when predicting default); train-test contamination, where preprocessing like scaling, imputation, or feature selection is fit on the full dataset before splitting, letting test statistics bleed into training; and temporal leakage, using future information to predict the past. Prevention: split data first, then fit all transformations only on the training fold, ideally inside a pipeline that is cross-validated as a unit; use time-based splits for temporal data; and scrutinize any feature that is suspiciously predictive. A model that seems too good to be true usually has leakage.
mediumWhat are hyperparameters and how do you tune them?
Hyperparameters are the configurations that are set before the learning process begins. These include parameters such as learning rate, the number of epochs, and the depth of a decision tree. Unlike model parameters, which are learned during training, hyperparameters must be chosen prior to training. Tuning hyperparameters can be achieved through methods such as grid search, random search, or more advanced techniques like Bayesian optimization. The goal is to find the combination that yields the best model performance on validation data, which helps avoid overfitting and ensures good generalization to unseen data.
mediumWhat is transfer learning and in what scenarios is it useful?
Transfer learning is a machine learning technique where a model developed for a particular task is reused as the starting point for a model on a second related task. This approach is particularly useful when there is limited labeled data for the new task, allowing practitioners to leverage the knowledge captured in a model trained on a larger dataset. Common scenarios include using pretrained models from large image datasets (like ImageNet) for image classification tasks or leveraging natural language processing models (like BERT) for specific NLP applications. Transfer learning can significantly reduce training time and improve performance in tasks where the available data is scarce.
mediumWhat is a neural network and how does it mimic the human brain?
A neural network is a computational model inspired by the way biological neural networks in the human brain process information. It consists of layers of interconnected nodes (neurons), where each connection has an associated weight. The input data is fed into the input layer, then processed through hidden layers, where neurons apply nonlinear transformations to the data, and finally output predictions from the output layer. During training, the network learns by adjusting the weights based on the error of its predictions, using methods like backpropagation.
mediumWhat are ensemble methods and how do they improve model performance?
Ensemble methods combine multiple models to produce a single, stronger model. The idea is that by aggregating the predictions from a variety of models, you reduce the risk of overfitting and increase accuracy. Common ensemble techniques include bagging, such as Random Forest, which builds multiple trees from random subsets of data, and boosting, such as AdaBoost or XGBoost, which adjusts the weights of misclassified observations to focus on difficult cases. These methods leverage the diversity of models to create robust predictions.
mediumWhat are activation functions in neural networks and why are they important?
Activation functions are mathematical equations that determine the output of a neural network node, or neuron, based on its input. They introduce non-linearity into the model, enabling it to learn complex patterns in data. Without activation functions, a neural network would simply perform linear transformations, limiting its capability. Common activation functions include ReLU (Rectified Linear Unit), sigmoid, and tanh, each having distinct characteristics and use cases, such as ReLU being favored for deep networks due to its performance benefits.
mediumWhy is feature scaling important in machine learning?
Feature scaling is crucial in machine learning because it standardizes the range of independent variables or features of data. This helps in optimizing the performance of algorithms that depend on the distance between data points, such as k-nearest neighbors and gradient-based algorithms like gradient descent. If features have different units or scales, those with larger scales can disproportionately influence the model's prediction, leading to suboptimal performance. Common methods of feature scaling include normalization (scaling features to a range of [0,1]) and standardization (scaling features to have a mean of 0 and a standard deviation of 1).
No questions match your filter.