What is the curse of dimensionality?

The curse of dimensionality refers to problems that arise as the number of features grows. As dimensions increase, the volume of the feature space grows exponentially, so data becomes sparse and points spread far apart. This breaks distance-based methods like k-NN and k-means, because in high dimensions the distances between all pairs of points become nearly equal, so 'nearest' loses meaning. It also means the amount of data needed to cover the space and generalize grows exponentially, worsening overfitting when features outnumber samples. Practical consequences include increased compute and unreliable density estimates. Mitigations include dimensionality reduction (PCA, t-SNE, autoencoders), feature selection to keep only informative variables, regularization, and gathering more data. It is a key reason simpler models with fewer features often generalize better.