K-Nearest Neighbors (KNN) is a simple yet powerful supervised machine learning algorithm used for both classification and regression tasks. It operates on a straightforward principle: given a new data point, the algorithm identifies the K closest data points in the training set (its "nearest neighbors") and makes predictions based on their known labels or values. For classification, the new point is assigned the most common class among its neighbors; for regression, the prediction is the average of the neighbors' values.
KNN is often described as a "lazy learner" because it does not build an explicit model during training. Instead, it stores the entire training dataset and performs all computation at prediction time, comparing the new input against every stored example using a distance metric such as Euclidean, Manhattan, or Minkowski distance. The choice of K — the number of neighbors to consider — significantly impacts performance: too small a value makes the model sensitive to noise, while too large a value can blur the boundaries between classes.
Despite its simplicity, KNN remains a valuable tool in many practical applications including recommendation systems, anomaly detection, image recognition, and medical diagnosis. Its main limitations are computational cost on large datasets and sensitivity to irrelevant features and the curse of dimensionality. Techniques such as KD-trees, ball trees, and dimensionality reduction methods like PCA can help mitigate these challenges and make KNN viable for real-world, large-scale use cases.