[algorithm] K nearest neighbor algorithm (K-NEAREST NEIGHBOURS (KNN))

KNearest neighbor algorithm (k-nearest neighbours, KNN)

algorithm

1. Classifying an element
2. Look at its K nearest neighbor.
3. Among these neighbors, which is more, this element has greater probability than this category.

 

Use

Use KNN to do two basic tasks.Classification and regression：

•   Classification is grouping.；
•   Regression is the result of prediction.（For example, a number.

 

feature extraction

Used to determine two elements.Similarity degree

 

Method

Pythagorean formula

 

1. Converts a comparable category to a set of coordinates.
2. Pythagorean formula

 

 

Regression (regression)

Regression can predict the result.

 

Method

1. Classifying an element (finding influence factors)
2. Check its K neighbors
3. According to the neighbor’s performance, the average can be calculated.

 

 

Selecting suitable features

Principle

•   And the elements to be analyzedClosely related characteristics；
•   Impartiality（For example, if you only allow users to score a comedy, you can’t tell if they like action movies.

 

Factors that need to be considered must be taken into account in selecting suitable features.

 

Machine learning brief introduction

Machine learning aims at making computers smarter.

 

OCR

Optical character recognition (optical character recognition)

It refers to taking photos of printed pages, and the computer will automatically recognize the text.

principle

Using KNN

(1) Browse a large number of digital images and extract the features of these figures.

(2) When the new image is encountered, extract the characteristics of the image and find out who its nearest neighbors are.

Generally speaking, OCR algorithm extracts features such as line segments, points and curves.

 

OCRThe first step is to look at a large number of digital images and extract features, which is called training (training).

 

Simple Bias classifier (Naive Bayes classifier)

First, we need to use some data to train the classifier.

The simple Bias classifier can calculate some probability of events, and its application domain is similar to KNN.

 

Summary

•    KNNFor classification and regression, we need to consider the nearest neighbors.
•   Classification is grouping.
•   Regression is the prediction result, such as the number.
•   Feature extraction means converting items (such as fruit or user) into a series of comparable figures.
•   It is a matter of success or failure of KNN algorithm to select suitable features.