首页 > 其他 > 详细

KNN

时间:2020-11-10 10:16:34      阅读:26      评论:0      收藏:0      [点我收藏+]

可以参见 如下连接了解原理:

https://zhuanlan.zhihu.com/p/61341071

一.KNN算法概述

KNN可以说是最简单的分类算法之一,同时,它也是最常用的分类算法之一,注意KNN算法是有监督学习中的分类算法,它看起来和另一个机器学习算法Kmeans有点像(Kmeans是无监督学习算法),但却是有本质区别的。那么什么是KNN算法呢,接下来我们就来介绍介绍吧。

二.KNN算法介绍

KNN的全称是K Nearest Neighbors,意思是K个最近的邻居,从这个名字我们就能看出一些KNN算法的蛛丝马迹了。K个最近邻居,毫无疑问,K的取值肯定是至关重要的。那么最近的邻居又是怎么回事呢?其实啊,KNN的原理就是当预测一个新的值x的时候,根据它距离最近的K个点是什么类别来判断x属于哪个类别。听起来有点绕,还是看看图吧。

技术分享图片

 

 技术分享图片

 

 

 

实例:

import csv #用于处理csv文件 
import random #用于随机数 
import math 
import operator # 
from sklearn import neighbors 

#加载数据集 
def loadDataset(filename,split,trainingSet=[],testSet = []): # 加载数据集 split以某个值为界限分类train和test 

with open(filename, ‘r‘) as csvfile: 
lines = csv.reader(csvfile) #读取所有的行 

dataset = list(lines) #转化成列表 
for x in range(len(dataset)-1): 
for y in range(4): 
dataset[x][y] = float(dataset[x][y]) 
if random.random() < split: # 将所有数据加载到train和test中 
trainingSet.append(dataset[x]) 
else: 
testSet.append(dataset[x])

#计算距离 
def euclideanDistance(instance1,instance2,length): # 计算距离 
distance = 0 # length表示维度 数据共有几维
for x in range(length): 
distance += pow((instance1[x] - instance2[x]),2)
return math.sqrt(distance) 

#返回K个最近邻 
def getNeighbors(trainingSet,testInstance,k): 
distances = [] 
length = len(testInstance) -1 
#计算每一个测试实例到训练集实例的距离 
for x in range(len(trainingSet)): 
dist = euclideanDistance(testInstance, trainingSet[x], length) 
distances.append((trainingSet[x],dist)) 
#对所有的距离进行排序 
distances.sort(key=operator.itemgetter(1)) 
neighbors = [] 
#返回k个最近邻 
for x in range(k): 
neighbors.append(distances[x][0]) 
return neighbors 

#对k个近邻进行合并,返回value最大的key 
def getResponse(neighbors): # 根据少数服从多数,决定归类到哪一类
classVotes = {} 
for x in range(len(neighbors)): 
response = neighbors[x][-1] # 统计每一个分类的多少 
if response in classVotes: 
classVotes[response] += 1 
else: 
classVotes[response] = 1 
print(classVotes.items()) 
sortedVotes = sorted(classVotes.items(), key=operator.itemgetter(1), reverse=True) #reverse按降序的方式排列 
return sortedVotes[0][0] 

#计算准确率 
def getAccuracy(testSet,predictions): 
correct = 0 
for x in range(len(testSet)): 
if testSet[x][-1] == predictions[x]: 
correct+=1 
return (correct/float(len(testSet))) * 100.0


def main(): 
trainingSet = [] #训练数据集 
testSet = [] #测试数据集 
split = 0.68 #分割的比例 
loadDataset("D:\SAB\Desktop\iris.txt", split, trainingSet, testSet) 
print ("Train set :" + repr(len(trainingSet)))
print ( "Test set :" + repr(len(testSet))) 

predictions = [] 
k = 3 
for x in range(len(testSet)): 
neighbors = getNeighbors(trainingSet, testSet[x], k) 
result = getResponse(neighbors) 
predictions.append(result) 
print (">predicted = " + repr(result) + ",actual = " + repr(testSet[x][-1]) )
accuracy = getAccuracy(testSet, predictions) 
print ("Accuracy:" + repr(accuracy) + "%" )

if __name__ =="__main__": 
main()

所用数据如下:

5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa
4.6,3.1,1.5,0.2,Iris-setosa
5.0,3.6,1.4,0.2,Iris-setosa
5.4,3.9,1.7,0.4,Iris-setosa
4.6,3.4,1.4,0.3,Iris-setosa
5.0,3.4,1.5,0.2,Iris-setosa
4.4,2.9,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.4,3.7,1.5,0.2,Iris-setosa
4.8,3.4,1.6,0.2,Iris-setosa
4.8,3.0,1.4,0.1,Iris-setosa
4.3,3.0,1.1,0.1,Iris-setosa
5.8,4.0,1.2,0.2,Iris-setosa
5.7,4.4,1.5,0.4,Iris-setosa
5.4,3.9,1.3,0.4,Iris-setosa
5.1,3.5,1.4,0.3,Iris-setosa
5.7,3.8,1.7,0.3,Iris-setosa
5.1,3.8,1.5,0.3,Iris-setosa
5.4,3.4,1.7,0.2,Iris-setosa
5.1,3.7,1.5,0.4,Iris-setosa
4.6,3.6,1.0,0.2,Iris-setosa
5.1,3.3,1.7,0.5,Iris-setosa
4.8,3.4,1.9,0.2,Iris-setosa
5.0,3.0,1.6,0.2,Iris-setosa
5.0,3.4,1.6,0.4,Iris-setosa
5.2,3.5,1.5,0.2,Iris-setosa
5.2,3.4,1.4,0.2,Iris-setosa
4.7,3.2,1.6,0.2,Iris-setosa
4.8,3.1,1.6,0.2,Iris-setosa
5.4,3.4,1.5,0.4,Iris-setosa
5.2,4.1,1.5,0.1,Iris-setosa
5.5,4.2,1.4,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
5.0,3.2,1.2,0.2,Iris-setosa
5.5,3.5,1.3,0.2,Iris-setosa
4.9,3.1,1.5,0.1,Iris-setosa
4.4,3.0,1.3,0.2,Iris-setosa
5.1,3.4,1.5,0.2,Iris-setosa
5.0,3.5,1.3,0.3,Iris-setosa
4.5,2.3,1.3,0.3,Iris-setosa
4.4,3.2,1.3,0.2,Iris-setosa
5.0,3.5,1.6,0.6,Iris-setosa
5.1,3.8,1.9,0.4,Iris-setosa
4.8,3.0,1.4,0.3,Iris-setosa
5.1,3.8,1.6,0.2,Iris-setosa
4.6,3.2,1.4,0.2,Iris-setosa
5.3,3.7,1.5,0.2,Iris-setosa
5.0,3.3,1.4,0.2,Iris-setosa
7.0,3.2,4.7,1.4,Iris-versicolor
6.4,3.2,4.5,1.5,Iris-versicolor
6.9,3.1,4.9,1.5,Iris-versicolor
5.5,2.3,4.0,1.3,Iris-versicolor
6.5,2.8,4.6,1.5,Iris-versicolor
5.7,2.8,4.5,1.3,Iris-versicolor
6.3,3.3,4.7,1.6,Iris-versicolor
4.9,2.4,3.3,1.0,Iris-versicolor
6.6,2.9,4.6,1.3,Iris-versicolor
5.2,2.7,3.9,1.4,Iris-versicolor
5.0,2.0,3.5,1.0,Iris-versicolor
5.9,3.0,4.2,1.5,Iris-versicolor
6.0,2.2,4.0,1.0,Iris-versicolor
6.1,2.9,4.7,1.4,Iris-versicolor
5.6,2.9,3.6,1.3,Iris-versicolor
6.7,3.1,4.4,1.4,Iris-versicolor
5.6,3.0,4.5,1.5,Iris-versicolor
5.8,2.7,4.1,1.0,Iris-versicolor
6.2,2.2,4.5,1.5,Iris-versicolor
5.6,2.5,3.9,1.1,Iris-versicolor
5.9,3.2,4.8,1.8,Iris-versicolor
6.1,2.8,4.0,1.3,Iris-versicolor
6.3,2.5,4.9,1.5,Iris-versicolor
6.1,2.8,4.7,1.2,Iris-versicolor
6.4,2.9,4.3,1.3,Iris-versicolor
6.6,3.0,4.4,1.4,Iris-versicolor
6.8,2.8,4.8,1.4,Iris-versicolor
6.7,3.0,5.0,1.7,Iris-versicolor
6.0,2.9,4.5,1.5,Iris-versicolor
5.7,2.6,3.5,1.0,Iris-versicolor
5.5,2.4,3.8,1.1,Iris-versicolor
5.5,2.4,3.7,1.0,Iris-versicolor
5.8,2.7,3.9,1.2,Iris-versicolor
6.0,2.7,5.1,1.6,Iris-versicolor
5.4,3.0,4.5,1.5,Iris-versicolor
6.0,3.4,4.5,1.6,Iris-versicolor
6.7,3.1,4.7,1.5,Iris-versicolor
6.3,2.3,4.4,1.3,Iris-versicolor
5.6,3.0,4.1,1.3,Iris-versicolor
5.5,2.5,4.0,1.3,Iris-versicolor
5.5,2.6,4.4,1.2,Iris-versicolor
6.1,3.0,4.6,1.4,Iris-versicolor
5.8,2.6,4.0,1.2,Iris-versicolor
5.0,2.3,3.3,1.0,Iris-versicolor
5.6,2.7,4.2,1.3,Iris-versicolor
5.7,3.0,4.2,1.2,Iris-versicolor
5.7,2.9,4.2,1.3,Iris-versicolor
6.2,2.9,4.3,1.3,Iris-versicolor
5.1,2.5,3.0,1.1,Iris-versicolor
5.7,2.8,4.1,1.3,Iris-versicolor
6.3,3.3,6.0,2.5,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
7.1,3.0,5.9,2.1,Iris-virginica
6.3,2.9,5.6,1.8,Iris-virginica
6.5,3.0,5.8,2.2,Iris-virginica
7.6,3.0,6.6,2.1,Iris-virginica
4.9,2.5,4.5,1.7,Iris-virginica
7.3,2.9,6.3,1.8,Iris-virginica
6.7,2.5,5.8,1.8,Iris-virginica
7.2,3.6,6.1,2.5,Iris-virginica
6.5,3.2,5.1,2.0,Iris-virginica
6.4,2.7,5.3,1.9,Iris-virginica
6.8,3.0,5.5,2.1,Iris-virginica
5.7,2.5,5.0,2.0,Iris-virginica
5.8,2.8,5.1,2.4,Iris-virginica
6.4,3.2,5.3,2.3,Iris-virginica
6.5,3.0,5.5,1.8,Iris-virginica
7.7,3.8,6.7,2.2,Iris-virginica
7.7,2.6,6.9,2.3,Iris-virginica
6.0,2.2,5.0,1.5,Iris-virginica
6.9,3.2,5.7,2.3,Iris-virginica
5.6,2.8,4.9,2.0,Iris-virginica
7.7,2.8,6.7,2.0,Iris-virginica
6.3,2.7,4.9,1.8,Iris-virginica
6.7,3.3,5.7,2.1,Iris-virginica
7.2,3.2,6.0,1.8,Iris-virginica
6.2,2.8,4.8,1.8,Iris-virginica
6.1,3.0,4.9,1.8,Iris-virginica
6.4,2.8,5.6,2.1,Iris-virginica
7.2,3.0,5.8,1.6,Iris-virginica
7.4,2.8,6.1,1.9,Iris-virginica
7.9,3.8,6.4,2.0,Iris-virginica
6.4,2.8,5.6,2.2,Iris-virginica
6.3,2.8,5.1,1.5,Iris-virginica
6.1,2.6,5.6,1.4,Iris-virginica
7.7,3.0,6.1,2.3,Iris-virginica
6.3,3.4,5.6,2.4,Iris-virginica
6.4,3.1,5.5,1.8,Iris-virginica
6.0,3.0,4.8,1.8,Iris-virginica
6.9,3.1,5.4,2.1,Iris-virginica
6.7,3.1,5.6,2.4,Iris-virginica
6.9,3.1,5.1,2.3,Iris-virginica
5.8,2.7,5.1,1.9,Iris-virginica
6.8,3.2,5.9,2.3,Iris-virginica
6.7,3.3,5.7,2.5,Iris-virginica
6.7,3.0,5.2,2.3,Iris-virginica
6.3,2.5,5.0,1.9,Iris-virginica
6.5,3.0,5.2,2.0,Iris-virginica
6.2,3.4,5.4,2.3,Iris-virginica
5.9,3.0,5.1,1.8,Iris-virginica

 

运行结果:

 

Train set :98
Test set :51
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-setosa‘, 3)])
>predicted = ‘Iris-setosa‘,actual = ‘Iris-setosa‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 2), (‘Iris-virginica‘, 1)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 2), (‘Iris-virginica‘, 1)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-virginica‘, 2), (‘Iris-versicolor‘, 1)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 2), (‘Iris-virginica‘, 1)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-versicolor‘, 3)])
>predicted = ‘Iris-versicolor‘,actual = ‘Iris-versicolor‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 2), (‘Iris-versicolor‘, 1)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-versicolor‘, 1), (‘Iris-virginica‘, 2)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
dict_items([(‘Iris-virginica‘, 3)])
>predicted = ‘Iris-virginica‘,actual = ‘Iris-virginica‘
Accuracy:96.07843137254902%

KNN

原文:https://www.cnblogs.com/yusuf/p/13952143.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!