首页 > 其他 > 详细

词频统计

时间:2019-07-19 20:28:35      阅读:126      评论:0      收藏:0      [点我收藏+]

词频统计

一.中文词频统计

代码:

import jieba

f = open(r'C:\Users\Administrator\Desktop\Markdown文档\110.txt', 'r')
data = f.read()
print(data)

data_jieba = jieba.lcut(data)
print(data_jieba)

count_dict = {}
for word in data_jieba:
    if len(word) == 1:
        continue
        
    if word in {"将军", "却说", "荆州", "二人", "不可", "不能", "如此", "商议"}:
        continue

    if word == '孔明曰':
        word = '孔明'
    elif word == '玄德曰':
        word = '玄德'

    if '曰' in word:
        word = word.replace('曰', '')

    if word in count_dict:
        count_dict[word] += 1
    else:
        count_dict[word] = 1

def func(i):
    return i[1]

data_list = list(count_dict.items())
data_list.sort(key=func)
data_list.reverse()

print(data_list)

效果:

技术分享图片

二.英文词频统计

代码:

f = open(r'C:\Users\Administrator\Desktop\Markdown文档\120.txt','r',encoding='utf8')
data = f.read()
print(data)

data_split = data.split(' ')
count_dict = {}
for word in data_split:
    if word not in count_dict:
        count_dict[word] = 1
    else:
        count_dict[word] += 1

# print(count_dict)
def func(i):
    return i[1]
#
lt = list(count_dict.items())
lt.sort(key=func)

lt.reverse()
lt.reverse()

for i in lt[0:10]:
    print(f'{i[0]:^15}{i[1]:^2}')

效果:
技术分享图片
---------------------
作者:罗文祥
来源:祥SHAO
原文:https://www.cnblogs.com/LWX-YEER/p/11215490.html
版权声明:本文为博主原创文章,转载请附上博文链接!

词频统计

原文:https://www.cnblogs.com/LWX-YEER/p/11215490.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!