首页 > 编程语言 > 详细

python使用jieba实现简单的词频统计

时间:2020-09-23 23:12:09      阅读:113      评论:0      收藏:0      [点我收藏+]
import jieba
def getText():
    txt=open("hamlet.txt","r").read()
    txt=txt.lower()
    for ch in |"#$%&()*+,-./:;<>+?@[\\]^_{|}~:
        txt=txt.replace(ch," ")
    return txt
harmTxt=getText()
words=harmTxt.split()
counts={}
for word in words:
    counts[word]=counts.get(word,0)+1

items=list(counts.items())

#按照第二个元素有大到小排序
items.sort(key=lambda  x:x[1],reverse=True)

for i in range(10):
    word, count=items[i]
    print(word,end=":")
    print(count)


运行结果

the:1138
and:965
to:754
of:668
you:549
a:542
i:540
my:514
hamlet:456
in:436

 

import jieba
txt=open("threekingdoms.txt","r",encoding="utf-8").read()
#总结一些不是人名的词
excludes={"将军","却说","二人","荆州","二人","不可","不能","如此","商议","不能","如此","左右","引兵","如何","主公"}
words=jieba.lcut(txt)
counts={}
for word in words:
    if len(word)==1:
        continue
    elif word=="诸葛亮" or word=="孔明曰":
        rword="孔明"
    elif word=="关公" or word=="云长":
        rword="关羽"
    elif word=="玄德" or word=="玄德曰":
        rword="刘备"
    elif word=="孟德" or word=="丞相":
        rword="曹操"
    else:
        rword=word
    counts[rword]=counts.get(rword,0)+1
for word in excludes:
    del counts[word]
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(10):
    word,count=items[i]
    print(word,end=":")
    print(count)


运行结果:

曹操:1451
孔明:1383
刘备:1252
关羽:784
张飞:358
军士:317
吕布:300
军马:293
赵云:278
次日:271

 

python使用jieba实现简单的词频统计

原文:https://www.cnblogs.com/837634902why/p/13721355.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!