加载数据
import numpy as np
import pandas as pd
df = pd.read_csv(‘./data/usa_election.txt‘,low_memory=False)
查看数据的基本信息
df.info()
>>>
<class ‘pandas.core.frame.DataFrame‘>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 16 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cmte_id 536041 non-null object
1 cand_id 536041 non-null object
2 cand_nm 536041 non-null object
3 contbr_nm 536041 non-null object
4 contbr_city 536026 non-null object
5 contbr_st 536040 non-null object
6 contbr_zip 535973 non-null object
7 contbr_employer 525088 non-null object
8 contbr_occupation 530520 non-null object
9 contb_receipt_amt 536041 non-null float64
10 contb_receipt_dt 536041 non-null object
11 receipt_desc 8479 non-null object
12 memo_cd 49718 non-null object
13 memo_text 52740 non-null object
14 form_tp 536041 non-null object
15 file_num 536041 non-null int64
dtypes: float64(1), int64(1), object(14)
memory usage: 65.4+ MB
指定数据截取，将如下字段的数据进行提取，其他数据舍弃
df = df[[‘cand_nm‘,‘contbr_nm‘,‘contbr_st‘,‘contbr_employer‘,‘contbr_occupation‘,‘contb_receipt_amt‘,‘contb_receipt_dt‘]]
对新数据进行总览,查看是否存在缺失数据
df.info()
>>>
<class ‘pandas.core.frame.DataFrame‘>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cand_nm 536041 non-null object
1 contbr_nm 536041 non-null object
2 contbr_st 536040 non-null object
3 contbr_employer 525088 non-null object
4 contbr_occupation 530520 non-null object
5 contb_receipt_amt 536041 non-null float64
6 contb_receipt_dt 536041 non-null object
dtypes: float64(1), object(6)
memory usage: 28.6+ MB
用统计学指标快速描述数值型属性的概要
df.describe()
空值处理,可能因为忘记填写或者保密等等原因，相关字段出现了空值，将其填充为NOT PROVIDE
df.fillna(value=‘NOT PROVIDE‘,inplace=True,axis=0)
df.info()
>>>
<class ‘pandas.core.frame.DataFrame‘>
RangeIndex: 536041 entries, 0 to 536040
Data columns (total 7 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 cand_nm 536041 non-null object
1 contbr_nm 536041 non-null object
2 contbr_st 536041 non-null object
3 contbr_employer 536041 non-null object
4 contbr_occupation 536041 non-null object
5 contb_receipt_amt 536041 non-null float64
6 contb_receipt_dt 536041 non-null object
dtypes: float64(1), object(6)
memory usage: 28.6+ MB
异常值处理。将捐款金额<=0的数据删除
df.loc[df[‘contb_receipt_amt‘] <= 0] #异常值对应的行数据
indexs = df.loc[df[‘contb_receipt_amt‘] <= 0].index#提取异常数据的行号
>>>
Int64Index([ 43, 265, 650, 666, 720, 810, 835, 836,
837, 838,
...
535929, 535937, 535942, 535945, 535971, 535975, 536003, 536005,
536006, 536007],
dtype=‘int64‘, length=5727)
df.drop(labels=indexs,axis=0,inplace=True)#删除这些异常数据
新建一列为各个候选人所在党派party
parties = {
‘Bachmann, Michelle‘: ‘Republican‘,
‘Romney, Mitt‘: ‘Republican‘,
‘Obama, Barack‘: ‘Democrat‘,
"Roemer, Charles E. ‘Buddy‘ III": ‘Reform‘,
‘Pawlenty, Timothy‘: ‘Republican‘,
‘Johnson, Gary Earl‘: ‘Libertarian‘,
‘Paul, Ron‘: ‘Republican‘,
‘Santorum, Rick‘: ‘Republican‘,
‘Cain, Herman‘: ‘Republican‘,
‘Gingrich, Newt‘: ‘Republican‘,
‘McCotter, Thaddeus G‘: ‘Republican‘,
‘Huntsman, Jon‘: ‘Republican‘,
‘Perry, Rick‘: ‘Republican‘
}
df[‘party‘] = df[‘cand_nm‘].map(parties)
df.head()
查看party这一列中有哪些不同的元素
df[‘party‘].unique()
>>>
array([‘Republican‘, ‘Democrat‘, ‘Reform‘, ‘Libertarian‘], dtype=object)
统计party列中各个元素出现次数
df[‘party‘].value_counts()
>>>
Democrat 289999
Republican 234300
Reform 5313
Libertarian 702
Name: party, dtype: int64
查看各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by=‘party‘)[‘contb_receipt_amt‘].sum()
>>>
party
Democrat 8.259441e+07
Libertarian 4.132769e+05
Reform 3.429658e+05
Republican 1.251181e+08
Name: contb_receipt_amt, dtype: float64
查看具体每天各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by=[‘contb_receipt_dt‘,‘party‘])[‘contb_receipt_amt‘].sum()
>>>
contb_receipt_dt party
01-APR-11 Reform 50.00
Republican 12635.00
01-AUG-11 Democrat 182198.00
Libertarian 1000.00
Reform 1847.00
...
31-MAY-11 Republican 313839.80
31-OCT-11 Democrat 216971.87
Libertarian 4250.00
Reform 3205.00
Republican 751542.36
Name: contb_receipt_amt, Length: 1183, dtype: float64
将表中日期格式转换为‘yyyy-mm-dd‘
def transform_date(date):
day,month,year = date.split(‘-‘)
month = months[month]
return ‘20‘+year+‘-‘+str(month)+‘-‘+day
df[‘contb_receipt_dt‘] = df[‘contb_receipt_dt‘].map(transform_date)
df.head()
查看老兵(捐献者职业)DISABLED VETERAN主要支持谁
df.columns
>>>
Index([‘cand_nm‘, ‘contbr_nm‘, ‘contbr_st‘, ‘contbr_employer‘,
‘contbr_occupation‘, ‘contb_receipt_amt‘, ‘contb_receipt_dt‘, ‘party‘],
dtype=‘object‘)
#1.先把老兵这个职业对应的行数据取出
old_df = df.loc[df[‘contbr_occupation‘] == ‘DISABLED VETERAN‘]
#2.对old_df根据候选人分组，然后对钱数聚合
old_df.groupby(by=‘cand_nm‘)[‘contb_receipt_amt‘].sum()
>>>
cand_nm
Cain, Herman 300.00
Obama, Barack 4205.00
Paul, Ron 2425.49
Santorum, Rick 250.00
Name: contb_receipt_amt, dtype: float64
原文：https://www.cnblogs.com/remixnameless/p/13215795.html