首页 > 编程语言 > 详细

[Spark][Python][DataFrame][RDD]从DataFrame得到RDD的例子

时间:2017-10-07 16:52:51      阅读:232      评论:0      收藏:0      [点我收藏+]

[Spark][Python][DataFrame][RDD]从DataFrame得到RDD的例子

$ hdfs dfs -cat people.json

{"name":"Alice","pcode":"94304"}
{"name":"Brayden","age":30,"pcode":"94304"}
{"name":"Carla","age":19,"pcoe":"10036"}
{"name":"Diana","age":46}
{"name":"Etienne","pcode":"94104"}

 

$pyspark

sqlContext = HiveContext(sc)

peopleDF = sqlContext.read.json("people.json")

peopleRDD = peopleDF.rdd
peopleRDD.take(5)

[Row(age=None, name=u‘Alice‘, pcode=u‘94304‘, pcoe=None),
Row(age=30, name=u‘Brayden‘, pcode=u‘94304‘, pcoe=None),
Row(age=19, name=u‘Carla‘, pcode=None, pcoe=u‘10036‘),
Row(age=46, name=u‘Diana‘, pcode=None, pcoe=None),
Row(age=None, name=u‘Etienne‘, pcode=u‘94104‘, pcoe=None)]

 

[Spark][Python][DataFrame][RDD]从DataFrame得到RDD的例子

原文:http://www.cnblogs.com/gaojian/p/7635011.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!