首页 > 其他 > 详细

Spark1.4源码走读笔记之模式匹配

时间:2016-09-19 13:19:24      阅读:294      评论:0      收藏:0      [点我收藏+]

RDD里的模式匹配:

def hasNext: Boolean = (thisIter.hasNext, otherIter.hasNext) match {

          case (true, true) => true

          case (false, false) => false

          case _ => throw new SparkException("Can only zip RDDs with " +

            "same number of elements in each partition")

        }


jobResult = jobResult match {

          case Some(value) => Some(f(value, taskResult.get))

          case None => taskResult

        }


    take(1) match {

      case Array(t) => t

      case _ => throw new UnsupportedOperationException("empty collection")

    }

下面的比较好理解:

val len = rdd.dependencies.length

      len match {

        case 0 => Seq.empty

        case 1 =>

          val d = rdd.dependencies.head

          debugString(d.rdd, prefix, d.isInstanceOf[ShuffleDependency[_, _, _]], true)

        case _ => //所有的都到碗里来

          val frontDeps = rdd.dependencies.take(len - 1)

          val frontDepStrings = frontDeps.flatMap(

            d => debugString(d.rdd, prefix, d.isInstanceOf[ShuffleDependency[_, _, _]]))


          val lastDep = rdd.dependencies.last

          val lastDepStrings =

            debugString(lastDep.rdd, prefix, lastDep.isInstanceOf[ShuffleDependency[_, _, _]], true)


          (frontDepStrings ++ lastDepStrings)

      }


Spark1.4源码走读笔记之模式匹配

原文:http://36006798.blog.51cto.com/988282/1853979

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!