您当前的位置: 首页 >  大数据

段智华

暂无认证

  • 0浏览

    0关注

    1232博文

    0收益

  • 0浏览

    0点赞

    0打赏

    0留言

私信
关注
热门博文

大数据 IMF传奇 sparkpi 运行5万次

段智华 发布时间:2016-02-07 20:11:35 ,浏览量:0

1.这个可以运行的  ./spark-submit   --class org.apache.spark.examples.SparkPi  --master spark://master:7077    ../lib/spark-examples-1.6.0-hadoop2.6.0.jar   1000 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 993.0 in stage 0.0 (TID 993) in 29 ms on worker8 (992/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 994.0 in stage 0.0 (TID 994) in 140 ms on worker5 (993/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 995.0 in stage 0.0 (TID 995) in 140 ms on worker7 (994/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 996.0 in stage 0.0 (TID 996) in 135 ms on worker4 (995/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 998.0 in stage 0.0 (TID 998) in 124 ms on worker6 (996/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 999.0 in stage 0.0 (TID 999) in 122 ms on worker8 (997/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 997.0 in stage 0.0 (TID 997) in 131 ms on worker3 (998/1000) 16/02/07 19:28:23 INFO scheduler.TaskSetManager: Finished task 982.0 in stage 0.0 (TID 982) in 208 ms on worker2 (999/1000) 16/02/07 19:28:24 INFO scheduler.TaskSetManager: Finished task 935.0 in stage 0.0 (TID 935) in 1422 ms on worker1 (1000/1000) 16/02/07 19:28:24 INFO scheduler.DAGScheduler: ResultStage 0 (reduce at SparkPi.scala:36) finished in 60.116 s 16/02/07 19:28:24 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool  16/02/07 19:28:24 INFO scheduler.DAGScheduler: Job 0 finished: reduce at SparkPi.scala:36, took 62.080983 s Pi is roughly 3.14156192 2.这个运行报错 root@master:/usr/local/spark-1.6.0-bin-hadoop2.6/bin#  ./spark-submit   --class org.apache.spark.examples.SparkPi  --master spark://master:7077   --deploy-mode cluster   --supervise      /usr/local/spark-1.6.0-bin-hadoop2.6/lib/spark-examples-1.6.0-hadoop2.6.0.jar   1000 3.运行100百万次的运行,跑不起来 root@master:/usr/local/spark-1.6.0-bin-hadoop2.6/bin#  ./spark-submit   --class org.apache.spark.examples.SparkPi  --master spark://master:7077    ../lib/spark-examples-1.6.0-hadoop2.6.0.jar   1000000 4.运行5万次 ./spark-submit   --class org.apache.spark.examples.SparkPi  --master spark://master:7077    ../lib/spark-examples-1.6.0-hadoop2.6.0.jar  50000 L, 2156 bytes) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49990.0 in stage 0.0 (TID 49990) in 41 ms on worker5 (49992/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49991.0 in stage 0.0 (TID 49991) in 141 ms on worker3 (49993/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49992.0 in stage 0.0 (TID 49992) in 140 ms on worker2 (49994/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49995.0 in stage 0.0 (TID 49995) in 130 ms on worker7 (49995/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49996.0 in stage 0.0 (TID 49996) in 129 ms on worker4 (49996/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49994.0 in stage 0.0 (TID 49994) in 138 ms on worker1 (49997/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49998.0 in stage 0.0 (TID 49998) in 120 ms on worker8 (49998/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49997.0 in stage 0.0 (TID 49997) in 124 ms on worker6 (49999/50000) 16/02/07 19:51:54 INFO scheduler.TaskSetManager: Finished task 49999.0 in stage 0.0 (TID 49999) in 115 ms on worker5 (50000/50000) 16/02/07 19:51:54 INFO scheduler.DAGScheduler: ResultStage 0 (reduce at SparkPi.scala:36) finished in 330.839 s 16/02/07 19:51:54 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool  16/02/07 19:51:54 INFO scheduler.DAGScheduler: Job 0 finished: reduce at SparkPi.scala:36, took 384.529375 s

Pi is roughly 3.141592859822136

关注
打赏
1659361485
查看更多评论
立即登录/注册

微信扫码登录

0.0405s