xiaoxiong's Blog

Happy coding

spark 1.2.1 的集群配置

xiaoxiong posted @ 2015年4月07日 10:17 in spark , 718 阅读

搭建hadoop已经是上学期的事情了,当时候还好留了这个,不然完蛋,打不出来了,话不多说,进入正题

1  下载并在linux 集群 上安装scala,这个是必须的。 然后跟上面的配置java环境一样,去配置scala就ok了,我是在 ~/.bashrc中配置的

2  配置完scala之后,当然就是spark了,下载spark然后跟上面配置scala的方法一样,配置spark的环境变量

3  当在环境变量中配好 spark 和scala之后,就开始进入spark 中配置 scala 了,在 spark的conf, spark-env.sh中, 加入

      SCALA_HOME=/root/hjava/spark/scala-2.11.0

   在slaves中加入

xiaoxiong2

xiaoxiong3

xiaoxiong4

 然后搞定, 去启动spark hdfs  yarn 然后就开始学习了

 

 

Avatar_small
Junior Dakil Result 说:
2022年9月07日 01:35

Bangladesh Education Board DPE has conducted the class 8th grade of Junior School Certificate Exam and Junior Dakhil Certificate Exam on 1st to 15th November 2022 at all centers in division wise under Ministry of Primary and Mass Education (MOPME), and the class 8th grade terminal examination tests are successfully conducted for all eligible JSC/JDC students for the academic year of 2022. Junior Dakil Result jessore Board The Bangladesh government Minister of Secondary Education is going to announce the JSC Result 2022 in student wise for division students in education board wise, and the result documents will be submitted to the Prime Minister of the country after then the result with mark sheet will be announced to public to check the individual result.

Avatar_small
Emma 说:
2023年1月18日 19:07

In order to configure Spark for a cluster, we need to first setup a Spark master node. This can be done by following the instructions on the official Apache Spark website. After the CBD Oil master node is setup, we need to configure the slaves. Each slave should have its own unique ID, and all slaves should be registered with the master. After the slaves are registered, we can start the Spark cluster by running the start-all.sh script on the master node.


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter