spark-2.4.0-bin-hadoop2.6.tgz下载https://archive.apache.org/dist/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.6.tgz将spark-2.4.0-bin-hadoop2.6.tgz移到到主机/opt目录并解压到/usr/localmv /home/master/Desktop/spark-2.4.0-bin-hadoop2.6.tgz /opt/ tar -zxf /opt/spark-2.4.0-bin-hadoop2.6.tgz -C /usr/local/#进入目录下 cd /usr/local/spark-2.4.0-bin-hadoop2.6/conf #复制slaves.template cp slaves.template slaves #修改slaves vi slaves先删除其中的localhost然后添加slave1slave2slave3#复制spark-defaults.conf.template cp spark-defaults.conf.template spark-defaults.conf #修改spark-defaults.conf vi spark-defaults.conf确保有以下配置无需理会原有内容直接添加下面的内容即可spark.master spark://master:7077spark.eventLog.enabled truespark.eventLog.dir hdfs://master:8020/spark-logs/spark.history.fs.logDirectory hdfs://master:8020/spark-logs/spark.driver.extraClassPath /usr/local/spark-2.4.0-bin-hadoop2.6/jars/mysql-connector-java-5.1.32-bin.jarspark.executor.extraClassPath /usr/local/spark-2.4.0-bin-hadoop2.6/jars/mysql-connector-java-5.1.32-bin.jar#复制并修改spark-env.sh cp spark-env.sh.template spark-env.sh vi spark-env.sh添加JAVA_HOME/usr/java/jdk1.8.0_151HADOOP_CONF_DIR/usr/local/hadoop-2.6.5/etc/hadoopSPARK_MASTER_IPmasterSPARK_MASTER_PORT7077SPARK_WORKER_MEMORY512mSPARK_WORKER_CORES1SPARK_EXECUTOR_MEMORY512mSPARK_EXECUTOR_CORES1SPARK_WORKER_INSTANCES1启动Hadoop集群后在HDFS中新建目录# 删除原有目录避免权限残留 hdfs dfs -rm -r /spark-logs hdfs dfs -mkdir -p /spark-logs # 给目录赋 777 权限测试环境确保 Spark 可读写 hdfs dfs -chmod 777 /spark-logs hdfs dfs -chown root:root /spark-logs hdfs dfs -ls / # 输出中会看到 drwxrwxrwx ... root root ... /spark-logs将Spark安装包分发到其他节点scp -r /usr/local/spark-2.4.0-bin-hadoop2.6/ slave1:/usr/local/ scp -r /usr/local/spark-2.4.0-bin-hadoop2.6/ slave2:/usr/local/ scp -r /usr/local/spark-2.4.0-bin-hadoop2.6/ slave3:/usr/local/在所有节点配置Spark环境变量vi /etc/profile添加export SPARK_HOME/usr/local/spark-2.4.0-bin-hadoop2.6export PATH$PATH:$SPARK_HOME/bin执行命令使配置生效source /etc/profile启动sparkcd /usr/local/spark-2.4.0-bin-hadoop2.6/sbin ./start-all.sh ./start-history-server.sh查看网页http://master:8080/http://master:18080/