Hadoop/spark安装实战（系列篇3） Hadoop 伪分布模式安装
Hadoop 的安装与配置 伪分布模式安装
1/解压缩hadoop tar -zxvf hadoop-1.2.1-bin.tar.gz 2/ 将文件移到/usr/local mv hadoop-1.2.1 /usr/local
3/设置环境变量HADOOP_HOME，修改文件“/etc/profile” [root@localhost local]# vi /etc/profile export HADOOP_HOME=/usr/local/hadoop-1.2.1 export PATH=.:$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin 生效 [root@localhost local]# source /etc/profile 检验 [root@localhost local]# echo $PATH .:.:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/l
ocal/jdk1.7.0_79/bin:/root/bin:/usr/local/jdk1.7.0_79/bin:/usr/local/ha
doop-1.2.1/bin [root@localhost local]#
4 修改hadoop 环境变量脚本文件hadoop-env.sh 增加 export JAVA_HOME=/usr/local/jdk1.7.0_79
5 修改hadoop 核心配置文件core-site.xml
hadoop.tmp.dir /home/hadoop/tmp hadoop.tmp.dir fs.default.name hdfs://spark0:9000 HDFS fs.default.name
6 hdfs 配置文件hdfs-site.xml，存储副本数1份
dfs.replication 1 1
7 MapReduce 配置文件mapred-site.xml
mapred.job.tracker spark0:9001 JobTracker
8格式化文件系统 在第一次使用之前需要进行格式化，只在第一次启动的时候格式化，不要每次启
动都格式化 [root@localhost bin]# hadoop namenode -format
15/09/12 06:31:11 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = localhost.localdomain/127.0.0.1 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 1.2.1 STARTUP_MSG: build =
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013 STARTUP_MSG: java = 1.7.0_79 ************************************************************/ 15/09/12 06:31:13 INFO util.GSet: Computing capacity for map BlocksMap 15/09/12 06:31:13 INFO util.GSet: VM type = 32-bit 15/09/12 06:31:13 INFO util.GSet: 2.0% max memory = 1013645312 15/09/12 06:31:13 INFO util.GSet: capacity = 2^22 = 4194304
entries
9 启动hadoop 程序
[root@localhost bin]# start-all.sh
[root@localhost bin]# start-all.sh Warning: $HADOOP_HOME is deprecated.
starting namenode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-namenode-localhost.localdomain.out localhost: starting datanode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-datanode-localhost.localdomain.out localhost: starting secondarynamenode, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-secondarynamenode-
localhost.localdomain.out starting jobtracker, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-jobtracker-localhost.localdomain.out localhost: starting tasktracker, logging to /usr/local/hadoop-
1.2.1/libexec/../logs/hadoop-root-tasktracker-localhost.localdomain.out
检验
[root@localhost bin]# jps
[root@localhost bin]# jps 27375 Jps 26949 DataNode 27046 SecondaryNameNode 26851 NameNode 27240 TaskTracker 27124 JobTracker [root@localhost bin]#
10 检验 在本地物理机浏览器输入 http://192.168.2.100:50070/dfshealth.jsp
NameNode 'spark0:9000'
Started: Sat Sep 12 06:34:38 PDT 2015 Version: 1.2.1, r1503152 Compiled: Mon Jul 22 15:23:09 PDT 2013 by mattf Upgrades: There are no upgrades in progress. Browse the filesystem Namenode Logs -----------------------------------------------------------------------
--------- Cluster Summary 7 files and directories, 1 blocks = 8 total. Heap Size is 31.32 MB /
966.69 MB (3%)
Configured Capacity : 5.65 GB DFS Used : 28.01 KB Non DFS
Used : 3.17 GB DFS Remaining : 2.47 GB DFS Used% : 0 % DFS
Remaining% : 43.82 % Live Nodes : 1 Dead Nodes : 0
Decommissioning Nodes : 0 Number of Under-Replicated Blocks : 0
-----------------------------------------------------------------------
--------- NameNode Storage:
Storage Directory Type State /home/hadoop/tmp/dfs/name IMAGE_AND_EDITS
Active
-----------------------------------------------------------------------
--------- This is Apache Hadoop release 1.2.1
至此 hadoop安装完成
