win7搭建为伪分布式hadoop的步骤

这篇文章主要介绍“win7搭建为伪分布式hadoop的步骤”，在日常操作中，相信很多人在win7搭建为伪分布式hadoop的步骤问题上存在疑惑，小编查阅了各式资料，整理出简单好用的操作方法，希望对大家解答”win7搭建为伪分布式hadoop的步骤”的疑惑有所帮助！接下来，请跟着小编一起来学习吧！

window7下安装hadoop [32位]
1.下载hadoop
hadoop-2.2.0.tar.gz
解压
2.配置环境变量hadoop
3.修改%HADOOP_HOME%\etc\hadoop的hadoop-env.sh
export JAVA_HOME=D:\Java\jdk1.6.0_10#不能有空格
末尾添加[会出错就不添加]
set HADOOP_PREFIX=E:\hadoop\hadoop-2.2.0\
set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop  
set YARN_CONF_DIR=%HADOOP_CONF_DIR%  
set PATH=%PATH%;%HADOOP_PREFIX%\bin 

下载hadoop-common-2.2.0-bin-master

把里面的文件拷贝到hadoop/bin下

4.配置core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>  
      <property>  
        <name>fs.default.name</name>  
        <value>hdfs://0.0.0.0:19000</value>  
      </property>  
 </configuration>  
 
5.配置hdfs-site.xml

  <?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->

    <configuration>  
      <property>  
        <name>dfs.replication</name>  
        <value>1</value>  
      </property>  
    </configuration>  

    
6.配置mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

    <configuration>  
      
       <property>  
         <name>mapreduce.job.user.name</name>  
         <value>%USERNAME%</value>  
       </property>  
      
       <property>  
         <name>mapreduce.framework.name</name>  
         <value>yarn</value>  
       </property>  
      
      <property>  
        <name>yarn.apps.stagingDir</name>  
        <value>/user/%USERNAME%/staging</value>  
      </property>  
      
      <property>  
        <name>mapreduce.jobtracker.address</name>  
        <value>local</value>  
      </property>  
      
    </configuration>  


7.创建yarn-site.xml文件
<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

    <configuration>  
      <property>  
        <name>yarn.server.resourcemanager.address</name>  
        <value>0.0.0.0:8020</value>  
      </property>  
      
      <property>  
        <name>yarn.server.resourcemanager.application.expiry.interval</name>  
        <value>60000</value>  
      </property>  
      
      <property>  
        <name>yarn.server.nodemanager.address</name>  
        <value>0.0.0.0:45454</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.aux-services</name>  
        <value>mapreduce_shuffle</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>  
      </property>  
      
      <property>  
        <name>yarn.server.nodemanager.remote-app-log-dir</name>  
        <value>/app-logs</value>  
      </property>  
      
      <property>  
        <name>yarn.nodemanager.log-dirs</name>  
        <value>/dep/logs/userlogs</value>  
      </property>  
      
      <property>  
        <name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name>  
        <value>0.0.0.0</value>  
      </property>  
      
      <property>  
        <name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name>  
        <value>0.0.0.0</value>  
      </property>  
      
      <property>  
        <name>yarn.log-aggregation-enable</name>  
        <value>true</value>  
      </property>  
      
      <property>  
        <name>yarn.log-aggregation.retain-seconds</name>  
        <value>-1</value>  
      </property>  
      
      <property>  
        <name>yarn.application.classpath</name>  
        <value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value>  
      </property>  
    </configuration>  

    切换到E:\hadoop\hadoop-2.2.0\etc\hadoop运行hadoop-env.cmd脚本，设置当前命令窗口执行环境变量
   执行 hdfs namenode -format
   16/02/27 11:36:23 INFO util.GSet: VM type       = 32-bit
16/02/27 11:36:23 INFO util.GSet: 0.029999999329447746% max memory = 992.3 MB
16/02/27 11:36:23 INFO util.GSet: capacity      = 2^16 = 65536 entries
16/02/27 11:36:24 INFO common.Storage: Storage directory \tmp\hadoop-goudcheng\d
fs\name has been successfully formatted.
16/02/27 11:36:24 INFO namenode.FSImage: Saving image file \tmp\hadoop-goudcheng
\dfs\name\current\fsimage.ckpt_0000000000000000000 using no compression
16/02/27 11:36:24 INFO namenode.FSImage: Image file \tmp\hadoop-goudcheng\dfs\na
me\current\fsimage.ckpt_0000000000000000000 of size 201 bytes saved in 0 seconds
.
16/02/27 11:36:24 INFO namenode.NNStorageRetentionManager: Going to retain 1 ima
ges with txid >= 0
16/02/27 11:36:24 INFO util.ExitUtil: Exiting with status 0
16/02/27 11:36:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at CDCH20100020-5/172.31.168.244

到此，关于“win7搭建为伪分布式hadoop的步骤”的学习就结束了，希望能够解决大家的疑惑。理论与实践的搭配能更好的帮助大家学习，快去试试吧！若想继续学习更多相关知识，请继续关注亿速云网站，小编会继续努力为大家带来更多实用的文章！