温馨提示×

温馨提示×

您好,登录后才能下订单哦!

密码登录×
登录注册×
其他方式登录
点击 登录注册 即表示同意《亿速云用户服务条款》

Apache Storm环境配置

发布时间:2020-07-24 15:11:14 来源:网络 阅读:1613 作者:wu5yanan 栏目:大数据

I. 准备

apache-storm-0.9.3 zookeeper-3.4.6


II. 结构

一台server装nimbus和zookeeper,两台supervisor。


III. 配置

ZooKeeper:

# vi conf/zoo.cfg

clientPort=10205
server.0=nimbus:2888:3888

#vi /var/zookeeper/myid

<
与zoo.cfg中server.后面的数字一致,如nimbus机器改为0
>


Storm:

# vi conf/storm.yaml

storm.zookeeper.servers:
     - "nimbus"
     - "supervisor1"
     - "supervisor2"
     - "supervisor3"
storm.zookeeper.port: 10205
storm.local.dir: "/mnt/storm" #
表示storm需要用到的本地目录,
比如topology的jar文件保存,
java.library.path: /usr/local/lib:/opt/local/lib:/usr/lib
nimbus.host: "nimbus"
supervisor.slots.ports:  #
表示supervisor节点的槽数,即每个supervisor能开几个worker进程。
    - 6700
    - 6701
    - 6702
    - 6703


IV. 步骤

1.在nimbus上启动zookeeper。zk必须要先启动,否则nimbus启动会报错。

#./zkServer.sh start
JMX enabled by default
Using config: /root/storm/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
# bin/zkCli.sh stat /测试服务是否正常启动
Connecting to localhost:2181
2015-05-28 05:32:11,600 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-05-28 05:32:11,604 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=mats03.webex.com
2015-05-28 05:32:11,605 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.7.0_45
2015-05-28 05:32:11,607 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2015-05-28 05:32:11,607 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64/jre
2015-05-28 05:32:11,607 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/root/storm/zookeeper-3.4.6/bin/../build/classes:/root/storm/zookeeper-3.4.6/bin/../build/lib/*.jar:/root/storm/zookeeper-3.4.6/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/storm/zookeeper-3.4.6/bin/../lib/slf4j-api-1.6.1.jar:/root/storm/zookeeper-3.4.6/bin/../lib/netty-3.7.0.Final.jar:/root/storm/zookeeper-3.4.6/bin/../lib/log4j-1.2.16.jar:/root/storm/zookeeper-3.4.6/bin/../lib/jline-0.9.94.jar:/root/storm/zookeeper-3.4.6/bin/../zookeeper-3.4.6.jar:/root/storm/zookeeper-3.4.6/bin/../src/java/lib/*.jar:/root/storm/zookeeper-3.4.6/bin/../conf::/home/oracle/OraHome1/jlib:/home/oracle/OraHome1/rdbms/jlib
2015-05-28 05:32:11,607 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=:/home/oracle/OraHome1/lib:/opt/webex/mats/bin:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-279.9.1.el6.x86_64
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2015-05-28 05:32:11,608 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2015-05-28 05:32:11,609 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/root/storm/zookeeper-3.4.6
2015-05-28 05:32:11,610 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@75ab5e0b


2.启动主节点nimbus和ui

nohup bin/storm nimbus &
nohup bin/storm ui &

3.启动从节点supervisor

nohup bin/storm supervisor &

4.查看界面:http://<nimbusIP>:18080/index.html

Apache Storm环境配置


V. Example

这是storm安装包里提供的example,路径apache-storm-0.9.3\examples\storm-starter\src\jvm\storm\starter,提交一个topology命名为wordcountexample。

# ./storm jar /home/slimttyagent/
storm-starter-topologies-0.9.3.jar storm/starter/WordCountTopology wordcountexample
519  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
525  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar /home/slimttyagent/storm-starter-topologies-0.9.3.jar to assigned location: /var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jar
Start uploading file '/home/slimttyagent/storm-starter-topologies-0.9.3.jar' to '/var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jar' (3248859 bytes)
[==================================================] 3248859 / 3248859
File '/home/slimttyagent/storm-starter-topologies-0.9.3.jar' uploaded to '/var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jar' (3248859 bytes)
557  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /var/storm/nimbus/inbox/stormjar-d8c7a8d9-3e76-444f-b86b-b0e5a45b99da.jar
558  [main] INFO  backtype.storm.StormSubmitter - Submitting topology wordcountexample in distributed mode with conf {"topology.workers":3,"topology.debug":true}
737  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: wordcountexample

Apache Storm环境配置

Apache Storm环境配置


向AI问一下细节

免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。

AI