Apache配置与应用之AWstats日志分析实验流程

Apache配置与应用之AWstats日志分析实验流程

实验环境：centos7操作系统、一台win10虚拟机操作系统，需要软件包：

bind.x86_64 32:9.11.4-9.P2.el7 httpd.x86_64 0:2.4.6-90.el7.centos（这里为了方便使用yum直接安装）

awstats-7.6.tar.gz（需要自己准备，链接：https://pan.baidu.com/s/1apCEUdFoAoUTVeWwYyJaow
提取码：wt6u ）

具体操作流程如下：

1）解压缩awstats软件包

tar zxvf awstats-7.6.tar.gz -C /usr/local

ls /usr/local/

2）安装bind和httpd服务配置bind相关文件

yum install -y bind httpd

[root@localhost tools]# vim /etc/named.conf 
[root@localhost tools]# head -21 /etc/named.conf  | tail 
options {
        listen-on port 53 { any; };
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        recursing-file  "/var/named/data/named.recursing";
        secroots-file   "/var/named/data/named.secroots";
        allow-query     { any; };
[root@localhost tools]# vim /etc/named.rfc1912.zones 
[root@localhost tools]# head -29 /etc/named.rfc1912.zones  | tail -5
zone "lcc.com" IN {
        type master;
        file "lcc.com.zone";
        allow-update { none; };
};
[root@localhost tools]# cd /var/named/
[root@localhost named]# ls
data  dynamic  named.ca  named.empty  named.localhost  named.loopback  slaves
[root@localhost named]# cp -p named.localhost lcc.com.zone
[root@localhost named]# vim lcc.com.zone 
[root@localhost named]# cat lcc.com.zone 
$TTL 1D
@       IN SOA  @ rname.invalid. (
                                        0       ; serial
                                        1D      ; refresh
                                        1H      ; retry
                                        1W      ; expire
                                        3H )    ; minimum
        NS      @
        A       127.0.0.1
www IN  A       192.168.68.145
[root@localhost named]# systemctl stop firewalld.service 
[root@localhost named]# setenforce 0
[root@localhost named]# systemctl start named
[root@localhost named]# systemctl start httpd
[root@localhost named]# netstat -antp | grep httpd
tcp6       0      0 :::80                   :::*                    LISTEN      88534/httpd         
[root@localhost named]# netstat -antp | grep named
tcp        0      0 192.168.68.145:53       0.0.0.0:*               LISTEN      88258/named         
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      88258/named         
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      88258/named         
tcp6       0      0 ::1:53                  :::*                    LISTEN      88258/named         
tcp6       0      0 ::1:953                 :::*                    LISTEN      88258/named  

在win10的虚拟机上测试一下是否可以成功解析方才设置的域名

3）为要统计的站点建立配置文件

指定httpd主配置文件的路径——》设置日志类型——》为指定web站点创建配置文件

[root@localhost tar]# cd /usr/local/awstats-7.6/tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  logresolvemerge.pl
awstats_configure.pl         maillogconvert.pl
awstats_exportlib.pl         nginx
awstats_updateall.pl         urlaliasbuilder.pl
dolibarr                     webmin
geoip_generator.pl           xslt
httpd_conf
[root@localhost tools]# ./awstats_configure.pl 

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/usr/local/awstats-7.6
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ? y   //输入y

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf       //HTTPd的主配置文件的路径

-----> Check and complete web server config file '/etc/httpd/conf/httpd.conf'
  All AWStats directives are already present.

-----> Update model config file '/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y             //输入y

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.lcc.com                                       //自己设置的域名

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>                                                                      //默认下面全都回车即可

-----> Create config file '/etc/awstats/awstats.www.lcc.com.conf'
 Config file /etc/awstats/awstats.www.lcc.com.conf created.

-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:  //执行日志分析的方法（两种）
/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl -update -config=www.lcc.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats-7.6/tools/awstats_updateall.pl now      //会报错，解决方法在文末
Press ENTER to continue...  

A SIMPLE config file has been created: /etc/awstats/awstats.www.lcc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.lcc.com' with command:
> perl awstats.pl -update -config=www.lcc.com
You can also read your statistics for 'www.lcc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.lcc.com   //这里就是我们要测试的地址，需要将localhost换为www.lcc.com

Press ENTER to finish...

[root@localhost tools]# systemctl restart httpd

4）修改Apache 2.4以上版本自动生成的awstats访问权限

[root@localhost tools]# vim /etc/httpd/conf/httpd.conf 
[root@localhost tools]# tail /etc/httpd/conf/httpd.conf 
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats-7.6/wwwroot">
    Options None
    AllowOverride None
    #Order allow,deny            //注释掉
    #Allow from all             //注释掉
    Require all granted         //添加（所有权限）
</Directory>

5）修改站点统计配置文件

[root@localhost tools]# vim /etc/awstats/awstats.www.lcc.com.conf 
[root@localhost tools]# head -50 /etc/awstats/awstats.www.lcc.com.conf  | tail -1
LogFile="/var/log/httpd/access_log"
[root@localhost tools]# head -220 /etc/awstats/awstats.www.lcc.com.conf  | tail -1
DirData="/var/lib/awstats"
[root@localhost tools]# cd /var/lib/               //创建上面路径的文件夹awstats
[root@localhost lib]# mkdir awstats

6）执行日志分析

//在上面的第三步过程中是给出方法来执行日志分析的方式的，而且也可以通过执行周期性计划任务来方便管理日志
//每执行一次该命令，刷新访问页面就会对相关数据进行统计显示出来
[root@localhost tools]# /usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl -update -config=www.lcc.com
Create/Update database for config "/etc/awstats/awstats.www.lcc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 0
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 0 new qualified records.

为了方便管理，那么我们可以使用周期性任务计划来管理执行有兴趣可以使用crontab -e 命令来创建周期性任务计划表

小结

本次实验是为了实现使用awstats进行日志分析，过程并不复杂，但是需要注意的地方还是需要留意的，对应之处本人稍作了一些注释。

当然该实验有两个问题，这边仔细讲述一下：

1.使用http://www.lcc.com/awstats/awstats.pl?config=www.lcc.com 进行访问不容易让人记住，所有需要优化一下

[root@localhost html]#  cd /usr/local/awstats-7.6/tools/
[root@localhost tools]# cd -
/var/www/html
[root@localhost html]# vim aws.html
[root@localhost html]# cat aws.html 
<html>
<head>
<meta http-equiv=refresh content="0;
url=http://www.lcc.com/awstats/awstats.pl?config=www.lcc.com">
</head>
<body>
</body>
</html>

这样设置之后可以使用http://www.lcc.com/aws.html进行访问

2.在执行日志分析命令时使用第二种方法会遇到如下error

[root@localhost tools]# /usr/local/awstats-7.6/tools/awstats_updateall.pl now
Error: Can't find AWStats program ('awstats.pl').
Use -awstatsprog option to solve this.

这个原因是因为未指定awstats.pl的绝对路径，根据如下操作即可

[root@localhost tools]# find /usr/local/ -name awstats.pl
/usr/local/awstats-7.6/wwwroot/cgi-bin/awstats.pl
[root@localhost tools]# cd ../wwwroot/cgi-bin/
[root@localhost cgi-bin]# ls
awdownloadcsv.pl  awredir.pl  awstats.model.conf  awstats.pl  lang  lib  plugins
[root@localhost tools]# /usr/local/awstats-7.6/tools/awstats_updateall.pl now -awstatsprog="../wwwroot/cgi-bin/awstats.pl"
Running '"../wwwroot/cgi-bin/awstats.pl" -update -config=www.lcc.com -configdir="/etc/awstats"' to update config www.lcc.com
Create/Update database for config "/etc/awstats/awstats.www.lcc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 101)
Jumped lines in file: 101
 Found 101 already parsed records.
Parsed lines in file: 236
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 2 corrupted records,
 Found 0 old records,
 Found 234 new qualified records.