Apache 的配置与应用、日志管理（rotatelogs、cronolog、AWStats）

@[toc]
前言：
apache配置剖析

• apache链接保持
• apache访问控制
• 使用黑白名单，身份验证进行访问控制
  apache日志管理
• 日志分割
• AWStats日志分析

日志文件如何产生
管理日志的工具学习

一：apache连接保持

1.1 apache 连接保持相关参数

1.1.1 KeepAlive

• 是否打开连接保持，OFF关闭，ON打开

  1.1.2 KeepAliveTimeout

• 一次连接多次请求之间的最大间隔时间，两次请求超过该时间连接断开————可以对资源进行优化

  1.1.3 MaxKeepAliveRequests

• 一次连接能够传输的最大请求数量————并发量

二：apache访问控制概述

2.1 apache访问控制

2.1.1 作用

• 控制对网站资源的访问
• 为特定的网站目录添加访问授权

  2.1.2 常用访问控制方式

• 客户机地址限制
• 用户授权限制
  

  2.2 基于客户端地址的访问控制

• 使用Require配置项实现访问控制，按先后顺序限制
• 可用于<Location>、<Directory>、<Files>、<Limit>配置段中
• Require配置项的常见语法
  Require all granted——————允许所有权限
  Require all denied——————拒绝所有权限
  Require local——————————允许本地权限
  Require [not] host <主机名或域名列表>————允许或拒绝主机名
  Reuire [not] ip <IP地址或网段列表>——————允许或拒绝IP网段

备注：使用not禁止访问时要将其置于<RequireAll></RequireAll>容器中并在容器中指定相应的限制策略

[root@localhost html]# vim /etc/httpd/conf/extra/vhost.conf 
<VirtualHost 192.168.247.150:80>
  DocumentRoot "/var/www/html/kgc"
  ServerName www.kgc.com
  Errorlog "logs/www.kgc.com.error_log"
  Customlog "logs/www.kgc.comaccess_log" common
  <Directory "/var/www/html">
  <RequireAll>
  Require not ip 192.168.247.157
   Require all granted
  </RequireAll>
  </Directory>
</VirtualHost>

[root@localhost html]# systemctl restart httpd

三：用户授权限制

3.1 创建用户认证数据库

选项 -c ，代表新建用户认证数据库，若是想向现有数据库写入用户数据，不需要加-c，即可写入

[root@localhost named]# htpasswd -c /etc/httpd/conf/httppasswd zhangsan
New password: 
Re-type new password: 
Adding password for user zhangsan
[root@localhost named]# cat /etc/httpd/conf/httppasswd 
zhangsan:$apr1$IivUd6IL$J8zc5KAHgsQsoqSkPI1EP.
[root@localhost named]# htpasswd  /etc/httpd/conf/httppasswd lisi
New password: 
Re-type new password: 
Adding password for user lisi
[root@localhost named]# cat /etc/httpd/conf/httppasswd 
zhangsan:$apr1$IivUd6IL$J8zc5KAHgsQsoqSkPI1EP.
lisi:$apr1$FRyBVvZl$FBdIus.U9PpGVvmEgyAIK0
[root@localhost named]# htpasswd -c /etc/httpd/conf/httppasswd wangermazi
New password: 
Re-type new password: 
Adding password for user wangermazi
[root@localhost named]# cat /etc/httpd/conf/httppasswd 
wangermazi:$apr1$WwVYzzto$/ydcv1CaajW6e4Qi87D7u.

3.2 添加用户授权配置，开启用户认证数据库从服务器自身的层面去控制服务

以上次配置的虚拟web主机之一为例

<VirtualHost 192.168.247.150:80>
  DocumentRoot "/var/www/html/accp"
  ServerName www.accp.com
  Errorlog "logs/www.accp.com.error_log"
  Customlog "logs/www.accp.comaccess_log" common
  <Directory "/var/www/html">
   authname "documentroot"
   authtype basic
   authuserfile /etc/httpd/conf/httppasswd
   require valid-user
 </Directory>
</VirtualHost>

AuthName "DocumentRoot" ————制定受保护的领域名称，就是web站点目录
AuthType Basic ————认证类型
AuthUserFile /etc/httpd/conf/httppasswd ————用户认证帐号文件，也就是刚才创建的用户认证数据库
Require valid-user ————要求通过认证才能访问

然后重启httpd服务

[root@localhost extra]# systemctl restart httpd

3.3 客户机实验

四：日志分割

4.1 随着网站的访问量增加，默认情况下Apache的单个日志文件也会越来越大

• 日志文件在日积月累下，占用磁盘空间会很大
• 查看相关信息会不方便

  4.2 对日志文件进行分割的两种工具

• Apache自带rotatelogs分割工具实现
• 第三方工具cronolog分割

  4.3 rotatelogs分割工具

• 配置网站的日志文件转交给rotatelogs分割处理
• 配置格式————

ErrorLog "| /usr/sbin/rotatelogs -l logs/error_%Y%m%d.log 86400"
CustomLog "| /usr/sbin/rotatelogs -l logs/access_%Y%m%d.log 86400" combined

备注：在实际生产环境中，一个服务器绝大多数对应N个子域名站点，为了方便同意管理，可以用虚拟主机的方式进行配置，并用网站名标识日志文件
例如：ErrorLog “| rotatelogs(命令的绝对路径) -l 日志文件路径/网站名-error_%Y%m%d.log 86400”

4.4 实验：

恢复快照，开始配置

4.4.1 关闭防火墙，增强服务

[root@localhost ~]# systemctl stop firewalld.service 
[root@localhost ~]# setenforce 0
[root@localhost ~]# 

4.4.2 安装Apache

[root@localhost ~]# yum install httpd -y

4.4.3 此时日志目录中没有日志文件

[root@localhost ~]# cd /etc/httpd/logs
[root@localhost logs]# ls
[root@localhost logs]# cd ..
[root@localhost httpd]# ls -l
total 0
drwxr-xr-x. 2 root root  37 Dec 12 20:07 conf
drwxr-xr-x. 2 root root  82 Dec 12 20:07 conf.d
drwxr-xr-x. 2 root root 146 Dec 12 20:07 conf.modules.d
lrwxrwxrwx. 1 root root  19 Dec 12 20:07 logs -> ../../var/log/httpd
lrwxrwxrwx. 1 root root  29 Dec 12 20:07 modules -> ../../usr/lib64/httpd/modules
lrwxrwxrwx. 1 root root  10 Dec 12 20:07 run -> /run/httpd

4.4.4 开启服务，查看日志目录下，发现出现了错误日志和登陆日志，如果不对其进行配置，则每日都会把

[root@localhost httpd]# systemctl start httpd
[root@localhost httpd]# ls logs
access_log  error_log

4.4.5 配置两个文件属性

[root@localhost httpd]# which rotatelogs
/usr/sbin/rotatelogs
[root@localhost httpd]# vim /etc/httpd/conf/httpd.conf 
ErrorLog "| /usr/sbin/rotatelogs -l logs/error_%Y%m%d.log 86400"
CustomLog "| /usr/sbin/rotatelogs -l logs/access_%Y%m%d.log 86400" combined

4.4.6 重启服务，查看日志目录

[root@localhost httpd]# systemctl restart httpd
[root@localhost httpd]# ls logs
access_log  error_20191212.log  error_log
[root@localhost httpd]# 

4.4.7 访问一次

访问一次

access日志文件也出现了

[root@localhost httpd]# ls logs
access_20191212.log  access_log  error_20191212.log  error_log

4.5 cronolog工具

4.5.1 快照恢复新环境

重新安装thppd

4.5.2 挂载共享目录导入cronolog 软件包到系统内

[root@localhost ~]# mount.cifs //192.168.254.10/linuxs /abc
Password for root@//192.168.254.10/linuxs:  
[root@localhost ~]# cd /abc
[root@localhost abc]# ls
cronolog-1.6.2-14.el7.x86_64.rpm  LAMP-php5.6.txt             修改网卡为静态IP地址.txt
dhcp.txt                          MAC 记录与端口扫描脚本.txt  开发系统监控脚本.txt
extundelete-0.2.4.tar.bz2         pxe.txt                     引导系统脚本.txt
httpd2.4.2版本                    pxe速成秘籍.txt             测试网段是否存活，测试存活网段的21端口.txt
john-1.8.0.tar.gz                 qqq.html                    监控系统内存cpu磁盘容量1.0.txt
LAMP-C7                           vsftpd添加虚拟用户脚本.TXT
LAMP-C7.rar                       不同域名创建虚拟主机.TXT
[root@localhost abc]# cp -p cronolog-1.6.2-14.el7.x86_64.rpm /opt
-bash: ls/opt: No such file or directory
[root@localhost abc]# cd /opt
[root@localhost opt]# ls
cronolog-1.6.2-14.el7.x86_64.rpm  rh
[root@localhost opt]# rpm -ivh cronolog-1.6.2-14.el7.x86_64.rpm 
warning: cronolog-1.6.2-14.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:cronolog-1.6.2-14.el7            ################################# [100%]   
[root@localhost opt]# cron
crond       cronolog    cronosplit  crontab     
[root@localhost opt]# which cronolog
/usr/sbin/cronolog

4.5.3 关闭防火墙和增强服务

[root@localhost opt]# systemctl stop firewalld.service 
[root@localhost opt]# setenforce 0

4.5.4 修改配置文件中的日志参数

[root@localhost opt]# vim /etc//httpd/conf/httpd.conf 
ErrorLog "| /usr/sbin/cronolog logs/error_%Y%m%d.log"
CustomLog "| /usr/sbin/cronolog logs/access_%Y%m%d.log" combined

4.5.5 开启服务

[root@localhost opt]# systemctl start httpd

4.5.6 查看日志目录

[root@localhost logs]# ls
error_20191213.log

4.5.6 访问apache会产生access记录

[root@localhost logs]# ls
access_20191213.log  error_20191213.log

4.5.7 时间加速验证——使用date -s 验证

[root@localhost logs]# date -s 2019-12-31
Tue Dec 31 00:00:00 CST 2019
[root@localhost logs]# ls
access_20191213.log  error_20191213.log
[root@localhost logs]# systemctl restart httpd
[root@localhost logs]# ls -l
total 12
-rw-r--r--. 1 root root 1546 Dec 13 09:11 access_20191213.log
-rw-r--r--. 1 root root  888 Dec 13 09:11 error_20191213.log
-rw-r--r--. 1 root root  750 Dec 31 00:00 error_20191231.log

五：部署AWStats日志分析系统

AWStats 日志分析系统介绍

• Perl语言开发的一款开源日志分析系统
• 可用来分析Apache、Samba、Vsftpd、IIS等服务器的访问日志
• 信息结合crond等计划任务服务，可对日志内容定期进行分析
  

  5.1 安装AWStats软件包

  [root@localhost ~]# mount.cifs //192.168.254.10/linuxs /opt
  Password for root@//192.168.254.10/linuxs:  
  [root@localhost ~]# cd /opt
  [root@localhost opt]# ls
  12.17                             LAMP-php5.6.txt
  8.tar.gz                          MAC 记录与端口扫描脚本.txt
  awstats-7.6.tar.gz                pxe.txt

[root@localhost opt]# tar -xzf awstats-7.6.tar.gz -C /mnt
[root@localhost opt]# cd /mnt
[root@localhost mnt]# ls
12.17  awstats-7.6
[root@localhost mnt]# umount /opt
[root@localhost mnt]# ls /opt
rh
[root@localhost mnt]# mv awstats-7.6 /usr/local/awstats

awstats内的文件

[root@localhost mnt]# cd /usr/local/awstats/
[root@localhost awstats]# ls
docs  README.md  tools  wwwroot
[root@localhost awstats]# cd tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  awstats_updateall.pl  httpd_conf          nginx               xslt
awstats_configure.pl         dolibarr              logresolvemerge.pl  urlaliasbuilder.pl
awstats_exportlib.pl         geoip_generator.pl    maillogconvert.pl   webmin

备注：此时可以也配置bind和httpd进行实操演练

[root@localhost ~]# yum install bind httpd -y
Installed:
  bind.x86_64 32:9.11.4-9.P2.el7                httpd.x86_64 0:2.4.6-90.el7.centos   

 [root@localhost tools]# vim /etc/named.conf 
 12 options {
 13         listen-on port 53 { any; };
 14         listen-on-v6 port 53 { ::1; };
 15         directory       "/var/named";
 16         dump-file       "/var/named/data/cache_dump.db";
 17         statistics-file "/var/named/data/named_stats.txt";
 18         memstatistics-file "/var/named/data/named_mem_stats.txt";
 19         recursing-file  "/var/named/data/named.recursing";
 20         secroots-file   "/var/named/data/named.secroots";
 21         allow-query     { any; };

[root@localhost tools]# vim /etc/named.rfc1912.zones 
zone "kgc.com" IN {
        type master;
        file "kgc.com.zone";
        allow-update { none; };
};

[root@localhost tools]# cp -p /var/named/named.localhost kgc.com.zone
[root@localhost tools]# vim kgc.com.zone 
        A       192.168.247.149
www     IN      A       192.168.247.149

[root@localhost named]# systemctl start named
[root@localhost named]# netstat -natp | grep named
tcp        0      0 192.168.247.149:53      0.0.0.0:*               LISTEN      37654/named         
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      37654/named         
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      37654/named         
tcp6       0      0 ::1:53                  :::*                    LISTEN      37654/named         
tcp6       0      0 ::1:953                 :::*                    LISTEN      37654/named     

[root@localhost named]# systemctl stop firewalld.service 
[root@localhost named]# setenforce 0

配置httpd

[root@localhost named]# vim /etc/httpd/conf/httpd.conf 
41 Listen 192.168.247.149:80
42 #Listen 80
95 ServerName www.kgc.com:80

5.2 为要统计的站点建立配置文件

[root@localhost named]# cd /usr/local/awstats/
[root@localhost awstats]# ls
docs  README.md  tools  wwwroot
[root@localhost awstats]# cd tools/
[root@localhost tools]# ls
awstats_buildstaticpages.pl  dolibarr            logresolvemerge.pl  webmin
awstats_configure.pl         geoip_generator.pl  maillogconvert.pl   xslt
awstats_exportlib.pl         httpd_conf          nginx
awstats_updateall.pl         kgc.com.zone        urlaliasbuilder.pl

pl为结尾的是脚本文件，若是不可执行，需要额外给其增加执行权限

[root@localhost tools]# ./awstats_configure.pl 

----- AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
-----------------------------------------输入配置文件路径-------------------------------------------------
> /etc/httpd/conf/httpd.conf

-----> Check and complete web server config file '/etc/httpd/conf/httpd.conf'
  Add 'Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"'
  Add 'Alias /awstatscss "/usr/local/awstats/wwwroot/css/"'
  Add 'Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"'
  Add 'ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"'
  Add '<Directory>' directive
  AWStats directives added to Apache config file.

-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
---------------------------------------------同意----------------------------------------------------
file (required if first install) [y/N] ? y

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
----------------------------------------------输入域名--------------------------------------------------
> www.kgc.com

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
--------------------------------------------默认即可，回车-------------------------------------------------
> 

-----> Create config file '/etc/awstats/awstats.www.kgc.com.conf'
 Config file /etc/awstats/awstats.www.kgc.com.conf created.

-----> Restart Web server with '/sbin/service httpd restart'
Redirecting to /bin/systemctl restart httpd.service

-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.kgc.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue... 
--------------------------------------------回车----------------------------------------------------

A SIMPLE config file has been created: /etc/awstats/awstats.www.kgc.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.kgc.com' with command:
> perl awstats.pl -update -config=www.kgc.com
You can also read your statistics for 'www.kgc.com' with URL:
> http://localhost/awstats/awstats.pl?config=www.kgc.com
//http://localhost/awstats/awstats.pl?config=www.kgc.com这个路径是登陆awstats的站点
Press ENTER to finish...
-------------------------------------------回车完成--------------------------------------------------------------------

[root@localhost tools]# 

5.3 为要统计的站点建立配置文件

下面是awstats写入的数据

#
# Directives to allow use of AWStats as a CGI
#
Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"
Alias /awstatscss "/usr/local/awstats/wwwroot/css/"
Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"
ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"

#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

对其进行修改

[root@localhost tools]# vim /etc/httpd/conf/httpd.conf 
<Directory "/usr/local/awstats/wwwroot">
    Options None
    AllowOverride None
#    Order allow,deny
#    Allow from all
    Require all granted
</Directory>

5.4 修改站点统计配置文件

[root@localhost tools]# vim /etc/awstats/awstats.www.kgc.com.conf 
 50 LogFile="/var/log/httpd/access_log"
220 DirData="/var/lib/awstats"

dirdata 数据存储目录，因为这个目录不存在，所以创建这个目录

[root@localhost tools]# mkdir /var/lib/awstats

5.4.2 重新启动httpd

[root@localhost tools]# systemctl restart httpd
[root@localhost tools]# 

备注：http://localhost/awstats/awstats.pl?config=www.kgc.com中的localhost修改为域名

5.5 刷新日志数据./awstats_updateall.pl now

[root@localhost tools]# ./awstats_updateall.pl now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.kgc.com -configdir="/etc/awstats"' to update config www.kgc.com
Create/Update database for config "/etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/var/log/httpd/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 250
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 1 corrupted records,
 Found 0 old records,
 Found 249 new qualified records.

5.6 执行日志分析，并设置cron计划任务

5.6.1 首先先获取该执行脚本的绝对路径

[root@localhost tools]# pwd
/usr/local/awstats/tools
[root@localhost tools]# ls
awstats_buildstaticpages.pl  awstats_updateall.pl  httpd_conf          nginx               xslt
awstats_configure.pl         dolibarr              logresolvemerge.pl  urlaliasbuilder.pl
awstats_exportlib.pl         geoip_generator.pl    maillogconvert.pl   webmin
[root@localhost tools]# 

5.6.2 然后进行添加crond任务，开启crond，设置开机自启动

[root@localhost tools]# crontab -e
*/5 * * * * /usr/local/awstats/tools/awstats_updateall.pl now
[root@localhost tools]# crontab -l
*/5 * * * * /usr/local/awstats/tools/awstats_updateall.pl now
[root@localhost tools]# systemctl start crond
[root@localhost tools]# systemctl enable crond

六 ：访问AWStats分析系统

6.1 查看统计页面

6.2 设置页面自动跳转，方便访问

[root@localhost tools]# cd /var
[root@localhost var]# ls
account  cache  db     games   kerberos  local  log   named  opt       run    target  www
adm      crash  empty  gopher  lib       lock   mail  nis    preserve  spool  tmp     yp
[root@localhost var]# cd www/
[root@localhost www]# ls
cgi-bin  html
[root@localhost www]# cd html/
[root@localhost html]# ls
[root@localhost html]# vim aws.html
<html>
 <head>
   <meta http-equiv=refresh content="0;url=http://www.kgc.com/awstats/awstats.pl?config=www.kgc.com">
 </head>
 <body></body>
</html>

6.2.2 然后重新启动httpd

[root@localhost html]# systemctl restart httpd

总结

• Apache链接保持相关参数
• Apache访问控制作用及常用控制方式
• Apache日志分割方法
• AWStats分析系统部署及应用