Squid 缓存的相关操作及使用说明

一、查看squid命中情况

除了之前在squid的access.log中查看squid缓存命中的方法之外，还可以通过以下方法进行查看：

这里对图片服务器的httpd配置文件添加一些新的配置如下：

ExpiresActive on

  ExpiresDefault "access plus 12 month"

  ExpiresByType text/html "access plus 12 months"

  ExpiresByType text/css "access plus 12 months"

  ExpiresByType image/gif "access plus 12 months"

  ExpiresByType image/jpeg "access plus 12 months"  

  ExpiresByType image/jpg "access plus 12 months"

  ExpiresByType image/png "access plus 12 months"

  ExpiresByType application/x-javascript "access plus 12 months"

  ExpiresByType vedio/x-flv "access plus 12 months"

把以上内容追加到httpd.conf文件中，然后重新启动httpd服务。

1）使用curl命令查看

[root@squid ~]# curl -I -s http://192.168.49.135/lnmp.jpg

HTTP/1.0 200 OK

Date: Mon, 31 Oct 2016 03:10:20 GMT

Server: Apache/2.2.15 (CentOS)

Last-Modified: Thu, 26 Jun 2014 01:12:31 GMT

ETag: "bf742-2f37a-4fcb2e3149dc0"

Accept-Ranges: bytes

Content-Length: 193402

Content-Type: image/jpeg

Age: 22522

X-Cache: HIT from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

[root@squid ~]# curl -I -s http://192.168.49.135/lnmp.jpg|grep -i x-cache

X-Cache: HIT from cache1.contoso.com

[root@squid ~]# curl -I -s http://192.168.49.135/php.jpg|grep -i x-cache

X-Cache: HIT from cache1.contoso.com

[root@squid ~]# curl -I -s http://192.168.49.135/nginx.jpg|grep -i x-cache

X-Cache: HIT from cache1.contoso.com

[root@squid ~]# curl -I -s http://192.168.49.135/mysql.png|grep -i x-cache

X-Cache: HIT from cache1.contoso.com

2）使用Firefox的firebug插件进行查看

可以看到，在响应头信息中，有max-age也就是我在httpd中设置的expire时间，同样也可以在Expires看到过期时间，Server项显示web服务器的类型，这里是Apache；Via表示通过哪个代理服务器进行访问，这里是我在squid.conf中配置的名称cache1.contoso.com（后面还有squid的版本信息）；X-Cache中HIT from cache1.contoso.com表示请求内容在cache1.contoso.com上命中。

3）可能遇到的问题及原因

问题1：日志看不到命中

原因：curl的不是squid缓存服务器地址，而是web服务器，所以看不到命中。

问题2：看到的总是MISS

原因：使用了Ctrl+F5刷新，所以总是cache丢失。

问题3：是否做了服务器时间同步？

服务器时间同步会影响缓存结果，需要在服务器上进行时间同步。

二、修改squid的默认日志格式

squid默认记录日志格式如下：

logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %Hs %h" "%{User-Agent}>h" %Ss:%Sh

修改squid配置文件，添加如下内容：

# 下面是logs相关配置

emulate_httpd_log on

logformat squid %{X-Forwarded-For}>h %ui %un [%tl] "%rm %ru HTTP/%rv" %Hs %h" "%{User-Agent}>h" %Ss:%Sh  #修改后的access.log格式

access_log /usr/local/squid/var/logs/access.log squid

cache_log /usr/local/squid/var/logs/cache.log

cache_store_log /usr/local/squid/var/logs/store.log

logfile_rotate 20

修改完成后，重启squid服务，再进行日志的查看：

killall squid

squid -k parse

squid -D

[root@squid etc]# tail -f ../var/logs/access.log

1477907509.273      0 192.168.49.139 TCP_HIT/200 358 HEAD http://192.168.49.135/php.jpg - NONE/- image/jpeg

1477907514.597      0 192.168.49.139 TCP_HIT/200 358 HEAD http://192.168.49.135/nginx.jpg - NONE/- image/jpeg

1477907521.269      0 192.168.49.139 TCP_HIT/200 360 HEAD http://192.168.49.135/lnmp.jpg - NONE/- image/jpeg

# 上面是修改之前的日志格式，下面是修改之后的日志格式

- - - [31/Oct/2016:06:06:13 -0400] "HEAD http://192.168.49.135/lnmp.jpg HTTP/1.1" 200 360 "-" "curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2" TCP_HIT:NONE

- - - [31/Oct/2016:06:07:15 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 304 250 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_IMS_HIT:NONE

- - - [31/Oct/2016:06:07:26 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 200 193829 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_CLIENT_REFRESH_MISS:FIRST_UP_PARENT

- - - [31/Oct/2016:06:07:27 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 200 193829 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_CLIENT_REFRESH_MISS:FIRST_UP_PARENT

- - - [31/Oct/2016:06:07:30 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 304 286 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_IMS_HIT:NONE

- - - [31/Oct/2016:06:07:31 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 304 286 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_IMS_HIT:NONE

- - - [31/Oct/2016:06:07:35 -0400] "GET http://192.168.49.135/lnmp.jpg HTTP/1.1" 304 286 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" TCP_IMS_HIT:NONE

# 修改之后的日志格式记录的信息更详细，另外时间也不再是一串看不懂的数字，更易于阅读

其中的对端编码含义如下表所示：（参考squid权威指南：http://zyan.cc/book/squid/chap13.html）

三、查看squid缓存文件情况

squidclient -h localhost -p 80 mgr:objects

这个命令能知道如下重要信息：

1、打印出的是所有缓存在内存和硬盘上的数据，对象是以key来表示，每个key代表不同的对象。

2、列出对象是否存放在内存，还是硬盘。

              IN_MEMORY   在内存中

     GET

命令执行结果示例：

[root@squid etc]# squidclient -h localhost -p 80 mgr:objects

HTTP/1.0 200 OK

Server: squid/3.0.STABLE20

Mime-Version: 1.0

Date: Mon, 31 Oct 2016 13:33:30 GMT

Content-Type: text/plain

Expires: Mon, 31 Oct 2016 13:33:30 GMT

Last-Modified: Mon, 31 Oct 2016 13:33:30 GMT

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

KEY 9C60DB2318E2EB2A91411D99F2EDD2C0

STORE_OK      IN_MEMORY     SWAPOUT_NONE PING_NONE   

SPECIAL,CACHABLE,VALIDATED

LV:1477908352 LU:1477908352 LM:1476530330 EX:-1       

0 locks, 0 clients, 0 refs

Swap Dir -1, File 0XFFFFFFFF

GET http://cache1.contoso.com/squid-internal-static/icons/anthony-image.gif

inmem_lo: 0

inmem_hi: 452

swapout: 0 bytes queued

KEY B44033DA692B451D4ED732876ECB50AC

STORE_OK      NOT_IN_MEMORY SWAPOUT_DONE PING_NONE   

CACHABLE,DISPATCHED,VALIDATED

LV:1476808880 LU:1476808880 LM:1476773657 EX:-1       

0 locks, 0 clients, 1 refs

Swap Dir 0, File 0X0001DF

KEY B5C0B572C5A3971F66058A0096D49366

STORE_OK      NOT_IN_MEMORY SWAPOUT_DONE PING_NONE   

CACHABLE,DISPATCHED,VALIDATED

LV:1476654128 LU:1476736902 LM:1473492811 EX:1508190128

0 locks, 0 clients, 1 refs

Swap Dir 0, File 0X000119

KEY B9406F5E8C58201286101BE33EBE58FF

STORE_OK      NOT_IN_MEMORY SWAPOUT_DONE PING_NONE   

CACHABLE,DISPATCHED,VALIDATED

LV:1460859669 LU:1476751308 LM:1459827216 EX:1492395669

0 locks, 0 clients, 1 refs

Swap Dir 0, File 0X000162

KEY C400A8CBAF19FB0C8A768B638D2E53CB

STORE_OK      IN_MEMORY     SWAPOUT_DONE PING_DONE   

CACHABLE,DISPATCHED,VALIDATED

LV:1477908447 LU:1477908456 LM:1403745151 EX:1509012447

0 locks, 0 clients, 5 refs

Swap Dir 0, File 0X000198

GET http://192.168.49.135/lnmp.jpg

inmem_lo: 0

inmem_hi: 193735

swapout: 193735 bytes queued

KEY E720447A2A98968C2673DD228AEADFC7

STORE_OK      IN_MEMORY     SWAPOUT_NONE PING_NONE   

SPECIAL,CACHABLE,VALIDATED

LV:1477908352 LU:1477908352 LM:1476530331 EX:-1       

0 locks, 0 clients, 0 refs

Swap Dir -1, File 0XFFFFFFFF

GET http://cache1.contoso.com/squid-internal-static/icons/anthony-xpm.gif

inmem_lo: 0

inmem_hi: 393

swapout: 0 bytes queued

KEY 63C142B374127C8B784BC7F5AE4DD63C

STORE_OK      NOT_IN_MEMORY SWAPOUT_DONE PING_NONE   

CACHABLE,DISPATCHED,VALIDATED

LV:1476589869 LU:1476636053 LM:1476589526 EX:-1       

0 locks, 0 clients, 1 refs

Swap Dir 0, File 0X000024

KEY 88C1A24267886989BC04C47A1484B4DA

STORE_OK      IN_MEMORY     SWAPOUT_NONE PING_NONE   

SPECIAL,CACHABLE,VALIDATED

LV:1477908352 LU:1477908352 LM:1476530331 EX:-1       

0 locks, 0 clients, 0 refs

Swap Dir -1, File 0XFFFFFFFF

GET http://cache1.contoso.com/squid-internal-static/icons/anthony-unknown.gif

inmem_lo: 0

inmem_hi: 388

swapout: 0 bytes queued

其他查看缓存的相关命令：

       取得squid运行状态信息：squidclient -p 80 mgr:info

       取得squid内存使用情况：squidclient -p 80 mgr:mem

       取得squid已经缓存的列表：squidclient -p 80 mgr:objects

       取得squid的磁盘使用情况：squidclient -p 80 mgr:diskd

常用的查看请求的命令：

   squidclient -h localhost -p 80 mgr:5min

    squidclient -h localhost -p 80 mgr:5min |egrep "client_http.requests|client_http.hits"

    squidclient -h localhost -p 80 mgr:mem

    squidclient -h localhost -p 80 mgr:info

[root@squid etc]# squidclient -h localhost -p 80 mgr:5min

HTTP/1.0 200 OK

Server: squid/3.0.STABLE20

Mime-Version: 1.0

Date: Mon, 31 Oct 2016 13:53:10 GMT

Content-Type: text/plain

Expires: Mon, 31 Oct 2016 13:53:10 GMT

Last-Modified: Mon, 31 Oct 2016 13:53:10 GMT

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

sample_start_time = 1477921689.217542 (Mon, 31 Oct 2016 13:48:09 GMT)

sample_end_time = 1477921989.280484 (Mon, 31 Oct 2016 13:53:09 GMT)

client_http.requests = 0.006665/sec

client_http.hits = 0.000000/sec

client_http.errors = 0.000000/sec

client_http.kbytes_in = 0.000000/sec

client_http.kbytes_out = 0.013331/sec

client_http.all_median_svc_time = 0.000000 seconds

client_http.miss_median_svc_time = 0.000000 seconds

client_http.nm_median_svc_time = 0.000000 seconds

client_http.nh_median_svc_time = 0.000000 seconds

client_http.hit_median_svc_time = 0.000000 seconds

server.all.requests = 0.000000/sec

server.all.errors = 0.000000/sec

server.all.kbytes_in = 0.000000/sec

server.all.kbytes_out = 0.000000/sec

server.http.requests = 0.000000/sec

server.http.errors = 0.000000/sec

server.http.kbytes_in = 0.000000/sec

server.http.kbytes_out = 0.000000/sec

server.ftp.requests = 0.000000/sec

server.ftp.errors = 0.000000/sec

server.ftp.kbytes_in = 0.000000/sec

server.ftp.kbytes_out = 0.000000/sec

server.other.requests = 0.000000/sec

server.other.errors = 0.000000/sec

server.other.kbytes_in = 0.000000/sec

server.other.kbytes_out = 0.000000/sec

icp.pkts_sent = 0.000000/sec

icp.pkts_recv = 0.000000/sec

icp.queries_sent = 0.000000/sec

icp.replies_sent = 0.000000/sec

icp.queries_recv = 0.000000/sec

icp.replies_recv = 0.000000/sec

icp.replies_queued = 0.000000/sec

icp.query_timeouts = 0.000000/sec

icp.kbytes_sent = 0.000000/sec

icp.kbytes_recv = 0.000000/sec

icp.q_kbytes_sent = 0.000000/sec

icp.r_kbytes_sent = 0.000000/sec

icp.q_kbytes_recv = 0.000000/sec

icp.r_kbytes_recv = 0.000000/sec

icp.query_median_svc_time = 0.000000 seconds

icp.reply_median_svc_time = 0.000000 seconds

dns.median_svc_time = 0.000000 seconds

unlink.requests = 0.000000/sec

page_faults = 0.000000/sec

select_loops = 196.805376/sec

select_fds = 0.026661/sec

average_select_fd_period = 0.000000/fd

median_select_fds = 0.000000

swap.outs = 0.000000/sec

swap.ins = 0.000000/sec

swap.files_cleaned = 0.000000/sec

aborted_requests = 0.000000/sec

syscalls.disk.opens = 0.000000/sec

syscalls.disk.closes = 0.000000/sec

syscalls.disk.reads = 0.000000/sec

syscalls.disk.writes = 0.000000/sec

syscalls.disk.seeks = 0.000000/sec

syscalls.disk.unlinks = 0.000000/sec

syscalls.sock.accepts = 0.013331/sec

syscalls.sock.sockets = 0.000000/sec

syscalls.sock.connects = 0.000000/sec

syscalls.sock.binds = 0.000000/sec

syscalls.sock.closes = 0.006665/sec

syscalls.sock.reads = 0.006665/sec

syscalls.sock.writes = 0.013331/sec

syscalls.sock.recvfroms = 0.000000/sec

syscalls.sock.sendtos = 0.000000/sec

cpu_time = 1.622753 seconds

wall_time = 300.062942 seconds

cpu_usage = 0.540804%

[root@squid etc]# squidclient -h localhost -p 80 mgr:5min |egrep "client_http.requests|client_http.hits"

client_http.requests = 0.009998/sec

client_http.hits = 0.000000/sec

[root@squid etc]# squidclient -h localhost -p 80 mgr:mem

HTTP/1.0 200 OK

Server: squid/3.0.STABLE20

Mime-Version: 1.0

Date: Mon, 31 Oct 2016 13:55:18 GMT

Content-Type: text/plain

Expires: Mon, 31 Oct 2016 13:55:18 GMT

Last-Modified: Mon, 31 Oct 2016 13:55:18 GMT

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

Current memory usage:

Pool Obj Size Chunks Allocated  In Use Idle Allocations Saved   Hit Rate

  (bytes) KB/ch obj/ch (#) used free part %Frag (#) (KB) high (KB) high (hrs) %Tot (#) (KB) high (KB) high (hrs) %alloc (#) (KB) high (KB) (#) %cnt %vol (#) / sec

Total                  1    0    0    0    0    0    0 -1 0  0 0 4.1e+05 -1 0 0 0 4.1e+05 -1 0  0 0 0 -1 -1 0

Cumulative allocated volume: 0.00 MB

Current overhead: 752 bytes (-1%)

Idle pool limit: 5 MB

Total Pools created: 94

Pools ever used:     94 (shown above)

Currently in use:    0

String Pool Impact

  (%strings) (%volume)

Short Strings       97 97

Medium Strings       0 0

Long Strings         0 0

Other Strings       3 3

Large buffers: 0 (0 KB)

[root@squid etc]# squidclient -h localhost -p 80 mgr:info

HTTP/1.0 200 OK

Server: squid/3.0.STABLE20

Mime-Version: 1.0

Date: Mon, 31 Oct 2016 13:52:19 GMT

Content-Type: text/plain

Expires: Mon, 31 Oct 2016 13:52:19 GMT

Last-Modified: Mon, 31 Oct 2016 13:52:19 GMT

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

Squid Object Cache: Version 3.0.STABLE20

Start Time: Mon, 31 Oct 2016 10:05:52 GMT

Current Time: Mon, 31 Oct 2016 13:52:19 GMT

Connection information for squid:

Number of clients accessing cache: 3

Number of HTTP requests received: 16

Number of ICP messages received: 0

Number of ICP messages sent: 0

Number of queued ICP replies: 0

Number of HTCP messages received: 0

Number of HTCP messages sent: 0

Request failure ratio: 0.00

Average HTTP requests per minute since start: 0.1

Average ICP messages per minute since start: 0.0

Select loop called: 8245010 times, 1.648 ms avg

Cache information for squid:

Hits as % of all requests: 5min: 0.0%, 60min: 0.0%

Hits as % of bytes sent: 5min: 100.0%, 60min: 100.0%

Memory hits as % of hit requests: 5min: 0.0%, 60min: 0.0%

Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0%

Storage Swap size: 16992 KB

Storage Swap capacity: 16.6% used, 83.4% free

Storage Mem size: 296 KB

Storage Mem capacity: 0.3% used, 99.7% free

Mean Object Size: 25.32 KB

Requests given to unlinkd: 3

Median Service Times (seconds)  5 min    60 min:

HTTP Requests (All):   0.00000  0.00000

Cache Misses:          0.00000  0.00000

Cache Hits:            0.00000  0.00000

Near Hits:             0.00000  0.00000

Not-Modified Replies:  0.00000  0.00000

DNS Lookups:           0.00000  0.00000

ICP Queries:           0.00000  0.00000

Resource usage for squid:

UP Time: 13587.130 seconds

CPU Time: 42.919 seconds

CPU Usage: 0.32%

CPU Usage, 5 minute avg: 0.55%

CPU Usage, 60 minute avg: 0.56%

Process Data Segment Size via sbrk(): 3680 KB

Maximum Resident Size: 104608 KB

Page faults with physical i/o: 0

Memory usage for squid via mallinfo():

Total space in arena:    3812 KB

Ordinary blocks:         3673 KB      7 blks

Small blocks:               0 KB      0 blks

Holding blocks:         37528 KB      6 blks

Free Small blocks:          0 KB

Free Ordinary blocks:     138 KB

Total in use:           41201 KB 100%

Total free:               138 KB 0%

Total size:             41340 KB

Memory accounted for:

Total accounted:            0 KB   0%

memPool accounted:          0 KB   0%

memPool unaccounted:    41340 KB 100%

memPoolAlloc calls:         0

memPoolFree calls:          0

File descriptor usage for squid:

Maximum number of file descriptors:   64000

Largest file desc currently in use:     18

Number of file desc currently in use:   13

Files queued for open:                   0

Available number of file descriptors: 63987

Reserved number of file descriptors:   100

Store Disk files open:                   0

Internal Data Structures:

  697 StoreEntries

   27 StoreEntries with MemObjects

   26 Hot Object Cache Items

  671 on-disk objects

查看squid命中率：cat /usr/local/squid/var/logs/access.log |awk ‘{print $4}‘ |sort |uniq -c|sort -nr

[root@squid etc]# cat /usr/local/squid/var/logs/access.log.1 |awk ‘{print $4}‘ |sort |uniq -c|sort -nr

  26854 TCP_MISS/200

    831 TCP_MISS/503

    455 TCP_MISS/304

    390 TCP_MISS/404

    153 TCP_MISS/504

     77 TCP_CLIENT_REFRESH_MISS/200

     50 TCP_DENIED/403

     35 TCP_REFRESH_UNMODIFIED/304

     21 TCP_MISS/302

     20 TCP_MISS/000

     16 TCP_MISS/502

     12 TCP_MISS/206

      9 TCP_MISS/403

      6 TCP_REFRESH_MODIFIED/200

      1 TCP_CLIENT_REFRESH_MISS/000

四、清理squid缓存

1）编辑squid.conf文件，添加如下几行：

acl PURGE method PURGE  #定义一个PURGE方法

http_access allow PURGE localhost   #允许localhost使用PURGE方法

http_access deny PURGE   #禁止其他任何服务器使用PURGE方法

2）重启squid

killall squid

squid -k parse

squid -D

3)使用squidclient命令清除squid缓存

[root@squid etc]# squidclient -m PURGE -p 80 http://192.168.49.135/lnmp.jpg

HTTP/1.0 200 OK

Server: squid/3.0.STABLE20

Mime-Version: 1.0

Date: Mon, 31 Oct 2016 14:23:02 GMT

Content-Length: 0

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

4）测试缓存是否清除成功

[root@squid etc]# curl -I -s http://192.168.49.135/lnmp.jpg

HTTP/1.0 200 OK

Date: Mon, 31 Oct 2016 14:25:33 GMT

Server: Apache/2.2.15 (CentOS)

Last-Modified: Thu, 26 Jun 2014 01:12:31 GMT

ETag: "bf742-2f37a-4fcb2e3149dc0"

Accept-Ranges: bytes

Content-Length: 193402

Cache-Control: max-age=31104000

Expires: Thu, 26 Oct 2017 14:25:33 GMT

Content-Type: image/jpeg

X-Cache: MISS from cache1.contoso.com

Via: 1.0 cache1.contoso.com (squid/3.0.STABLE20)

Connection: close

如果需要删除多个链接，可以使用shell脚本然后将链接整理到一个文件中，然后循环调用squidclient进行删除。

如果要清除全部缓存，可以先关闭squid服务，然后删除或移除cache文件夹，然后再创建cache目录并重新初始化。

squid -k shutdown 

cd /usr/local/squid/var

mv cache oldcache

mkdir cache

chown squid:squid cache

squid -z

squid -s

rm -rf oldcache &

案例补充：

缓存不同步 问题如何解决？

现象：不同地区显示页面不一致

解决：在8台cache服务器手工刷新缓存，由业务部门提供有问题的URL链接。

squidclient -m purge -u squid -g squid http://

本文出自 “” 博客，请务必保留此出处