首页 > 其他 > 详细

varnish

时间:2014-05-13 13:12:46      阅读:821      评论:0      收藏:0      [点我收藏+]

varnish
1、缓存相关的HTTP首部

HTTP协议提供了多个首部用以实现页面缓存及缓存失效的相关功能,这其中最常用的有:
(1)Expires:用于指定某web对象的过期日期/时间,通常为GMT格式;一般不应该将此设定的未来过长的时间,一年的长度对大多场景来说足矣;其常用于为纯静态内容如JavaScripts样式表或图片指定缓存周期;
(2)Cache-Control:用于定义所有的缓存机制都必须遵循的缓存指示,这些指示是一些特定的指令,包括public、private、no-cache(表示可以存储,但在重新验正其有效性之前不能用于响应客户端请求)、no-store、max-age、s-maxage以及must-revalidate等;Cache-Control中设定的时间会覆盖Expires中指定的时间;
(3)Etag:响应首部,用于在响应报文中为某web资源定义版本标识符;
(4)Last-Mofified:响应首部,用于回应客户端关于Last-Modified-Since或If-None-Match首部的请求,以通知客户端其请求的web对象最近的修改时间;
(5)If-Modified-Since:条件式请求首部,如果在此首部指定的时间后其请求的web内容发生了更改,则服务器响应更改后的内容,否则,则响应304(not modified);
(6)If-None-Match:条件式请求首部;web服务器为某web内容定义了Etag首部,客户端请求时能获取并保存这个首部的值(即标签);而后在后续的请求中会通过If-None-Match首部附加其认可的标签列表并让服务器端检验其原始内容是否有可以与此列表中的某标签匹配的标签;如果有,则响应304,否则,则返回原始内容;
(7)Vary:响应首部,原始服务器根据请求来源的不同响应的可能会有所不同的首部,最常用的是Vary: Accept-Encoding,用于通知缓存机制其内容看起来可能不同于用户请求时Accept-Encoding-header首部标识的编码格式;
(8)Age:缓存服务器可以发送的一个额外的响应首部,用于指定响应的有效期限;浏览器通常根据此首部决定内容的缓存时长;如果响应报文首部还使用了max-age指令,那么缓存的有效时长为“max-age减去Age”的结果;
varnish各状态引擎

VCL的函数不接受参数并且没有返回值,因此,其并非真正意义上的函数,这也限定了VCL内部的数据传递只能隐藏在HTTP首部内部进行。VCL的return语句用于将控制权从VCL状态引擎返回给Varnish,而非默认函数,这就是为什么VCL只有终止语句而没有返回值的原因。同时,对于每个“域”来说,可以定义一个或多个终止语句,以告诉Varnish下一步采取何种操作,如查询缓存或不查询缓存等

vcl_recv是在Varnish完成对请求报文的解码为基本数据结构后第一个要执行的子例程,它通常有四个主要用途:
(1)修改客户端数据以减少缓存对象差异性;比如删除URL中的www.等字符;
(2)基于客户端数据选用缓存策略;比如仅缓存特定的URL请求、不缓存POST请求等;
(3)为某web应用程序执行URL重写规则;
(4)挑选合适的后端Web服务器;

相对于vcl_recv是根据客户端的请求作出缓存决策来说,vcl_fetch则是根据服务器端的响应作出缓存决策。在任何VCL状态引擎中返回的pass操作都将由vcl_fetch进行后续处理。vcl_fetch中有许多可用的内置变量,比如最常用的用于定义某对象缓存时长的beresp.ttl变量

bubuko.com,布布扣


实验图

bubuko.com,布布扣

1,配置haproxy
frontend webs
 bind :80
   acl url_static       path_beg       -i /static /images /javascript /stylesheets
   acl url_static       path_end       -i .jpg .gif .png .css .js
   acl url_static       hdr_reg(host) -i ^img.* ^css.*

   use_backend static          if url_static
   default_backend             app

backend static
   balance     roundrobin
   server      static 192.168.100.200:80 check

backend app
        cookie SESSION_COOKIE insert  nocache
   balance     roundrobin
   server  app1 node1.xy.com:8080 cookie node1
   server  app1 node2.xy.com:80    cookie node2

2,配置httpd
Listen 8080
Listen 80
<VirtualHost *:8080>
   ServerAdmin webmaster@dummy-host.example.com
   DocumentRoot /var/www/node1
   #ServerName node1.xy.com
   ErrorLog logs/dummy-host.example.com-error_log
   CustomLog logs/dummy-host.example.com-access_log common
</VirtualHost>
<VirtualHost *:80>
   ServerAdmin webmaster@dummy-host.example.com
   DocumentRoot /var/www/node2
   #ServerName node2.xy.com
   ErrorLog logs/dummy-host.example.com-error_log
   CustomLog logs/dummy-host.example.com-access_log common
</VirtualHost>

准备测试页
# cd /var/www/node1
<h1>node1.xy.com</h1>
<img src="http://img.xy.com/1.jpg"></img>
<img src="http://css.xy.com/2.jpg"></img>
<img src="http://img.xy.com/images/1.jpg"></img>

# cd /var/www/node2
<h1>node2.xy.com</h1>
<img src="http://img.xy.com/1.jpg"></img>
<img src="http://css.xy.com/2.jpg"></img>
<img src="http://img.xy.com/images/1.jpg"></img>

按照测试页准备文件
img1.xy.com/192.168.100.26
[root@img1 html]# find
.
./images
./images/1.jpg
./status.html
./1.jpg
service httpd start

css1.xy.com/192.168.100.27
[root@css1 www]# find
.
./status.html
./2.jpg
service httpd start

3,配置varnish
安装varnish
# yum -y install varnish

修改启动脚本配置参数
VARNISH_LISTEN_PORT=80  监听端口
VARNISH_STORAGE="malloc,100m"  缓存大小

启动varnish
# service varnish start

提供一个vcl规则文件
# vim /etc/varnish/test.vcl

probe healthcheck {
   .url = "/status.html";
   .interval = 60s;
   .timeout = 3 s;
   .window = 8;
   .threshold = 3;
   .initial = 3;
   .expected_response = 200;
}
backend img1 {
  .host = "192.168.100.25";
  .port = "80";
  .probe = healthcheck;
}
backend img2 {
  .host = "192.168.100.26";
  .port = "80";
  .probe = healthcheck;
}
director imgs random {
    {.backend = img1; .weight=5; }
    {.backend = img2; .weight=6; }
}
backend css1 {
  .host = "192.168.100.27";
  .port = "80";
  .probe = healthcheck;
}
backend css2 {
  .host = "192.168.100.28";
  .port = "80";
  .probe = healthcheck;
}
director css random {
    {.backend = css1; .weight=5; }
    {.backend = css2; .weight=5; }
}
acl purgers {
  "localhost";
  "127.0.0.1";
  "192.168.100.0"/24;
}
sub vcl_hash {
     hash_data(req.url);
     if (req.http.host) {
         hash_data(req.http.host);
     } else {
         hash_data(server.ip);
     }
     return (hash);
 }
sub vcl_recv {
    if (req.http.referer ~ "http://.*") {
        if ( !(req.http.referer ~ "http://.*xy\.com"
            || req.http.referer ~ "http://.*google\.com"
            || req.http.referer ~ "http://.*google\.cn"
            || req.http.referer ~ "http://.*baidu\.com"
            || req.http.referer ~ "http://.*yahoo\.cn"
            )) {
                   error 200 "ok.";
            #set req.http.host = "www.xy.com";
            #set req.http.url = "/temp/error.jpg";
            }
    }
    if (req.http.x-forwarded-for) {
      set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
    } else {
      set req.http.X-Forwarded-For = client.ip;
    }
    if (req.request != "GET" &&
    req.request != "HEAD" &&
    req.request != "PUT" &&
    req.request != "POST" &&
    req.request != "TRACE" &&
    req.request != "OPTIONS" &&
    req.request != "DELETE") {
    return (pipe);
  }
  if (req.request != "GET" && req.request != "HEAD") {
    return (pass);
  }
    if (req.http.host ~ "^img.*\.xy\.com") {
        set req.backend = imgs;
    }
    if (req.http.host ~ "^css.*\.xy\.com") {
        set req.backend = css;
    }
    if (req.request == "PURGE") {
        if (!client.ip ~ purgers) {
            error 405 "Method not allowed";
        }
        return (lookup);
    }
}
sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged";
    }
}
sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 404 "Not in cache";
    }
}
sub vcl_pass {
    if (req.request == "PURGE") {
        error 502 "PURGE on a passed object";
    }
}
sub vcl_fetch {
  set beresp.ttl = 3600s;
  set beresp.http.expires = beresp.ttl;
  if ( req.http.host ~ "^video.*\.xy\.com" ) {  
    set beresp.ttl = 1d;
  }
  if ( req.http.host ~ "^img.*\.xy.com" ) {
     set beresp.ttl = 12h;
  }
  if ( req.http.host ~ "^css.*\.xy.com" ) {
     set beresp.ttl = 4h;
  }
  if (beresp.status != 200){
    return (hit_for_pass);
  }
  return (deliver);
}
sub vcl_deliver {
  if (obj.hits > 0){
    set resp.http.X-Cache = "HIT from " + server.hostname;
  }
  else {
    set resp.http.X-Cache = "MISS from " + server.hostname;
  }
 }

启用vcl文件
# varnishadm
varnish> vcl.load test3 test.vcl
200        
VCL compiled.
varnish> vcl.use test3
200        

查看命中
[root@node1 ~]# varnishstat -1
client_conn                376         0.04 Client connections accepted
client_drop                  0         0.00 Connection dropped, no sess/wrk
client_req                 356         0.03 Client requests received
cache_hit                  260         0.02 Cache hits
cache_hitpass                3         0.00 Cache hits for pass
cache_miss                  47         0.00 Cache misses

查看日志
# varnishlog
10 RxHeader     c Referer: http://node.xy.com/index.html
  10 RxHeader     c Accept-Encoding: gzip,deflate,sdch
  10 RxHeader     c Accept-Language: zh-CN,zh;q=0.8
  10 RxHeader     c X-Forwarded-For: 192.168.100.100

  0 Backend_health - css2 Still sick ------- 0 3 8 0.000000 0.000000
   0 Backend_health - img1 Still sick ------- 0 3 8 0.000000 0.000000
   0 Backend_health - img2 Still healthy 4--X-RH 8 3 8 0.001933 0.001966 HTTP/1.1 200 OK
   0 Backend_health - css1 Still healthy 4--X-RH 8 3 8 0.001992 0.002131 HTTP/1.1 200 OK
   0 Backend_health - css2 Still sick ------- 0 3 8 0.000000 0.000000
   0 Backend_health - img1 Still sick ------- 0 3 8 0.000000 0.000000
   0 Backend_health - img2 Still healthy 4--X-RH 8 3 8 0.001961 0.001965 HTTP/1.1 200 OK
   0 Backend_health - css1 Still healthy 4--X-RH 8 3 8 0.002249 0.002161 HTTP/1.1 200 OK

测试防盗链


bubuko.com,布布扣


bubuko.com,布布扣


图片分布式


bubuko.com,布布扣

本文出自 “小雨” 博客,请务必保留此出处http://evildexy.blog.51cto.com/8607019/1410043

varnish,布布扣,bubuko.com

varnish

原文:http://evildexy.blog.51cto.com/8607019/1410043

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!