Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
PostPosted: Wed Nov 24, 2010 4:44 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
Hi

I've asked this on irc and on nginx forums, but haven't had any replies. I'm trying to set up fastcgi_cache. Cached hits are served as 0kb image files.

nginx.conf
Code:
user www-data;
worker_processes 4;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
  worker_connections  1024;
}

http {
  server_names_hash_max_size 512;
  server_names_hash_bucket_size 128;

  index index.php index.html index.htm;

  include       mime.types;
  default_type  application/octet-stream;

  fastcgi_cache_path  /var/cache/nginx  levels=1:2 keys_zone=WORDPRESS:10m inactive=5m;

  access_log /var/log/nginx/access.log;
  sendfile   on;

  keepalive_timeout  65;

  gzip  on;
  gzip_disable "MSIE [1-6]\.(?!.*SV1)";

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}


mysite.com.conf
Code:
server {
  listen      1.2.3.4:80 default;
  server_name www.mysite.com;

  error_log  /var/log/nginx/www.mysite.com-error.log;
  access_log /var/log/nginx/www.mysite.com-access.log;

  root  /home/graq/sites/www.mysite.com;

 
  location / {
    try_files $uri $uri/ /index.php?q=$uri&$args;
  }

  location ~ \.php$ {
    set $wordpress_logged_in  "";
    set $comment_author_email "";
    set $comment_author       "";

    if ($http_cookie ~* "wordpress_logged_in_[^=]*=([^%]+)%7C") {
      set $wordpress_logged_in wordpress_logged_in_$1;
    }

    if ($http_cookie ~* "comment_author_email_[^=]*=([^;]+)(;|$)") {
      set $comment_author_email comment_author_email_$1;
    }

    if ($http_cookie ~* "comment_author_[^=]*=([^;]+)(;|$)") {
      set $comment_author comment_author_$1;
    }

    set $my_cache_key "$scheme://$host$uri$is_args$args$wordpress_logged_in$comment_author_email$comment_author";

    fastcgi_pass_header     Set-Cookie;
    fastcgi_cache_use_stale error timeout invalid_header http_500;
    fastcgi_cache_key       $my_cache_key;
    fastcgi_cache           WORDPRESS;
    fastcgi_cache_valid     200 1m;

    fastcgi_pass  unix:/usr/local/var/run/php-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO       $fastcgi_script_name;
    include fastcgi_params;
  }
}


Last edited by graq on Thu Dec 02, 2010 1:27 pm, edited 1 time in total.

Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 11:46 am 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
What are you trying to accomplish with all that stuff? Caching only for non-logged-in users? Separate cache for each logged-in user?

What version of nginx are you using? The newest versions have new options that might make much of that stuff unnecessary. Would you consider upgrading to the latest?

That capture ($1) in the set directives looks like it'd add a great deal of junk.

Please explain, thanks.


Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 11:55 am 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
And what is your Wordpress application sending for cache headers? Nginx will generally obey the cache headers Cache-Control and Expires from the backend.

So you either need to have something like this: http://wordpress.org/extend/plugins/ngi ... ntegrator/ (I haven't used that, but I made a small drupal module that would send the X-Accel-Expires=0 from the backend to keep nginx from caching logged in pages) and also do fastcgi_ignore_headers Cache-Control Expires; so that the nginx cache doesn't obey those.

Or instead of all that junk, use 0.8.46 or later that have the new directives fastcgi_cache_bypass and fastcgi_no_cache.


Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 3:13 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
Brian, you may just be about to become my hero.

brianmercer wrote:
What are you trying to accomplish with all that stuff?
....
Please explain, thanks.


I am trying to add (PHP) fastcgi caching to WordPress (MU if possible, but am willing to take small steps). Think of me as a student, ready for your teaching :) Feel free to ignore any ignorant rubbish from previous post.

Code:
nginx version: nginx/0.8.53
built by gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
TLS SNI support enabled
configure arguments: --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/body --http-proxy-temp-path=/var/lib/nginx/proxy --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --with-debug --with-http_stub_status_module --with-http_flv_module --with-http_ssl_module --with-http_dav_module --with-http_gzip_static_module --with-http_realip_module --with-mail --with-mail_ssl_module --with-ipv6 --add-module=/root/SERVER_BUILD/nginx-0.8.53/gnosek-nginx-upstream-fair-2131c73

Although I am a little geeky, and understand caching, some issues with caching (like dogpiling) - I'm more of a developer than a sysadmin and the nginx-fu hasn't quite sunk in with me yet.

I appreciate that caching the wp-admin pages is not a great idea (but again, small steps and all that).

This the non-cache nginx server{} WP config I am currently using on my non default sites:
Code:
server { 
  listen      1.2.3.4:80;
  server_name www.mysite.com;

  error_log  /var/log/nginx/www.mysite.com-error.log;
  access_log /var/log/nginx/www.mysite.com-access.log;

  root  /home/graq/sites/www.mysite.com;
 
  location / {
    try_files $uri $uri/ /index.php?q=$uri&$args;
  }

  location ~ \.php$ {
    fastcgi_pass  unix:/usr/local/var/run/php-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO       $fastcgi_script_name;
    include fastcgi_params;
  }
}

In case you want to know about PHP
Code:
phpinfo()
PHP Version => 5.3.3

System => Linux paprika 2.6.32.16-linode28 #1 SMP Sun Jul 25 21:32:42 UTC 2010 i686
Build Date => Nov 16 2010 12:48:36
Configure Command =>  './configure'  '--with-config-file-path=/usr/local/lib/php' '--with-curl' '--enable-exif' '--with-gd' '--with-jpeg-dir' '--with-png-dir' '--with-zlib' '--with-xpm-dir' '--with-freetype-dir' '--with-t1lib' '--with-mcrypt' '--with-mhash' '--with-mysql=mysqlnd' '--with-mysqli=mysqlnd' '--with-pdo-mysql=mysqlnd' '--with-openssl' '--enable-sysvmsg' '--enable-wddx' '--with-xsl' '--enable-zip' '--with-bz2' '--enable-bcmath' '--enable-calendar' '--enable-ftp' '--enable-mbstring' '--enable-soap' '--enable-sockets' '--enable-sqlite-utf8' '--with-gettext' '--enable-shmop' '--with-xmlrpc' '--enable-dba' '--enable-sysvsem' '--enable-sysvshm' '--enable-fpm' '--with-fpm-user=www-data' '--with-fpm-group=www-data'


Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 4:25 pm 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
Give this a try:
Code:
  location ~ \.php$ {

    set $nocache "";
    if ($http_cookie ~ (comment_author_.*|wordpress_logged_in.*|wp-postpass_.*)) {
      set $nocache "Y";
    }

    fastcgi_pass  unix:/usr/local/var/run/php-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;

    fastcgi_cache_use_stale error timeout invalid_header http_500;
    fastcgi_cache_key $host$request_uri;
    fastcgi_cache  WORDPRESS;
    fastcgi_cache_valid 200 1m;
    fastcgi_cache_bypass $nocache;
    fastcgi_no_cache $nocache;
  }


Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 5:15 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
:D It is a thing of beauty. Thank you. :mrgreen:

I have a few questions, if you know the answers:
1. It looks like it will work quite well with WordPressMU as well? I can supply my current WPMU if it needs tweaking.
2. The WP Plugin you mentioned earlier, 'just' adds header("X-Accel-Expires: 0"); if the cookie matches preg_match('/wordpress_(?!test_cookie)|comment_author|wp-postpass/', $key) . Which is already covered in your nginx snippet. However is there a header I can send in WP to trigger an update in nginx's cache? I'm imagining a site where all non-admin pages are cached, but (in a similar fashion to the 'edit page' option that often appears for admins) a link is added for admins to refresh the cache on an individual page. Plus there is probably a hook into the WP post update.
3. I liked the idea of playing with a cache key that allows for non-admin to get their own cached pages (e.g. subsribers/commentors). Is this a somewhat trivial exercise?

(sorry, I forgot a question I've been meaning to ask).

4. I thought the pages were cached into effectively static html? If that is the case, why is the PHP still run (see I told I didn't actually know what fastcgi_cache did)?


Top
   
 Post subject:
PostPosted: Thu Nov 25, 2010 6:13 pm 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
1. Since the $host is part of the key, it probably will. Test it out, let us know, we'll adjust as needed. I tested an nginx config for WP3.0 multisite but never used it for anything.

2. Yeah, that old module shouldn't be necessary with the 0.8.46+ directives.

There is a new module: http://wordpress.org/extend/plugins/ngi ... che-purge/ which requires nginx to be compiled with the cache purge contrib module. Selectively purging updated posts and the front page and feeds ...that's definitely the next step, but I haven't tried it.

3. Dunno how practical that is. If one person makes a comment, do you want to save every page they visit afterward in case that same person revisits that exact page? Doesn't sound worthwhile. I can understand trying to cache portions of a site for all commenters, but not on an individual basis.

4. The pages are cached as static files in /var/cache/nginx and you can go view them. The filenames are an md5 hash of the cache_key. If someone visits the same page and the cached page is still valid, the request is not passed to the php backend.


Top
   
 Post subject:
PostPosted: Fri Nov 26, 2010 5:31 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
brianmercer wrote:
1. Since the $host is part of the key, it probably will. Test it out, let us know, we'll adjust as needed. I tested an nginx config for WP3.0 multisite but never used it for anything.
I've got WPMU (dir based) set up as follows:
Code:
server{
  .. as above..
 
  ## MU settings - starts
  server_name_in_redirect off;
  port_in_redirect off;

  rewrite ^.*/files/(.*) /wp-includes/ms-files.php?file=$1;
  if (!-e $request_filename) {
   rewrite ^.+?(/wp-.*) $1 last;
   rewrite ^.+?(/.*\.php)$ $1 last;
   rewrite ^ /index.php last;
  }
  ## MU settings - ends

  location / {
    try_files $uri $uri/ /index.php?q=$uri&$args;
  }

  location ~ \.php$ {
    set $nocache "";
    if ($http_cookie ~ (comment_author_.*|wordpress_logged_in.*|wp-postpass_.*)) {
      set $nocache "Y";
    }

    fastcgi_pass  unix:/usr/local/var/run/php-fpm.sock;
    fastcgi_index index.php;
    fastcgi_param CONTENT-LENGTH  $content_length;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO       $fastcgi_script_name;
    include fastcgi_params;

    fastcgi_cache_use_stale error timeout invalid_header http_500;
    fastcgi_cache_key       $host$request_uri;
    fastcgi_cache           WORDPRESS;
    fastcgi_cache_valid     200 2m;
    fastcgi_cache_bypass    $nocache;
    fastcgi_no_cache        $nocache;
  }
}
What I'm especially liking about this is that I could 'export' the whole .php section into an include file and reduce the size of the site config. Can be included by both standard and MU.

brianmercer wrote:
2. Yeah, that old module shouldn't be necessary with the 0.8.46+ directives.

There is a new module: http://wordpress.org/extend/plugins/ngi ... che-purge/ which requires nginx to be compiled with the cache purge contrib module. Selectively purging updated posts and the front page and feeds ...that's definitely the next step, but I haven't tried it.
I may just try this. Hopefully soon. Will post a follow up when I do.

brianmercer wrote:
3. Dunno how practical that is. If one person makes a comment, do you want to save every page they visit afterward in case that same person revisits that exact page? Doesn't sound worthwhile. I can understand trying to cache portions of a site for all commenters, but not on an individual basis.
Yeah.. I think you are right.

brianmercer wrote:
4. The pages are cached as static files in /var/cache/nginx and you can go view them. The filenames are an md5 hash of the cache_key. If someone visits the same page and the cached page is still valid, the request is not passed to the php backend.
What confused me was my own ignorance. When I first played with fastcgi_cache I stuck some error_log('hello'); statements in the WP theme files to determine whether I was getting cached hits or not. I now realise just how naive and wrong that is.

Thanks for your help Brian. Really appreciate it.


Top
   
 Post subject:
PostPosted: Sat Nov 27, 2010 4:49 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
I am confused again.

I have added this configuration to 3 server{} blocks (2 standard WP 1 MU). Only the initial (default) server{} is caching.

Is this something that only works for 1 http{} block?


Top
   
 Post subject:
PostPosted: Sat Nov 27, 2010 8:49 pm 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
Hmmm...not sure why that would happen. Generally, you only have one http block. One cache should be enough for multiple server blocks.

I assume you have the fastcgi_cache_path defined at the http level in nginx.conf. Probably won't work anywhere else. And that you're using the same WORDPRESS zone for each server, and including the $host in the key to avoid collisions.

Can't think of anything. Paste the configs again if you like.


Top
   
 Post subject:
PostPosted: Sun Nov 28, 2010 4:47 am 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
I am an idiot. Complete plum. The sites not caching have plugins installed that filter on this WP core function:
Code:
function wp_get_nocache_headers() {
        $headers = array(
                'Expires' => 'Wed, 11 Jan 1984 05:00:00 GMT',
                'Last-Modified' => gmdate( 'D, d M Y H:i:s' ) . ' GMT',
                'Cache-Control' => 'no-cache, must-revalidate, max-age=0',
                'Pragma' => 'no-cache',
        );

        if ( function_exists('apply_filters') ) {
                $headers = apply_filters('nocache_headers', $headers);
        }
        return $headers;
}


Doing a wget -S http://www.site2.com/ clearly shows
Quote:
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Server: nginx/0.8.53
Date: Sun, 28 Nov 2010 08:30:04 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
X-Powered-By: PHP/5.3.3
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Last-Modified: Sun, 28 Nov 2010 08:30:04 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Pragma: no-cache
This seems like a massive reason for nginx not to cache my requests.

[*SLAP*]

Let me look into that before I go off half-cocked again.


Top
   
 Post subject:
PostPosted: Sun Nov 28, 2010 7:48 am 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
Good , good.
Code:
fastcgi_ignore_headers Expires Pragma Cache-Control


Top
   
 Post subject:
PostPosted: Sun Nov 28, 2010 1:48 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
Thank you! :D
brianmercer wrote:
Good , good.
Code:
fastcgi_ignore_headers Expires Pragma Cache-Control
From the docs (http://wiki.nginx.org/HttpFcgiModule#fastcgi_ignore_headers)
Code:
fastcgi_ignore_headers
syntax: fastcgi_ignore_headers name [name...]

context: http, server, location

This directive forbids processing of the named headers from the FastCGI-server reply. It is possible to specify headers like "X-Accel-Redirect", "X-Accel-Expires", "Expires" or "Cache-Control".

So Pragma is not valid. But that detracts nothing from your point. All three trial sites up and running with caching.

Nginx+fastcgi+PHP really does build a super webserver.


Top
   
 Post subject:
PostPosted: Sun Nov 28, 2010 2:22 pm 
Offline
Senior Member

Joined: Sun Feb 21, 2010 5:12 pm
Posts: 64
You are correct.
Code:
fastcgi_ignore_headers Expires Cache-Control;

should be enough. Apparently nginx ignores pragma. http://forum.nginx.org/read.php?2,2182,2183#msg-2183


Top
   
 Post subject:
PostPosted: Thu Dec 02, 2010 1:26 pm 
Offline
Senior Member

Joined: Sat Nov 13, 2010 3:05 am
Posts: 91
Website: http://www.graq.co.uk
brianmercer wrote:
There is a new module: http://wordpress.org/extend/plugins/ngi ... che-purge/ which requires nginx to be compiled with the cache purge contrib module. Selectively purging updated posts and the front page and feeds ...that's definitely the next step, but I haven't tried it.

http://labs.frickle.com/nginx_ngx_cache_purge/
Code:
  location ~ /purge(/.*) {
    access_log off;
    allow               127.0.0.1;
    deny                all;
    fastcgi_cache_purge WORDPRESS $host$1;
  }
I added the nginx module and installed the WP plugin. Works like a treat.


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group