My current config is Nginx webserver as a reverse proxy with Apache at backend for dynamic pages and Apache is configured with PHP-FPM. My dynamic pages don't change often so I am trying to use Nginx microcaching but can't seem to get working. Here is my nginx.conf (possible unnecessary stuff excluded): http { sendfile on; tcp_nopush on; tcp_nodelay on; client_header_timeout 10s; client_body_timeout 10s; client_max_body_size 200k; client_header_buffer_size 2k; client_body_buffer_size 200k; large_client_header_buffers 3 1k; send_timeout 3m; keepalive_timeout 60 60; reset_timedout_connection on; ignore_invalid_headers on; connection_pool_size 256; request_pool_size 4k; output_buffers 4 32k; postpone_output 1460; include mime.types; default_type application/octet-stream; # Compression gzip gzip on; gzip_vary on; gzip_disable "MSIE [1-6]\."; gzip_proxied any; gzip_min_length 512; gzip_comp_level 6; gzip_buffers 8 64k; gzip_types text/plain text/xml text/css text/js application/x-javascript application/xml image/png image/x-icon image/gif image/jpeg image/svg+xml application/xml+rss text/javascript application/atom+xml application$ # Proxy settings proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; #proxy_pass_header Set-Cookie; proxy_connect_timeout 90; proxy_send_timeout 90; proxy_read_timeout 90; proxy_buffers 32 4k; # SSL Settings ssl_session_cache shared:SSL:10m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA!RC4:EECDH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS"; # Logs log_format bytes '$body_bytes_sent'; #access_log /var/log/nginx/access.log main; access_log off; ## Advanced microcaching ## set cache dir, hierarchy, max key size and total size fastcgi_cache_path /dev/shm/microcache levels=1:2 keys_zone=microcache:50M max_size=2000M inactive=2h; ## set cache log location, so you can evaluate hits log_format cache '$remote_addr - $remote_user [$time_local] "$request" ' '$status $upstream_cache_status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; ## Set a cache_uid variable for authenticated users. map $http_cookie $cache_uid { default nil; ~SESS[[:alnum:]]+=(?<session_id>[[:alnum:]]+) $session_id; } ## mapping cache to request method map $request_method $no_cache { default 1; # by default do not cache HEAD 0; # cache HEAD requests GET 0; # cache GET requests } # Include additional configuration include /etc/nginx/cloudflare.inc; include /etc/nginx/conf.d/*.conf; } Code (markup): My vhost config (relevant stuff only) is: location / { location ~.*\.(3gp|gif|jpg|jpeg|png|ico|wmv|avi|asf|asx|mpg|mpeg|mp4|pls|mp3|mid|wav|swf|flv|html|htm|txt|js|css|exe|zip|tar|rar|gz|tgz|bz2|uha|7z|doc|docx|xls|xlsx|pdf|iso|woff|ttf|svg|eot|sh)$ { root /home/xtrefish/public_html; expires max; try_files $uri @backend; } error_page 405 = @backend; error_page 500 = @custom; add_header X-Cache "HIT from Backend"; add_header Strict-Transport-Security "max-age=31536000"; add_header X-Content-Type-Options nosniff; proxy_pass IPADDRESS:8181; include proxy.inc; } location @backend { internal; proxy_pass IPADDRESS:8181; include proxy.inc; } location @custom { internal; proxy_pass IPADDRESS:8181; include proxy.inc; location @custom { internal; proxy_pass IPADDRESS:8181; include proxy.inc; } location ~ .*\.(php|jsp|cgi|pl|py)?$ { ## Caching stuff is here # Setup var defaults set $no_cache ""; # If non GET/HEAD, don't cache & mark user as uncacheable for 1 second via cookie if ($request_method !~ ^(GET|HEAD)$) { set $no_cache "1"; } # Drop no cache cookie if need be # (for some reason, add_header fails if included in prior if-block) if ($no_cache = "1") { add_header Set-Cookie "_mcnc=1; Max-Age=2; Path=/"; add_header X-Microcachable "0"; } # Bypass cache if no-cache cookie is set if ($http_cookie ~* "_mcnc") { set $no_cache "1"; } # Bypass cache if flag is set fastcgi_no_cache $no_cache; fastcgi_cache_bypass $no_cache; fastcgi_cache microcache; fastcgi_cache_key $scheme$host$request_uri$request_method; fastcgi_cache_valid 200 301 302 10m; fastcgi_cache_use_stale updating error timeout invalid_header http_500; fastcgi_pass_header Set-Cookie; fastcgi_pass_header Cookie; fastcgi_ignore_headers Cache-Control Expires Set-Cookie; proxy_pass IPADDRESS:8181; include proxy.inc; } Code (markup): I can confirm I am using PHP-FPM as info.php responds Server API FPM/FastCGI Code (markup): To test this, I have a microcache_test.php containing <?php /** * Display microtime * Requires PHP 5, preferably PHP 5.4+ * This function is only available on operating systems that * support the gettimeofday() system call. * * Returns float in format seconds.microseconds */ echo ' Current microtime: ' . microtime(true); Code (markup): And /microcache_test.php responds: Request URL: https://example.com/microcache_test.php Request Method: GET Status Code: 200 Remote Address: 82.102.16.22:443 Referrer Policy: no-referrer-when-downgrade cache-control: public content-encoding: gzip content-type: text/html; charset=UTF-8 date: Tue, 19 Feb 2019 09:13:58 GMT server: nginx/1.15.8 status: 200 vary: Accept-Encoding :authority: example.com :method: GET :path: /mictest.php :scheme: https accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8 accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9 cache-control: max-age=0 cookie: PHPSESSID=864e0ajkc6l1rbhepr651dp1t2 save-data: on upgrade-insecure-requests: 1 user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36 Code (markup): So caching is certainly not working. My caching directories are 777 and I tried setting cache in htaccess in apache as well but to no avail (Nginx is set to ignore the headers anyway). Any ideas what am I missing?
Your test file is /microcache_test.php but the request is for /mictest.php. I am guessing they are the same files. Your configs are hard to read. Put it inside code. You are trying bypass caching on:- POST request - when you are updating something, you would obviously want the modified(which is not cached content) If _mcnc cookie(2 seconds expiry) is found My question would be, does your POST request redirect to a GET request of the same page or does POST request directly render the modified page? If POST redirects to GET, then you need to set cookie to bypass caching. Just want to confirm if this is the case. Assuming POST redirects to GET to fetch new contents. set $no_cache "0" ; if ( $request_method = POST ) { add_header Set-Cookie "_mcnc=1; Max-Age=3; Path=/"; } # Bypass cache if no-cache cookie is set if ($http_cookie ~* "_mcnc") { set $no_cache "1"; } Code (ApacheConf): If POST method also renders content then you dont need to maintain a cookie. set $no_cache "0"; if ( $request_method = POST ) { set $no_cache "1"; } Code (ApacheConf): If the above is not relevant, then start stripping conditions and enable caching for all requests then add condition for cache bypass use httpie to troubleshoot. Showing header values are irrelevant for readers try changing configs one by one and see results on httpie and do it methodically
Thanks for your response. Actually I would also like to cache POST along with GET (for now at least). Can you suggest caching parameters that will just microcache everything regardless of cookie or cache-headers?
First remove all ifs and start with minimal approach I have never used dynamic content caching. You will have to research on own. This link has some info to start from scratch
Make sure you have this cache configuration in nginx config.. proxy_cache_path /tmp/cache levels=1:2 keys_zone=default_cache:10m max_size=2g inactive=120m use_temp_path=off; proxy_cache_key "$scheme$request_method$host$request_uri"; proxy_cache_valid 200 302 60m;