Setting cache-control or expires with nginx static only app

ngkabra

I would like to use the 'cache-control' or 'expires' headers for the static pages on my django app. (Ideally, I want to implement something equivalent to "expires max".)

How can I do that on opalstack? Thanks.

sean

ngkabra custom options for static apps aren't available yet, but it's on our TODO list 🙂

Edit: done, for all apps!

stephenmwatts

Is this still a TODO? I would like to do the same. Google PageSpeed Insights lists this as a diagnostic issue on my pages.

sean

stephenmwatts yep, still on the list but we hope to have it ready soon.

anjanesh

Can I set Cache TTL to images in a static app ?

sean

anjanesh custom options for static apps aren't available yet, but it's on our TODO list 🙂

anjanesh

I brought this up only because Page Insights recommended for improved page performance.

tomhosking

Is there any news on this? I'm also getting this suggestion from Page Insights.

klynton

Hey,

It's still on our list but not currently fixed. We'll make sure to post here when it is fixed.

stephenmwatts

This is a bit round-about, but I am having trouble on Google Search Console with Mobile Usability. My site passes their live test, but fails the crawl. When I check to see if they have clues, what I see is that Google isn't loading all of my resources. This appears to be the root cause of the mobility issues - at least that's what people on other forums think. I was thinking about using AWS for my static and media, and that perhaps that would fix the issue - but then my mind came back to THIS THREAD. I'm wondering if being able to set cache policy on my server for serving static files might help, because Google Experts on their forum report that the issue is that the Google Search Console crawl is running out of Crawl Budget - hence, this issue. I don't know for sure if this would fix it, but it seems possible. Status? Opinion?

sean

stephenmwatts no progress on this yet, sorry.

If you'd like to test your theory:

Install a private nginx stack pointing at your existing static app directory.
Configure your nginx.conf with the cache settings you want to use.
Restart the stack.
Edit your site in your Opalstack dashboard to switch it over to your stack instead of the static app.

igor

stephenmwatts Had the same issue on PHP Apache app, caching headers did not help at all. I resolved this by using CloudFlare's caching. Seemed like the server (Opal5) is too slow when serving a lot of static files.

stephenmwatts

I haven't done much work on setting up servers - spoiled first by Webfaction and now Opalstack. While I'm mulling that over, can you tell me what the default settings are for Cache Policy that our servers use?

I guess you are also saying "Well, it's possible."

sean

stephenmwatts there are no default cache settings configured for any app types, including Nginx static-only apps.

In other apps it's possible to set them to whatever you need via the headers that your application returns, but this isn't possible for static apps.

We still plan to make some options available but for now the private stack is the best approach in my opinion. I'll update that procedure to include some examples for configuring the cache settings in the private Nginx config.

stephenmwatts

sean Thanks. I use Django. I've started looking over the suggestions you shared. Do you have anything directed toward this with a Django system, or should I just attempt to "translate" myself? The timing of the Mobilie Browsing Issues does seem to correlate with a significant hit to my traffic, so it's worth my time to see if this fixes the issue. I'd say they must've recently decided to more aggressively throttle their crawl budget - assuming, of course, that this is the problem.

sean

stephenmwatts Django has a cache control decorator that you can use to set cache headers on views.

If you're using our uwsgi setup, then you can use addheader with uwsgi's internal routing feature to add whatever cache headers you need to whichever URL patterns.

stephenmwatts

sean Okay - that sounds more promising than me trying to figure out how to translate the instructions from PHP to Python. I'll dig in to that. Any links for configuring your uwsgi system settings or does that all just happen in uwsgi.ini?

What I think you are saying, though, is that I can use this to add cache headers for static (static and media) content. If not, please let me know. I do use cache for my other content.

sean

stephenmwatts the django and uwsgi stuff won’t cache anything served by your static apps. It will cache the content served by your django app.

I’ll have the Nginx examples done soon.

stephenmwatts

sean Thanks again. It likely won't be tonight, but I will dig into it and see if I can make it happen and, if I manage it I will be sure to provide feedback here. As you can tell, it likely won't be the last time you hear from me on the topic. :-/ I greatly appreciate your timely assistance!

sean

To add some basic asset caching to a private Nginx stack, add the following within the location / block of your nginx.conf:

location ~* \.(js|css|jpg|jpeg|png|gif|js|css|ico|woff|woff2|svg)$ {
                expires 1y;
                etag off;
                if_modified_since off;
                add_header Cache-Control "public, no-transform";
}

If you want to cache everything without regard to the content type, then don't wrap the config in its own location block eg:

expires 1y;
etag off;
if_modified_since off;
add_header Cache-Control "public, no-transform";

stephenmwatts

sean I changed my mind a bit. I have a private nginx stack set up, and am trying to modify the config file to add appropriate header for cache and, hopefully, gzip to the nginx config file. Best I can tell, the files I load up with my browser still don't serve up with the right headers. Can you have a quick look at the setup for static_media_files_pvt_stk and see if I have things setup correctly? I am more interested in cache than gzip if it comes down to that.

sean

igor currently our Apache setup is not great. We're using mpm-itk to secure Apache in our multi-tenant environment, and unfortunately that setup has to spawn a separate httpd process for every single request.

This means, for example, if you have a page that loads 10 images, 2 JS files, and a CSS file the server has to spawn 14 httpd processes to serve the page and all of its resources.

We're working to improve the Apache setup and hope to have some changes rolled out soon.

stephenmwatts

igor That's what I'm hoping. I just finished a CloudFront implementation last night and submitted a request for verification on the failure. Jury is still out, but it's good to hear I MAY HAVE BEEN on the right track.

stephenmwatts

igor I have done something similar with AWS CloudFlare CDN. My Google Verification for Mobile issues still fails because (I believe) several of my resources are not loading due to the fact that there is no "cache-control: max-age<seconds>" header available to the browser. Are you pulling your resources from OpalStack as the origin, or did you set up an origin somewhere that actually handles cache control? If you are pulling from OpalStack, how did you accomplish adding max-age or expires? I can't figure out if I can do it on CloudFlare, and if so, how... I do think I can accomplish it if I move my static resources to S3 and make that the origin, but it would be simpler just to have the CDN add appropriate cache headers if it can.
If you can offer any assistance, I would appreciate it.