Forums

PythonAnywhere's web server not configure to handle compressed content

Hi, Are there any reason that PythonAnywhere's web server don't handle compressed content, as pointed in https://developers.google.com/speed/docs/best-practices/payload#GzipCompression?

Thanks

I'm not sure what you mean - are you talking about gzipped response bodies? If so, I would say it's up to the application to support it - the Python webserver is a simple front-end to your WSGI application.

For example, I have a little test page which will be delivered via Gzip compression provided that your browser sets a correct Accept-Encoding header. This doesn't need any support from the PAW servers, I just modified my Flask app to do the Gzip compression and set the appropriate header. I'm sure something similar is possible in other frameworks too - in fact most frameworks probably have something builtin to handle it for you (Flask may do, I didn't bother looking).

In the url cited , google says: "To enable compression, configure your web server to set the Content-Encoding header to gzip format for all compressible resources." So I thinked that this was only a web server configuration. PageSpeed points that my web2py app hasn't compression enable. So, i will investigate about it.

Thanks

You definitely can enable gzip on your own application servers. There's also an nginx setting we could switch on so that everyone's web apps that would make everything gzipped, regardless of whether you asked for it. I suspect that if we switch that on we'll have to be careful to make sure that it handles all of the crazy edge cases (for example, I believe that there are versions of Internet Explorer that say they handle gzip but don't). But config to do that is pretty easy to find.

What do you guys think -- should we switch it on? It would lower the effort required for you to have it switched on, but I do worry that some people's web apps would stop working for people with older browsers...

@puercoespin: It's a little more than just adding a Content-Encoding header to your response - you have to also actually gzip the content! I think the Google page is alluding to the fact that many frameworks and webserver have options and/or extension modules to handle the zipped transparently, so it's true that in many cases application code wouldn't have to do it manually.

@giles: I'd say any browser that doesn't handle gzip these days is so lame that their users must be used to the web simply Not Working. On the other hand, plaintext is ever so much more useful for debugging problems with command-line GET or nc. Certainly it should only add the encoding if the client adds the appropriate Accept-Encoding header - I'd hope that went without saying, though!

I'm of two minds, but I think on balance it's a teensy bit of a large step to take unilaterally - for example, some people may only want larger resources zipped, and I'm not sure a one-size-fits-all config could achieve that sort of thing. On the other hand it's probably fairly harmless to turn it on and turn it back off if people have problems.

Oh, and if someone's web framework already does it manually, you'd have to be careful not to gzip it again - I'd hope that nginx would only add the gzip compression if the response didn't already have Content-Encoding: gzip, but again worth checking.

Thanks, Cartroo -- as always, useful food for thought!

May be thats would be interesting:

http://betterexplained.com/articles/how-to-optimize-your-site-with-gzip-compression/

As article says, Google and Yahoo use gzip compression. I think my little web can do too...so @giles, i think that enable compression in the server would be by default

@Cartroo, how do you enable compression in Flask?, There are no information about web2py. May be that will help me.

Thanks

Well, I did gzip compression by hand (i.e. in the application itself). Being a microframework, I don't believe core Flask has any support for compression, but I'm sure there are simple third-party extensions to save you the bother - this looks like a simple example. I'm not sure why they're using StringIO instead of cStringIO, but hey ho.

Of course, if the PAW staff decide to implement this at nginx that's always going to be the most efficient solution, but doing it in your application / framework is good second best for now.

Thanks, Cartroo. Its very interesting that code. But its useful only if your host dont enable compression. And as you point, implement it at nginx is the mos efficient solution.

In http://www.howtoforge.com/how-to-save-traffic-with-nginxs-httpgzipmodule-debian-squeeze seems that configuring a nginx server its safe and easy.

Well, I think all are agreed. Please, enable web server compression!

Thanks

OK, it sounds like we should try turning it on and then switch if off again if it causes anyone any problems. As you say, it should be pretty easy -- however, it probably won't get out until the New Year -- we're racing to get multiple web applications live before Christmas and then will be in a support-only mode for a week or so. But I'll see if we can get this into our integration system so that it's ready to go live at the start of January.

@puercoespin: Yes, compression in nginx is the most efficient option, but there's a tradeoff with compatibility, unless every user can have their own nginx configuration (which is feasible but quite difficult to set up, and potentially a management nightmare). So I'm just saying, I can understand both sides.

@giles: Wise move - I'm sure those hosting production websites off PAW wouldn't appreciate getting a Nagios SMS while they're tucking into their turkey dinner! (^_^)

Fantastic!

And by the way, why not configure caching? as explained in https://developers.google.com/speed/docs/best-practices/caching#LeverageBrowserCaching and for nginx, http://www.howtoforge.com/make-browsers-cache-static-files-on-nginx

There another interestings docs in http://www.mnot.net/cache_docs/ or http://fortheloveofseo.com/blog/performance/leverage-browser-caching-how-to-add-expires-headers/

Thanks

OK, so gzip compression is now cooking in our testing system over the break, if all looks good we'll have it out early next year.

We'll take a look at caching too but that definitely does sound like the kind of thing we'd need people to be able to configure themselves -- we occasionally have problems with browsers caching bits of our JavaScript even though we send "don't cache" headers with it!

I would say that any dynamic output shouldn't have any caching headers added or removed by the hosting system - it's entirely up to the application to add whatever headers are required. Application developers either need to properly educate themselves ebout ETags and conditional requests, or they need to just make sure they mark every response as not cachable. Sorry, it's just one of those areas where you can't afford to take shortcuts (at least not in a production environment).

Static content is a little trickier. I don't think the hosting can add a one-size-fits-all Expires header, as this could cause some annoyances (although a short one of a few minutes probably wouldn't cause too much damage). However, I would hope that most browsers could still cache sensibly via conditional requests (i.e. adding an If-Modified-Since header and nginx supports this natively. I can see that nginx on PAW is already setting Last-Modified headers, which browsers can also use for short-term caching in lieu of any other caching mechanism.

From a little poking around it seems that nginx doesn't support ETags natively until version 1.3.3. There's at least one third party module to do it, but it might be a pain to install that and then be landed with the job of updating all the configuration when it moves into mainline.

Anyway, I think it would be lovely to add the ability for users to specify an Expires header for their static content (ideally on a per-directory basis), but realistically that's not going to be high on the priority list. It's probably a "nice to have" to add on to the static content todo list.

In the meantime, I guess the PAW staff could double-check that nginx is configured to support conditional requests properly, since that's something that should always be harmless as long as you trust your file timestamps.

Hi, I was using Google Pagespeed today and it seems that the servers at PAW still don't support compression.

How can I switch it on, if it's a user-setting?

Thanks!

I'm not aware of any settings for this - hopefully the PA devs can jump in to say whether they're still planning to implement this as an nginx feature.

In the meantime you can always see if your web framework has some middleware which you can easilly insert to perform the compression at that stage.

Not that the PageSpeed scores are all-important, but it'd be great if I can increase the current score of 71 to 90+, since I was testing the speeds of the same web page for different hosts. I suspect having compression will help push this one up.

We do have gzip compression enabled, but it looks like we don't have it enabled for application/javascript content type. We do have it enabled for text/javasccipt. I'll raise a ticket to check that we have it enabled for all content types where it will be useful.

Has gzip compression been enabled for application/javascript content type ?

It has not. I'll bump the ticket up the priority queue...

Thank you. Do you have an idea of when it will be enabled ?

I can't make any promises I'm afraid. We're focused on fixing up our backups and zero-downtime issues at the moment. Hopefully we'll be able to squeeze it in if it's quick and easy to implement, but I don't know if we'll find the time...

Someone asked about this the other day, and I realised we never updated the thread. gzip compression is now enabled for application/javascript:

$ curl -I -H 'Accept-Encoding: gzip,deflate' http://www.web2pyslices.com/init/static/bootstrap/js/tag-it.js
HTTP/1.1 200 OK
Server: ngx_openresty/1.4.3.6
Date: Thu, 06 Mar 2014 12:14:16 GMT
Content-Type: application/javascript
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: web2py
Last-Modified: Wed, 17 Jul 2013 12:33:32 GMT
Pragma: cache
Cache-Control: private
Content-Encoding: gzip

(webpyslices is one of our hosted domains)...

how can I enable the GZIP?

It's already enabled.