Cache and compression of webpages with Gzip or Deflate http

Today, the focus is often on site performance and bandwidth savings on mobile. Access and download times are measured (very precisely with many tools) in milliseconds. Many recommendations from Google and Yahoo refer to the compression of the pages before their transit on the network.

1. The server compresses data (HTML, CSS, JavaScript code...)
2. The files through the network via HTTP
3. The browser decompress on the fly data before interpreting them

Which represented an additional burden for the web servers at the time where their power was lower, can now become negligible next to improvements, particularly for mobile browsers. If compression imposes too high a load on your server, it is possible to pré-compresser the content, place them in a cache and deliver them directly.

These techniques, which are planned for HTTP/1.1 (1999) can quite be implemented for HTML but also CSS, JavaScript or XML documents. There is no need to use for binary files (images, videos, PDF...). They do not exempt you to initially reduce the size of files HTML or CSS ("minify") by applying other criteria such as the removal of surplus spaces or unnecessary comments.

Browsers

The first poll - in the ruthless world of web design and the wars between ancient and modern browsers - concerns the support for this feature. Gold, here, good news: we can consider that all of the browsers support decompression of pages with HTTP/1.1:

• Microsoft Internet Explorer since 4.0*
• Opera since 5.12
• Firefox all versions
• Google Chrome all versions
• Safari all versions
• Netscape since 4.06
• All mobile browsers

* with a few bugs until the versions 5.0 and 6.0

In addition, browserssend an HTTP header indicating the types of supported compressed pages are responsible. If this header is not in those received by the server, he simply do not enable compression.

GET / HTTP/1.1
Host: www.alsacreations.com
Accept-Encoding: gzip
User-Agent: Firefox/3.6

The server then responds in the same way, using Content-Encoding and by forwarding by the compressed page content.

HTTP/1.1 200 OK
Server: Apache
Content-Type: text/html
Content-Encoding: gzip
Content-Length: 13337

...

Both formats coexist:

• Deflatealgorithm that couple LZ77 and Huffman coding (zlib)
• Gzip, Deflate, a little more efficient, better supported, most common evolution

Implementation at the level of the web servers

Attention: the following information must be adjusted according to your configuration and your needs.

Apache is equipped with compression modules:

• in older versions 1.3: mod_gzip or mod_deflate
• Since version 2.0 and higher: the official module mod_deflate that uses zlib and replaces mod_gzip

These modules are disabled by default, but can be activated in the General server configuration if you have access. Default mod_deflate allows you to specify the types of files to compress on the fly thanks to the directive AddOutputFilterByType DEFLATE. Once these modules available you can also use .htaccess files in each directory for more flexibility (see next point).

Activation of the modules for Apache 2

If your server allows it, you can access command-line with root rights to activate the necessary modules with the a2enmod command:

a2enmod headers
a2enmod deflate
/etc/init.d/apache2 restart
# ou... service apache2 restart

Manually adding a file in mods-available/deflate.load and a symbolic link to it in mods-enabled/, or a line in httpd.conf :

LoadModule deflate_module /usr/lib/apache2/modules/mod_deflate.so

Under Windows, you specify the path of the eponymous .dll file. Attention: If a symbolic link to mod_deflate.conf is already present in mods-enabled with a general configuration directive, it is possible that all your files are already delivered compressed. Do a test first and foremost - see end of article.

Then you have to add directives to the configuration (for example in a file located in /etc/apache2/conf.d/) to compress a specific file types, in a specific directory him also. This is recommended when we place all style sheets in an independent directory, as well as the JavaScripts, because the goal is not to (re) compress all files hosted on the web server but to focus on the essential. It is therefore possible to specify <Location /css> to apply these rules on this directory (considered here the URL) and his descendants, or to use <Directory /chemin/absolu/vers/css> if one refers to the system of files.

<IfModule mod_deflate.c>
 SetOutputFilter DEFLATE
 DeflateCompressionLevel 9
</IfModule>

<Location />
 AddOutputFilterByType DEFLATE text/plain
 AddOutputFilterByType DEFLATE text/xml
 AddOutputFilterByType DEFLATE text/html
 AddOutputFilterByType DEFLATE text/css
 AddOutputFilterByType DEFLATE image/svg+xml
 AddOutputFilterByType DEFLATE application/xhtml+xml
 AddOutputFilterByType DEFLATE application/xml
 AddOutputFilterByType DEFLATE application/rss+xml
 AddOutputFilterByType DEFLATE application/atom_xml
 AddOutputFilterByType DEFLATE application/x-javascript

 # Pour les proxies
 Header append Vary User-Agent env=!dont-vary
</Location>

DeflateCompressionLevel

Specifies the factor of compression, 1 (low, by default) to 9 (loudly): is applicable only at the level of the overall configuration of the server, not in the .htaccess files

AddOutputFilterByType DEFLATE text/html

Applies compression on files of type mime text/html

SetOutputFilter DEFLATE

Active compression filter.

Don't forget to restart (restart) or reload (reload) Apache after each configuration change.

/etc/init.d/apache2 restart
# ou service apache2 restart

Apache 1.3

To activate the module in the httpd.conf configuration file, add the line:

LoadModule gzip_module modules/mod_gzip.so

Then configuration, similar to the one mentioned previously:

<IfModule mod_gzip.c>
   mod_gzip_on Yes
   mod_gzip_can_negotiate Yes
   mod_gzip_static_suffix .gz
   AddEncoding gzip .gz
   mod_gzip_update_static No
   mod_gzip_command_version '/mod_gzip_status'
   mod_gzip_temp_dir /tmp
   mod_gzip_keep_workfiles No
   mod_gzip_minimum_file_size 500
   mod_gzip_maximum_file_size 500000
   mod_gzip_maximum_inmem_size 60000
   mod_gzip_min_http 1000
   mod_gzip_handle_methods GET POST
   mod_gzip_item_exclude reqheader "User-agent: Mozilla/4.0[678]"
   mod_gzip_item_include file \.html$
   mod_gzip_item_include file \.htm$
   mod_gzip_item_include file \.php3$
   mod_gzip_item_include file \.php$
   mod_gzip_item_include file \.js$
   mod_gzip_item_include file \.css$
   mod_gzip_item_include mime ^text/
   mod_gzip_item_exclude mime ^httpd/unix-directory
   mod_gzip_item_exclude mime ^image/
   mod_gzip_dechunk Yes
   mod_gzip_add_header_count Yes
   mod_gzip_send_vary Yes
</IfModule>

Don't forget to reload Apache after each configuration change.

The options are relatively speaking (for English speakers), they will not be detailed in this article.

Microsoft IIS

IIS supports compression since version 4, but it is victim of bugs. In version 5, Microsoft's efforts have not been successful, since it is still unstable. It is finally in version 6 that HTTP compression has been finalized. However it requires some manipulations.

The other servers remain marginal. Lighttpd has aptly named mod_compress module.

Quick fixes with a .htaccess for Apache

.Htaccess files are files placed at the base of a directory and changing the behavior of the server for the files that it contains. You can place the configuration instructions mentioned above (without the rental or Directory directive).

Examples tested and operated on 51js.com. If you get HTTP 500 error after implementing the .htaccess file, check its syntax, the line with your type of server and the availability of the modules. You can also combine all with cache options (in the example for Apache 1.3 mod_expires) to avoid serving several times the same content to visitors and its compression by the server - this is a matter for another article.

Apache 2

The content of the .htaccess file, in the directory with the CSS and JavaScript files.

# Apache 2.0
SetOutputFilter DEFLATE
AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript

Apache 1.3

# Compression pour fichiers CSS
<IfModule mod_gzip.c>
    mod_gzip_on       Yes
    mod_gzip_dechunk  Yes
    mod_gzip_minimum_file_size 1024
    mod_gzip_maximum_file_size 100000
    mod_gzip_item_include file \.css$
    mod_gzip_item_include mime ^text/css$
</IfModule>

<IfModule mod_expires.c>
    ExpiresActive on
    ExpiresDefault "access plus 1 month"
    ExpiresByType text/css "access plus 1 day"
    ExpiresByType image/png "access plus 1 week"
    ExpiresByType image/gif "access plus 1 week"
    ExpiresByType image/jpeg "access plus 1 week"
</IfModule>

# Compression pour fichiers JS
<IfModule mod_gzip.c>
    mod_gzip_on       Yes
    mod_gzip_dechunk  Yes
    mod_gzip_minimum_file_size 512
    mod_gzip_maximum_file_size 1000000
    mod_gzip_item_include file \.js$
    mod_gzip_item_include mime ^application/x-javascript.*
</IfModule>

# Cache
<IfModule mod_expires.c>
    ExpiresActive on
    ExpiresByType application/x-javascript "access plus 2 month"
    ExpiresByType application/javascript "access plus 2 month"
    ExpiresByType text/javascript "access plus 2 month"
</IfModule>

Alternative in PHP

The ob_gzhandler function and all functions of type ob_* available since PHP4 allow the management of the output buffer, that is, data that will be sent to the browser. It is then possible to generate the complete content of the page and compress it with Gzip before sending. We activate the buffer at the beginning of script with ob_start, and we empty it at the end with ob_end_flush.

The ob_gzhandler function has the merit to check the cuts types supported by the browser (gzip, deflate or none) before returning the contents of the buffer in an appropriate manner. If the browser does not support compressed pages, this function will return false.

<?php
ob_start("ob_gzhandler");
?>
... Le reste du code ...
<?php
ob_end_flush();
?>

Of course, this is to adapt the structure of your site. It is not always enough to put these statements at the beginning and end of the PHP script because many CMS are already using their own buffer (buffer) system in-house.

Tests in practice

You can easily verify the success of the transfer and the decompression by checking the properties of the page in the browser (in Firefox: click right button, information on the page, General tab, size). Compare the size of the original file and read (at about byte) size.

See also the information provided by extensions of development (such as Firebug or development tools) available in modern browsers. These can be found in the network or Networktab, and mention the HTTP headers of compression as well as the amount of downloaded data.