Sunday, December 4, 2011

Memcache in Drupal

The Memcache contributed project provides memcache integration, and support for Drupal's caching, session, and lock backends.

Requirements

There are two different PECL packages supported by the 1.5+ release of the memcache module. If using the older memcache PECL package, 2.2.1 or higher is required to avoid a deletion bug. If using the more modern memcached PECL package, version 1.0 or higher is required.

Installation


These are the broad steps you need to take in order to use this software on Drupal 6.x.Information pertaining to the Drupal 7.x version is also found in the README.txt file from the module distribution.
For 6.x (order is important):
  1. Install the memcached binaries on your server. See How to install Memcache on Debian Etch or How to install Memcache on OSX
  2. Install the PECL memcache extension for PHP.
  3. In php.ini set memcache.hash_strategy="consistent".
  4. Put your site into offline mode.
  5. Download and install the memcache module.
  6. If you have previously been running the memcache module, run update.php.
  7. Start at least one instance of memcached on your server.
  8. Edit settings.php to configure the servers, clusters and bins that memcache is supposed to use. (see code snippet below)
  9. Edit settings.php to include memcache.inc. For example, $conf['cache_inc'] ='sites/all/modules/memcache/memcache.inc';
  10. Bring your site back online.
For 7.x (order is important):
  1. Install the memcached binaries on your server. See How to install Memcache on Debian Etch or How to install Memcache on OSX
  2. Install the PECL memcache extension for PHP. This must be version 2.2.1 or

    higher or you will experience errors.
  3. Put your site into offline mode.
  4. Download and install the memcache module.
  5. If you have previously been running the memcache module, run update.php.
  6. Start at least one instance of memcached on your server.
  7. Edit settings.php to mke memcache the default cache class, for example:
         $conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
         $conf['cache_default_class'] = 'MemCacheDrupal';
  8. Bring your site back online.
For instructions on 1 and 2 above, please see the INSTALLATION.txt file that comes with the memcache module download.

Servers

NOTE: As of 6.15, it is possible to efficiently run memcache in one daemon, shared by all bins. You can safely ignore advice found elsewhere on the internet that advises one memcached daemon per bin. In terms of reporting, there are still some advantages to having more daemons.
If you want the simple version, you can start one default memcache instance on your web server like this: memcached -m 24 -p 11211 -d If that is enough to meet your needs, there is no more configuration needed. If you want to utilize this module's sophisticated clustering feature and spread your cache over several machines, or if your cache is found on a machine other than your web server, read on.
You should probably lock down the memcache server so that it only listens for connections from the hosts that need to be served, as the default is that memcache listens to connections from all addresses. So, to close that hole, edit /etc/sysconfig/memcached with:
OPTIONS="-l ${HOSTIP}"
For example:
OPTIONS="-l 127.0.0.1"
The available memcached servers are specified in $conf in settings.php. If you do not specify any servers, memcache.inc assumes that you have a memcached instance running on 127.0.0.1:11211. If this is true, and it is the only memcached instance you wish to use, no further configuration is required.
If you have more than one memcached instance running, you need to add two arrays to $conf; memcache_servers and memcache_bins. The arrays follow this pattern:
<?php'memcache_servers' => array(host1:port => cluster, host2:port => cluster, hostN:port => cluster)'memcache_bins' => array(bin1 => cluster, bin2 => cluster, binN => cluster)?>
WARNING: Avoid the use of "localhost" and instead use real IP addresses or hostnames that will remain consistent across the network.
The bin/cluster/server model can be described as follows:
  • Servers are memcached instances identified by host:port.
  • Bins are groups of data that get cached together and map 1:1 to the $table param in cache_set(). Examples from Drupal core are cache_filter, cache_menu. The default is 'cache'.
  • Clusters are groups of servers that act as a memory pool.
  • Many bins can be assigned to a cluster.
  • The default cluster is 'default'.
Here is a simple setup that has two memcached instances, both running on 10.1.1.1. The 11212 instance belongs to the 'pages' cluster and the table cache_page is mapped to the 'pages' cluster. Thus everything that gets cached, with the exception of the page cache (cache_page), will be put into 'default', or the 11211 instance. The page cache will be in 11212.
<?php
$conf
= array(
 
'memcache_servers' => array('10.1.1.1:11211' => 'default',
                             
'10.1.1.1:11212' => 'pages'),
 
'memcache_bins' => array('cache_page' => 'pages'),
);
?>
?>
Here is an example configuration that has two clusters, 'default' and 'cluster2'. Five memcached instances are divided up between the two clusters. 'cache_filter' and 'cache_menu' bins go to 'cluster2'. All other bins go to 'default'.
D6:
<?php
$conf
= array(
 
'cache_inc' => './sites/all/modules/memcache/memcache.inc',
 
'memcache_servers' => array('10.1.1.1:11211' => 'default',
                             
'10.1.1.1:11212' => 'default',
                             
'10.1.1.2:11211' => 'default',
                             
'10.1.1.3:11211' => 'cluster2',
                             
'10.1.1.4:11211' => 'cluster2'),

 
'memcache_bins' => array('cache' => 'default',
                          
'cache_filter' => 'cluster2',
                          
'cache_menu' => 'cluster2'),
);
?>


?>
D7:
<?php
$conf
['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';$conf['cache_default_class'] = 'MemCacheDrupal';$conf = array(
 
'cache_default_class' = 'MemCacheDrupal',
 
'memcache_servers' => array('10.1.1.1:11211' => 'default',
                             
'10.1.1.1:11212' => 'default',
                             
'10.1.1.2:11211' => 'default',
                             
'10.1.1.3:11211' => 'cluster2',
                             
'10.1.1.4:11211' => 'cluster2'),

 
'memcache_bins' => array('cache' => 'default',
                          
'cache_filter' => 'cluster2',
                          
'cache_menu' => 'cluster2'),
);
?>

Prefixing

If you want to have multiple Drupal installations share memcached instances, you need to include a unique prefix for each Drupal installation in the $conf array of settings.php:
<?php
$conf
= array(
 
'memcache_key_prefix' => 'something_unique',
);
?>

Sessions

NOTE: Session.inc is not yet ported to Drupal 7.
Here is a sample config that uses memcache for sessions. Note you MUST have

a session and a users server set up for memcached sessions to work.
<?phpinclude_once('./includes/cache.inc');
include_once(
'./sites/all/modules/memcache/memcache.inc');$conf = array(
 
'cache_default_class' = 'MemCacheDrupal',
 
'session_inc' => './sites/all/modules/memcache/memcache-session.inc',
 
'memcache_servers' => array(
   
'localhost:11211' => 'default',
   
'localhost:11212' => 'filter',
   
'localhost:11213' => 'menu',
   
'localhost:11214' => 'page',
   
'localhost:11215' => 'session',
   
'localhost:11216' => 'users',
  ),
 
'memcache_bins' => array(
   
'cache' => 'default',
   
'cache_filter' => 'filter',
   
'cache_menu' => 'menu',
   
'cache_page' => 'page',
   
'session' => 'session',
   
'users' => 'users',
  ),
);
?>

Troubleshooting

PROBLEM: Error: Failed to set key: Failed to set key: cache_page-......

SOLUTION: Upgrade your PECL library to PECL package (2.2.1) (or higher).
PROBLEM: WARNING: Zlib compression at the php.ini level and Memcache conflict.

SOLUTION: See http://drupal.org/node/273824

Memcache Admin

A module offering a UI for memcache is included. It provides stats, a

way to clear the cache, and an interface to organize servers, bins, and clusters.

Memcached PECL Extension Support

We also now support the Memcached PECL extension. If you install this extension,

it will be used by default. This new extension backends to libmemcached and

allows you to use some of the newer advanced features in memcached 1.4.
NOTE: It is important to realize that the memcache php.ini options do not impact

the memcached extension, this new extension doesn't read in options that way.

Instead, it takes options directly from Drupal. Because of this, you must

configure memcached in settings.php. Please look here for possible options:
http://us2.php.net/manual/en/memcached.constants.php
An example configuration block is below, this block also illustrates our

default options. These will be set unless overridden in settings.php.
<?php
$conf
['memcache_options'] = array(
 
Memcached::OPT_COMPRESSION => FALSE,
 
Memcached::OPT_DISTRIBUTION => Memcached::DISTRIBUTION_CONSISTENT,
);
?>
These are as follows:
  • Turn off compression, as this takes more CPU cycles than its worth for most

    users
  • Turn on consistent distribution, which allows you to add/remove servers

    easily
If you are using memcached 1.4 or above, you should enable the binary protocol,

which is more advanced and faster, by adding the following to settings.php:
<?php
$conf
['memcache_options'] = array(
 
Memcached::OPT_BINARY_PROTOCOL => TRUE,
);
?>


Setup for Linux Memcached (daemon) with PHP PECL Memcache and Drupal Memcache Module - with localized UNIX sockets

No comments:

Post a Comment