Introduction¶

This is a subclass of Memcache which swallows exceptions and treats them like misses. This is meant to allow code to be a bit simpler, rather than catching all exceptions, you can just do things like the below example.

Examples¶

Basic example:

>>> import memcached2
>>> mc = memcached2.ExceptionsAreMissesMemcache(('memcached://localhost/',))
>>> data = mc.get('key')
>>> if not data:
>>>    data = [compute data]
>>>    mc.set('key', data)
>>> [use data]

Object Documentation¶

class memcached2.ExceptionsAreMissesMemcache(servers, value_wrapper=None, selector=None, hasher=None, server_pool=None)¶

A Memcache wrapper class which swallows server exceptions, except in the case of coding errors. This is meant for situations where you want to keep the code simple, and treat cache misses, server errors, and the like as cache misses. See memcached2.Memcache() for details of the use of this class, exceptions to that are noted here.

The methods that are protected against exceptions are those documented in this class. Everything should otherwise act like a Memcache instance.

Parameters:

servers (list) – One or more server URIs of the form: “memcache://hostname[:port]/” value_wrapper (ValueSuperStr or compatible object.) – (None) This causes values returned to be wrapped in the passed class before being returned. For example ValueSuperStr implements many useful additions to the string return. selector (SelectorBase) – (None) This code implements the server selector logic. If not specified, the default is used. The default is to use SelectorFirst if only one server is specified, and SelectorRehashDownServers if multiple servers are given. hasher (HasherBase) – (None) A “Hash” object which takes a key and returns a hash for persistent server selection. If not specified, it defaults to HasherZero if there is only one server specified, or HasherCMemcache otherwise. server_pool (ServerPool object.) – (None) A server connection pool. If not specified, a global pool is used.

delete(*args, **kwargs)¶

Remove this key from the server.

Exceptions are swallowed and treated as memcached misses. See delete() for details on this method. Changes from the base function are:

Raises:	Exceptions are swallowed and treated a misses.

get(*args, **kwargs)¶

Retrieve the specified key from a memcache server.

Exceptions are swallowed and treated as memcached misses. See get() for details on this method. Changes from the base function are:

Returns:	None if no value or exception, String, or “value_wrapper” as specified during object creation such as ~memcached2.ValueSuperStr.
Raises:	Exceptions are swallowed and treated a misses.

set(*args, **kwargs)¶

Update the value in the server. See set() for details on this method. Changes from the base function are:

Exceptions are swallowed and treated as memcached misses. See set() for details on this method. Changes from the base function are:

Raises:	Exceptions are swallowed and treated a misses.

set_multi(*args, **kwargs)¶

Update multiple values in the server. See set_multi() for details on this method. Changes from the base function are:

Exceptions are swallowed and treated as memcached misses. See set() for details on this method. Changes from the base function are:

Raises:	Exceptions are swallowed and treated a misses.

Introduction¶

This is a dictionary-like interface to memcache, but it swallows server exceptions, except in the case of coding errors. This is meant for situations where you want to keep the code simple, and treat cache misses, server errors, and the like as cache misses.

On the instantiation you specify the servers, and at that point it can be accessed as a dictionary, including access, setting, and deleting keys. See the examples for a demonstration.

For functionality beyond what you can get from the dictionary interface, you need to use the memcache attribute, which is an Memcache instance. See that documentation for access to flusing servers, statistics, and other things not supported by the mapping interface.

Note that NotImplementedException will be raised for situations where there are code errors. So it’s recommended that you don’t just trap these, either catch and log them, or just let them raise up so that application users can report the bug.

Examples¶

Basic example:

>>> import memcached2
>>> mcd = memcached2.ExceptionsAreMissesMapping(('memcached://localhost/',))
>>> 'foo' in mcd
False
>>> mcd['foo'] = 'hello'
>>> 'foo' in mcd
True
>>> mcd['foo']
'hello'
>>> len(mcd)
1
>>> del(mcd['foo'])
>>> len(mcd)
0

Object Documentation¶

class memcached2.ExceptionsAreMissesMapping(servers, selector=None, hasher=None)¶

A dictionary-like interface which swallows server exceptions.

This is a dictionary-like interface to memcache, but it swallows server exceptions, except in the case of coding errors. This is meant for situations where you want to keep the code simple, and treat cache misses, server errors, and the like as cache misses.

See ExceptionsAreMissesMapping Introduction and ExceptionsAreMissesMapping Examples for more information.

Introduction¶

This is a low-level interface to a group of memcache servers. This code tends to either return the requested data, or raises an exception if the data is not available or there is any sort of an error. If you want high level control, this is probably the interface for you. However, if you want something easy, like the old python-memcached module, you will want to wait for the higher level interfaces to be implemented.

Examples¶

Basic get() and exception example:

>>> import memcached2
>>> memcache = memcached2.Memcache(('memcached://localhost/',))
>>> try:
...    result = memcache.get('session_id')
...    print('Got cached results: {0}'.format(repr(result)))
... except memcached2.NoValue:
...    print('Cached value not available, need to recompute it')
...
Cached value not available, need to recompute it

Demonstrating set(), get() and ValueSuperStr:

>>> memcache.set('session_id', 'TEST SESSSION DATA')
>>> result = memcache.get('session_id')
>>> print('Got cached results: {0}'.format(repr(result)))
Got cached results: 'TEST SESSSION DATA'
>>> result.key
'session_id'
>>> result.flags
0

Example of get_multi() to retrieve multiple keys quickly:

>>> memcache.set('foo', '1')
>>> memcache.set('bar', '2')
>>> result = memcache.get_multi(['foo', 'bar', 'baz'])
>>> result.get('foo')
'1'
>>> result.get('bar')
'2'
>>> result.get('baz')
None

Usage of cache() to automatically cache values:

>>> numbers = range(10)
>>> calculate = lambda x: str(numbers.pop())
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.cache('foo', calculate)
'9'
>>> memcache.set('foo', 'hello')
>>> memcache.cache('foo', calculate)
'hello'
>>> memcache.flush_all()
>>> memcache.cache('foo', calculate)
'8'

Showing flags and expiration time and touch():

>>> memcache.set('foo', 'xXx', flags=12, exptime=30)
>>> result = memcache.get('foo')
>>> result
'xXx'
>>> result.key
'foo'
>>> result.flags
12
>>> import time
>>> time.sleep(30)
>>> result = memcache.get('foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 411, in get
    raise NoValue()
memcached2.NoValue
>>> memcache.set('foo', 'bar', exptime=1)
>>> memcache.touch('foo', exptime=30)
>>> time.sleep(2)
>>> memcache.get('foo')
'bar'

Usage of replace(), append(), and prepend():

>>> memcache.replace('unset_key', 'xyzzy')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 502, in replace
    self._storage_command(command, key)
  File "memcached2.py", line 945, in _storage_command
    raise NotStored()
memcached2.NotStored
>>> memcache.set('unset_key', 'old_data', exptime=30)
>>> memcache.replace('unset_key', 'xyzzy')
>>> memcache.get('unset_key')
'xyzzy'
>>> memcache.append('unset_key', '>>>')
>>> memcache.prepend('unset_key', '<<<')
>>> memcache.get('unset_key')
'<<<xyzzy>>>'

Example of using CAS (Check And Set) atomic operations:

>>> memcache.set('foo', 'bar')
>>> result = memcache.get('foo', get_cas=True)
>>> result.cas_unique
5625
>>> memcache.set('foo', 'baz', cas_unique=result.cas_unique)
>>> memcache.get('foo', get_cas=True)
'baz'
>>> memcache.set('foo', 'qux', cas_unique=result.cas_unique)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 464, in set
    self._storage_command(command, key)
  File "memcached2.py", line 947, in _storage_command
    raise CASFailure()
memcached2.CASFailure
>>> memcache.get('foo', get_cas=True)
'baz'

Usage of incr()/decr():

>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 878, in incr
    return self._incrdecr_command(command, key)
  File "memcached2.py", line 915, in _incrdecr_command
    raise NotFound()
memcached2.NotFound
>>> memcache.set('incrtest', 'a')
>>> memcache.incr('incrtest', 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "memcached2.py", line 878, in incr
    return self._incrdecr_command(command, key)
  File "memcached2.py", line 919, in _incrdecr_command
    raise NonNumeric()
memcached2.NonNumeric
>>> memcache.set('incrtest', '1')
>>> memcache.incr('incrtest', 1)
2
>>> memcache.decr('incrtest', 1)
1
>>> memcache.get('incrtest')
'1'

Statistics sample information:

>>> import pprint
>>> pprint.pprint(memcache.stats())
[{'accepting_conns': '1',
  'auth_cmds': 0,
  'auth_errors': 0,
  'bytes': 201,
  'bytes_read': 173542,
  'bytes_written': 516341,
  'cas_badval': 49,
  'cas_hits': 49,
  'cas_misses': 0,
  'cmd_flush': 1154,
  'cmd_get': 880,
  'cmd_set': 5778,
  'cmd_touch': '148',
  'conn_yields': 0,
  'connection_structures': 9,
  'curr_connections': 5,
  'curr_items': 3,
  'decr_hits': 49,
  'decr_misses': 48,
  'delete_hits': 49,
  'delete_misses': 49,
  'evicted_unfetched': 0,
  'evictions': 0,
  'expired_unfetched': 0,
  'get_hits': '681',
  'get_misses': '199',
  'hash_bytes': 262144,
  'hash_is_expanding': '0',
  'hash_power_level': 16,
  'incr_hits': 49,
  'incr_misses': 49,
  'libevent': '2.0.19-stable',
  'limit_maxbytes': 67108864,
  'listen_disabled_num': '0',
  'pid': 22356,
  'pointer_size': 32,
  'reclaimed': 0,
  'reserved_fds': 20,
  'rusage_system': 7.568473,
  'rusage_user': 8.904556,
  'threads': 4,
  'time': 1366722131,
  'total_connections': 1545,
  'total_items': 5634,
  'touch_hits': 98,
  'touch_misses': 50,
  'uptime': 370393,
  'version': '1.4.14'}]
>>> pprint.pprint(memcache.stats_settings())
[{'auth_enabled_sasl': 'no',
  'binding_protocol': 'auto-negotiate',
  'cas_enabled': True,
  'chunk_size': 48,
  'detail_enabled': False,
  'domain_socket': 'NULL',
  'evictions': 'on',
  'growth_factor': 1.25,
  'hashpower_init': 0,
  'inter': '127.0.0.1',
  'item_size_max': 1048576,
  'maxbytes': 67108864,
  'maxconns': 1024,
  'maxconns_fast': False,
  'num_threads': 4,
  'num_threads_per_udp': 4,
  'oldest': 216734,
  'reqs_per_event': 20,
  'slab_automove': False,
  'slab_reassign': False,
  'stat_key_prefix': ':',
  'tcp_backlog': 1024,
  'tcpport': 11211,
  'udpport': 11211,
  'umask': 700,
  'verbosity': 0}]
>>> pprint.pprint(memcache.stats_items())
[{'1': {'age': 766,
        'evicted': 0,
        'evicted_nonzero': 0,
        'evicted_time': 0,
        'evicted_unfetched': 0,
        'expired_unfetched': 0,
        'number': 3,
        'outofmemory': 0,
        'reclaimed': 0,
        'tailrepairs': 0}}]
>>> pprint.pprint(memcache.stats_sizes())
[[(64, 1), (96, 2)]]
>>> pprint.pprint(memcache.stats_slabs())
[{'active_slabs': 1,
  'slabs': {'1': {'cas_badval': 49,
                  'cas_hits': 49,
                  'chunk_size': 80,
                  'chunks_per_page': 13107,
                  'cmd_set': 5778,
                  'decr_hits': 49,
                  'delete_hits': 49,
                  'free_chunks': 13104,
                  'free_chunks_end': 0,
                  'get_hits': 681,
                  'incr_hits': 49,
                  'mem_requested': 201,
                  'total_chunks': 13107,
                  'total_pages': 1,
                  'touch_hits': 98,
                  'used_chunks': 3}},
  'total_malloced': 1048560}]

How to delete(), flush_all(), and close() the connection:

>>> memcache.delete('foo')
>>> memcache.flush_all()
>>> memcache.close()

Object Documentation¶

class memcached2.Memcache(servers, value_wrapper=None, selector=None, hasher=None, server_pool=None)¶

Create a new memcache connection, to the specified servers.

The list of servers, specified by URL, are consulted based on the hash of the key, effectively “sharding” the key space.

This is a low-level memcache interface. This interface will raise exceptions when backend connections occur, allowing a program full control over handling of connection problems.

Example:

>>> from memcached2 import *                               # noqa
>>> mc = Memcache(['memcached://localhost:11211/'])
>>> mc.set('foo', 'bar')
>>> mc.get('foo')
'bar'

Extensive examples including demonstrations of the statistics output is available in the documentation for Memcache Examples

Parameters:

servers (list) – One or more server URIs of the form: “memcache://hostname[:port]/” value_wrapper (ValueSuperStr or compatible object.) – (None) This causes values returned to be wrapped in the passed class before being returned. For example ValueSuperStr implements many useful additions to the string return. selector (SelectorBase) – (None) This code implements the server selector logic. If not specified, the default is used. The default is to use SelectorFirst if only one server is specified, and SelectorRehashDownServers if multiple servers are given. hasher (HasherBase) – (None) A “Hash” object which takes a key and returns a hash for persistent server selection. If not specified, it defaults to HasherZero if there is only one server specified, or HasherCMemcache otherwise. server_pool (ServerPool object.) – (None) A server connection pool. If not specified, a global pool is used.

add(key, value, flags=0, exptime=0)¶

Store, but only if the server doesn’t already hold data for it.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (str) – Value stored in memcache server for this key. flags (int (32 bits)) – If specified, the same value will be provided on get(). exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.

append(key, value)¶

Store data after existing data associated with this key.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (str) – Value stored in memcache server for this key.

close()¶

Close the connection to all the backend servers.

decr(key, value=1)¶

Decrement the value for the key, treated as a 64-bit unsigned value.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns:	int – (64 bits) The new value after the decrement.
Raises:	NotFound, NonNumeric, NotImplementedError

delete(key)¶

Delete the key if it exists.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used.
Returns:	Boolean indicating if key was deleted.
Raises:	NotImplementedError, NoAvailableServers

flush_all()¶

Flush the memcache servers.

Note

An attempt is made to connect to all backend servers before running this command.

Raises:	NotImplementedError

get(key, get_cas=False)¶

Retrieve the specified key from a memcache server.

Parameters:	key (str) – The key to lookup in the memcache server. get_cas (bool) – If True, the “cas unique” is queried and the return object has the “cas_unique” attribute set.
Returns:	String, or “value_wrapper” as specified during object creation such as ~memcached2.ValueSuperStr.
Raises:	NoValue, NotImplementedError, NoAvailableServers

incr(key, value=1)¶

Increment the value for the key, treated as a 64-bit unsigned value.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (int (64 bit)) – A numeric value (default=1) to add to the existing value.
Returns:	int – (64 bits) The new value after the increment.
Raises:	NotFound, NonNumeric, NotImplementedError

prepend(key, value)¶

Store data before existing data associated with this key.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (str) – Value stored in memcache server for this key.

replace(key, value, flags=0, exptime=0)¶

Store data, but only if the server already holds data for it.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (str) – Value stored in memcache server for this key. flags (int (32 bits)) – If specified, the same value will be provided on get(). exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value.

set(key, value, flags=0, exptime=0, cas_unique=None)¶

Set a key to the value in the memcache server.

Parameters:

key (str) – Key used to store value in memcache server and hashed to determine which server is used. value (str) – Value stored in memcache server for this key. flags (int (32 bits)) – If specified, the same value will be provided on get(). exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value. cas_unique (int (64 bits)) – If specified as the 64-bit integer from get() with cas_unique=True, the value is only stored if the value has not been updated since the get() call.

stats()¶

Get general statistics about memcache servers.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:	list – The statistics data is a dictionary of key/value pairs representing information about the server. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_items()¶

Get statistics about item storage per slab class from the memcache servers.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:	list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing the slab information. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_settings()¶

Gets statistics about settings (primarily from processing command-line arguments).

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:	list – The statistic information is a dictionary of key/value pairs. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_sizes()¶

Get statistics about object sizes.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Warning

This operation locks the cache while it iterates over all objects. Returns a list of (size,count) tuples received from the server.

Note

An attempt is made to connect to all servers before issuing this command.

Returns:	list – Each statistic is a dictionary of of size:count where the size is rounded up to 32-byte ranges. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

stats_slabs()¶

Gets information about each of the slabs created during memcached runtime. Returns a dictionary of slab IDs, each contains a dictionary of key/value pairs for that slab.

Examples of the results of this function is available in the documentation as Memcache Statistics Examples

Note

An attempt is made to connect to all servers before issuing this command.

Returns:	list – The statistic information is a dictionary keyed by the “slab class”, with the value another dictionary of key/value pairs representing statistic information about each of the slabs created during the memcace runtime. This data is returned as a list of statistics, one item for each server. If the server is not connected, None is returned for its position, otherwise data as mentioned above.

touch(key, exptime)¶

Update the expiration time on an item.

Parameters:	key (str) – Key used to store value in memcache server and hashed to determine which server is used. exptime (int) – If non-zero, it specifies the expriation time, in seconds, for this value. Note that setting exptime=0 causes the item to not expire based on time.
Raises:	NotFound, NotImplementedError, NoAvailableServers

Overview¶

The classes that throw exceptions all tend to raise exceptions that are children of the MemcachedException. For storage-related exceptions, they are children of StoreException, and for retrieval they are children of RetrieveException.

If you use the exception-exposing interfaces (“Memcache()”), will need to catch these exceptions as part of your code. They are thrown on exceptional conditions, read the description of the exceptions for details on when they may be thrown.

In specific error cases that likely indicate bugs in the python-memcached2 module, or where the server replies with unexpected data, the NotImplementedError is raised. These situations are extremely unusual and almost certainly should be reported to the developers of either this python-memcached2 module or the developers of the memcached server you are using. You probably don’t want to catch these

Exceptions¶

class memcached2.MemcachedException¶

Base exception that all other exceptions inherit from. This is never raised directly.

class memcached2.UnknownProtocol¶

An unknown protocol was specified in the memcached URI. Sublcass of MemcachedException.

class memcached2.InvalidURI¶

An error was encountered in parsing the server URI. Subclass of MemcachedException.

class memcached2.ServerDisconnect¶

The connection to the server closed. Subclass of MemcachedException.

class memcached2.NoAvailableServers¶

There are no servers available to cache on, probably because all are disconnected. This exception typically occurs after the code which would do a reconnection is run. Subclass of MemcachedException.

class memcached2.StoreException¶

Base class for storage related exceptions. Never raised directly. Subclass of MemcachedException.

class memcached2.MultiStorageException(message=None, results={})¶

During a SET operation the server returned CLIENT_ERROR. This is probably due to too long of a key being used. Subclass of StoreException.

class memcached2.NotStored¶

Item was not stored, but not due to an error. Normally means the condition for an “add” or “replace” was not met.. Subclass of StoreException.

class memcached2.CASFailure¶

Item you are trying to store with a “cas” command has been modified since you last fetched it (result=EXISTS). Subclass of StoreException.

class memcached2.CASRefreshFailure¶

When trying to refresh a CAS from the memcached, the retrieved value did not match the value sent with the last update. This may happen if another process has updated the value. Subclass of CASFailure.

class memcached2.NotFound¶

Item you are trying to store with a “cas” command does not exist.. Subclass of StoreException.

class memcached2.NonNumeric¶

The item you are trying to incr/decr is not numeric.. Subclass of StoreException.

class memcached2.RetrieveException¶

Base class for retrieve related exceptions. This is never raised directly.. Subclass of MemcachedException.

class memcached2.NoValue¶

Server has no data associated with this key.. Subclass of RetrieveException.