Wrap multiple BagOStuff objects, to implement different caching tiers. More...

Inherits Wikimedia\ObjectCache\BagOStuff.

Collaboration diagram for Wikimedia\ObjectCache\MultiWriteBagOStuff:

[legend]

Public Member Functions
	__construct ( $params)

	add ( $key, $value, $exptime=0, $flags=0)
	Insert an item if it does not already exist.

	changeTTL ( $key, $exptime=0, $flags=0)
	Change the expiration on an item.

	changeTTLMulti (array $keys, $exptime, $flags=0)
	Change the expiration of multiple items.

	delete ( $key, $flags=0)
	Delete an item if it exists.

	deleteMulti (array $keys, $flags=0)
	Delete a batch of items.

	deleteObjectsExpiringBefore ( $timestamp, callable $progress=null, $limit=INF, string $tag=null)
	Delete all objects expiring before a certain date.

	get ( $key, $flags=0)
	Get an item.

	getMulti (array $keys, $flags=0)
	Get a batch of items.

	incrWithInit ( $key, $exptime, $step=1, $init=null, $flags=0)
	Increase the value of the given key (no TTL change) if it exists or create it otherwise.

	lock ( $key, $timeout=6, $exptime=6, $rclass='')
	Acquire an advisory lock on a key string, exclusive to the caller.

	merge ( $key, callable $callback, $exptime=0, $attempts=10, $flags=0)
	Merge changes into the existing cache value (possibly creating a new one)

	set ( $key, $value, $exptime=0, $flags=0)
	Set an item.

	setMockTime (&$time)

	setMulti (array $valueByKey, $exptime=0, $flags=0)
	Set a batch of items.

	unlock ( $key)
	Release an advisory lock on a key string.

Public Member Functions inherited from Wikimedia\ObjectCache\BagOStuff
	__construct (array $params=[])

	clearLastError ()
	Clear the "last error" registry.

	getCurrentTime ()

	getLastError ( $watchPoint=0)
	Get the "last error" registry.

	getLogger ()

	getQoS ( $flag)

	getScopedLock ( $key, $timeout=6, $exptime=30, $rclass='')
	Get a lightweight exclusive self-unlocking lock.

	getSegmentationSize ()

	getSegmentedValueMaxSize ()

	getWithSetCallback ( $key, $exptime, $callback, $flags=0)
	Get an item, regenerating and setting it if not found.

	isKeyGlobal ( $key)
	Check whether a cache key is in the global keyspace.

	makeGlobalKey ( $keygroup,... $components)
	Make a cache key from the given components, in the "global" keyspace.

	makeKey ( $keygroup,... $components)
	Make a cache key from the given components, in the default keyspace.

	setLogger (LoggerInterface $logger)

	watchErrors ()
	Get a "watch point" token that can be used to get the "last error" to occur after now.

Protected Attributes
bool	$asyncWrites = false
	Use async secondary writes.

int[]	$cacheIndexes = []
	List of all backing cache indexes.

BagOStuff[]	$caches
	Backing cache stores in order of highest to lowest tier.

Protected Attributes inherited from Wikimedia\ObjectCache\BagOStuff
callable null	$asyncHandler

int[]	$attrMap = []
	Map of (BagOStuff:ATTR_* constant => BagOStuff:QOS_* constant)

string	$keyspace
	Default keyspace; used by makeKey()

int	$lastError = self::ERR_NONE
	BagOStuff:ERR_* constant of the last error that occurred.

int	$lastErrorId = 0
	Error event sequence number of the last error that occurred.

LoggerInterface	$logger

StatsFactory	$stats

Additional Inherited Members
Public Attributes inherited from Wikimedia\ObjectCache\BagOStuff
const	MAX_CONFLICTS_ONE = 1
	Abort after the first merge conflict.

const	READ_LATEST = 1
	Bitfield constants for get()/getMulti(); these are only advisory.

const	READ_VERIFIED = 2
	Promise that the caller handles detection of staleness.

const	WRITE_ALLOW_SEGMENTS = 16
	Allow partitioning of the value if it is a large string.

const	WRITE_BACKGROUND = 64
	If supported, do not block on write operation completion; instead, treat writes as succesful based on whether they could be buffered.

const	WRITE_CACHE_ONLY = 8
	Bitfield constants for set()/merge(); these are only advisory.

const	WRITE_PRUNE_SEGMENTS = self::WRITE_ALLOW_SEGMENTS
	Delete all the segments if the value is partitioned.

Public Attributes inherited from Wikimedia\LightweightObjectStore\StorageAwareness
const	ATTR_DURABILITY = 2
	Durability of writes; see QOS_DURABILITY_* (higher means stronger)

const	ATTR_EMULATION = 1

const	ERR_NO_RESPONSE = 1
	Storage medium failed to yield a complete response to an operation.

const	ERR_NONE = 0
	No storage medium error.

const	ERR_UNEXPECTED = 3
	Storage medium operation failed due to usage limitations or an I/O error.

const	ERR_UNREACHABLE = 2
	Storage medium could not be reached to establish a connection.

const	QOS_DURABILITY_DISK = 4
	Data is saved to disk and writes do not usually block on fsync()

const	QOS_DURABILITY_NONE = 1
	Data is never saved to begin with (blackhole store)

const	QOS_DURABILITY_RDBMS = 5
	Data is saved to disk and writes usually block on fsync(), like a standard RDBMS.

const	QOS_DURABILITY_SCRIPT = 2
	Data is lost at the end of the current web request or CLI script.

const	QOS_DURABILITY_SERVICE = 3
	Data is lost once the service storing the data restarts.

const	QOS_UNKNOWN = INF
	Generic "unknown" value; useful for comparisons (always "good enough")

Protected Member Functions inherited from Wikimedia\ObjectCache\BagOStuff
	fieldHasFlags ( $field, $flags)

	makeKeyInternal ( $keyspace, $components)
	Make a cache key for the given keyspace and components.

	mergeFlagMaps (array $bags)
	Merge the flag maps of one or more BagOStuff objects into a "lowest common denominator" map.

	proxyCall (string $method, int $arg0Sig, int $resSig, array $genericArgs, BagOStuff $wrapper)
	Call a method on behalf of wrapper BagOStuff instance.

	requireConvertGenericKey ()
	Whether ::proxyCall() must re-encode cache keys before calling read/write methods.

	setLastError ( $error)
	Set the "last error" registry due to a problem encountered during an attempted operation.

Static Protected Attributes inherited from Wikimedia\ObjectCache\BagOStuff
static int	$nextErrorMonitorId = 1
	Next sequence number to use for watch/error events.

Detailed Description

Wrap multiple BagOStuff objects, to implement different caching tiers.

The order of the caches is important. The first tier is considered the primary and highest tier which must handle the majority of the load for reads, and is generally less persistent, smaller, and faster (e.g. evicts data regularly based on demand, keeping fewer keys at a given time). The other caches are consider secondary and lower tiers, which should hold more data and retain it for longer than the primary tier.

Data writes ("set") go to all given BagOStuff caches. If the replication => async option is set, then only the primary write is blocking during the web request, with other writes deferred until after the web response is sent.

Data reads try each cache in the order they are given, until a value is found. When a value is found at a secondary tier, it is automatically copied (back) to the primary tier.

Example: Keep popular data in memcached, with a fallback to a MySQL database. This is how ParserCache is used at Wikimedia Foundation (as of 2024).

$wgObjectCaches['parsercache-multiwrite'] = [
   'class' => 'MultiWriteBagOStuff',
   'caches' => [
     0 => [
       'class' => 'MemcachedPeclBagOStuff',
       'servers' => [ '127.0.0.1:11212' ],
     ],
     1 => [
       'class' => 'SqlBagOStuff',
       'servers' => $parserCacheDbServers,
       'purgePeriod' => 0,
       'tableName' => 'pc',
       'shards' => 256,
       'reportDupes' => false
     ],
   ]
];

If you configure a memcached server for MultiWriteBagOStuff that is the same as the one used for MediaWiki more generally, it is recommended to specify the tier via ObjectCache::getInstance() so that the same object and Memcached connection can be re-used.

$wgObjectCaches['my-memcached'] = [ .. ];
$wgMainCacheType = 'my-memcached';
 
$wgObjectCaches['parsercache-multiwrite'] = [
   'class' => 'MultiWriteBagOStuff',
   'caches' => [
     0 => [
       'factory' => [ 'ObjectCache', 'getInstance' ],
       'args' => [ 'my-memcached' ],
     ],
     1 => [
       'class' => 'SqlBagOStuff',
       'servers' => $parserCacheDbServers,
       'purgePeriod' => 0,
       'tableName' => 'pc',
       'shards' => 256,
       'reportDupes' => false
     ],
   ]
];

The makeKey() method of this class uses an implementation-agnostic encoding. When it forward gets and sets to the other BagOStuff objects, keys are automatically re-encoded. For example, to satisfy the character and length constraints of MemcachedBagOStuff.

Stability: newable

Definition at line 103 of file MultiWriteBagOStuff.php.

Constructor & Destructor Documentation

◆ __construct()

Wikimedia\ObjectCache\MultiWriteBagOStuff::__construct ( $params )

Stability: stable: to call

Parameters

array

$params

caches: A numbered array of either ObjectFactory::getObjectFromSpec arrays yielding BagOStuff objects or direct BagOStuff objects. If using the former, the 'args' field must be set. The first cache is the primary one, being the first to be read in the fallback chain. Writes happen to all stores in the order they are defined. However, lock()/unlock() calls only use the primary store.
replication: Either 'sync' or 'async'. This controls whether writes to secondary stores are deferred when possible. To use 'async' writes requires the 'asyncHandler' option to be set as well. Async writes can increase the chance of some race conditions or cause keys to expire seconds later than expected. It is safe to use for modules when cached values: are immutable, invalidation uses logical TTLs, invalidation uses etag/timestamp validation against the DB, or merge() is used to handle races.

Definition at line 137 of file MultiWriteBagOStuff.php.

References $params, and Wikimedia\ObjectCache\BagOStuff\mergeFlagMaps().

Member Function Documentation

◆ add()

Wikimedia\ObjectCache\MultiWriteBagOStuff::add	(	$key,
		$value,
		$exptime = 0,
		$flags = 0 )

Insert an item if it does not already exist.

Parameters

string	$key
mixed	$value
int	$exptime
int	$flags	Bitfield of BagOStuff::WRITE_* constants (since 1.33)

Returns: bool Success (item created)

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 237 of file MultiWriteBagOStuff.php.

◆ changeTTL()

Wikimedia\ObjectCache\MultiWriteBagOStuff::changeTTL	(	$key,
		$exptime = 0,
		$flags = 0 )

Change the expiration on an item.

If an expiry in the past is given then the key will immediately be expired

For large values written using WRITE_ALLOW_SEGMENTS, this only changes the TTL of the main segment list key. While lowering the TTL of the segment list key has the effect of functionally lowering the TTL of the key, it might leave unused blobs in cache for longer. Raising the TTL of such keys is not effective, since the expiration of a single segment key effectively expires the entire value.

Parameters

string	$key
int	$exptime	TTL or UNIX timestamp
int	$flags	Bitfield of BagOStuff::WRITE_* constants (since 1.33)

Returns: bool Success (item found and updated)

Since: 1.28

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 276 of file MultiWriteBagOStuff.php.

◆ changeTTLMulti()

Wikimedia\ObjectCache\MultiWriteBagOStuff::changeTTLMulti	(	array	$keys,
			$exptime,
			$flags = 0 )

Change the expiration of multiple items.

See also: BagOStuff::changeTTL()

Parameters

string[]	$keys	List of keys
int	$exptime	TTL or UNIX timestamp
int	$flags	Bitfield of BagOStuff::WRITE_* constants (since 1.33)

Returns: bool Success (all items found and updated)

Since: 1.34

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 357 of file MultiWriteBagOStuff.php.

◆ delete()

Wikimedia\ObjectCache\MultiWriteBagOStuff::delete	(		$key,
			$flags = 0 )

Delete an item if it exists.

For large values set with WRITE_ALLOW_SEGMENTS, this only deletes the placeholder key with the segment list. To delete the underlying blobs, include WRITE_ALLOW_SEGMENTS in the flags for delete() as well. While deleting the segment list key has the effect of functionally deleting the key, it leaves unused blobs in storage.

The reason that this is not done automatically, is that to delete underlying blobs, requires first fetching the current segment list. Given that 99% of keys don't use WRITE_ALLOW_SEGMENTS, this would be wasteful.

Parameters

string	$key
int	$flags	Bitfield of BagOStuff::WRITE_* constants

Returns: bool Success (item deleted or not found)

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 227 of file MultiWriteBagOStuff.php.

◆ deleteMulti()

Wikimedia\ObjectCache\MultiWriteBagOStuff::deleteMulti	(	array	$keys,
			$flags = 0 )

Delete a batch of items.

This does not support WRITE_ALLOW_SEGMENTS to avoid excessive read I/O

WRITE_BACKGROUND can be used for bulk deletion where the response is not vital

Parameters

string[]	$keys	List of keys
int	$flags	Bitfield of BagOStuff::WRITE_* constants

Returns: bool Success (items deleted and/or not found)

Since: 1.33

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 347 of file MultiWriteBagOStuff.php.

◆ deleteObjectsExpiringBefore()

Wikimedia\ObjectCache\MultiWriteBagOStuff::deleteObjectsExpiringBefore	(		$timestamp,
		callable	$progress = null,
			$limit = INF,
		string	$tag = null )

Delete all objects expiring before a certain date.

Parameters

string \| int	$timestamp	The reference date in MW or TS_UNIX format
callable \| null	$progress	Optional, a function which will be called regularly during long-running operations with the percentage progress as the first parameter. [optional]
int \| float	$limit	Maximum number of keys to delete [default: INF]
string \| null	$tag	Tag to purge a single shard only. This is only supported when server tags are used in configuration.

Returns: bool Success; false if unimplemented

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 308 of file MultiWriteBagOStuff.php.

◆ get()

Wikimedia\ObjectCache\MultiWriteBagOStuff::get	(		$key,
			$flags = 0 )

Get an item.

If the key includes a deterministic input hash (e.g. the key can only have the correct value) or complete staleness checks are handled by the caller (e.g. nothing relies on the TTL), then the READ_VERIFIED flag should be set. This lets tiered backends know they can safely upgrade a cached value to higher tiers using standard TTLs.

Parameters

string	$key
int	$flags	Bitfield of BagOStuff::READ_* constants [optional]

Returns: mixed Returns false on failure or if the item does not exist

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 166 of file MultiWriteBagOStuff.php.

References Wikimedia\ObjectCache\BagOStuff\fieldHasFlags().

◆ getMulti()

Wikimedia\ObjectCache\MultiWriteBagOStuff::getMulti	(	array	$keys,
			$flags = 0 )

Get a batch of items.

Parameters

string[]	$keys	List of keys
int	$flags	Bitfield; supports READ_LATEST [optional]

Returns: mixed[] Map of (key => value) for existing keys

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 324 of file MultiWriteBagOStuff.php.

◆ incrWithInit()

Wikimedia\ObjectCache\MultiWriteBagOStuff::incrWithInit	(	$key,
		$exptime,
		$step = 1,
		$init = null,
		$flags = 0 )

Increase the value of the given key (no TTL change) if it exists or create it otherwise.

This will create the key with the value $init and TTL $exptime instead if not present. Callers should make sure that both ($init - $step) and $exptime are invariants for all operations to any given key. The value of $init should be at least that of $step.

The new value is returned, except if the WRITE_BACKGROUND flag is given, in which case the handler may choose to return true to indicate that the operation has been dispatched.

Parameters

string	$key	Key built via makeKey() or makeGlobalKey()
int	$exptime	Time-to-live (in seconds) or a UNIX timestamp expiration
int	$step	Amount to increase the key value by [default: 1]
int \| null	$init	Value to initialize the key to if it does not exist [default: $step]
int	$flags	Bit field of class WRITE_* constants [optional]

Returns: int|bool New value (or true if asynchronous) on success; false on failure

Since: 1.24

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 367 of file MultiWriteBagOStuff.php.

◆ lock()

Wikimedia\ObjectCache\MultiWriteBagOStuff::lock	(	$key,
		$timeout = 6,
		$exptime = 6,
		$rclass = '' )

Acquire an advisory lock on a key string, exclusive to the caller.

Parameters

string	$key
int	$timeout	Lock wait timeout; 0 for non-blocking [optional]
int	$exptime	Lock time-to-live in seconds; 1 day maximum [optional]
string	$rclass	If this thread already holds the lock, and the lock was acquired using the same value for this parameter, then return true and use reference counting so that only the unlock() call from the outermost lock() caller actually releases the lock (note that only the outermost time-to-live is used) [optional]

Returns: bool Success

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 286 of file MultiWriteBagOStuff.php.

◆ merge()

Wikimedia\ObjectCache\MultiWriteBagOStuff::merge	(		$key,
		callable	$callback,
			$exptime = 0,
			$attempts = 10,
			$flags = 0 )

Merge changes into the existing cache value (possibly creating a new one)

The callback function returns the new value given the current value (which will be false if not present), and takes the arguments: (this BagOStuff, cache key, current value, TTL). The TTL parameter is reference set to $exptime. It can be overridden in the callback. Nothing is stored nor deleted if the callback returns false.

Parameters

string	$key
callable	$callback	Callback method to be executed
int	$exptime	Either an interval in seconds or a unix timestamp for expiry
int	$attempts	The amount of times to attempt a merge in case of failure
int	$flags	Bitfield of BagOStuff::WRITE_* constants

Returns: bool Success

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 266 of file MultiWriteBagOStuff.php.

◆ set()

Wikimedia\ObjectCache\MultiWriteBagOStuff::set	(	$key,
		$value,
		$exptime = 0,
		$flags = 0 )

Set an item.

Parameters

string	$key
mixed	$value
int	$exptime	Either an interval in seconds or a unix timestamp for expiry
int	$flags	Bitfield of BagOStuff::WRITE_* constants If setting WRITE_ALLOW_SEGMENTS, remember to also set it in any delete() calls.

Returns: bool Success

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 217 of file MultiWriteBagOStuff.php.

◆ setMockTime()

Wikimedia\ObjectCache\MultiWriteBagOStuff::setMockTime ( & $time )

Access: internal: For testing only

Parameters

float | null &$time Mock UNIX timestamp

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 377 of file MultiWriteBagOStuff.php.

◆ setMulti()

Wikimedia\ObjectCache\MultiWriteBagOStuff::setMulti	(	array	$valueByKey,
			$exptime = 0,
			$flags = 0 )

Set a batch of items.

This does not support WRITE_ALLOW_SEGMENTS to avoid excessive read I/O

WRITE_BACKGROUND can be used for bulk insertion where the response is not vital

Parameters

mixed[]	$valueByKey	Map of (key => value)
int	$exptime	Either an interval in seconds or a unix timestamp for expiry
int	$flags	Bitfield of BagOStuff::WRITE_* constants (since 1.33)

Returns: bool Success

Since: 1.24

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 337 of file MultiWriteBagOStuff.php.

◆ unlock()

Wikimedia\ObjectCache\MultiWriteBagOStuff::unlock ( $key )

Release an advisory lock on a key string.

Parameters

string $key

Returns: bool Success

Reimplemented from Wikimedia\ObjectCache\BagOStuff.

Definition at line 297 of file MultiWriteBagOStuff.php.

Member Data Documentation

◆ $asyncWrites

bool Wikimedia\ObjectCache\MultiWriteBagOStuff::$asyncWrites = false

protected

Use async secondary writes.

Definition at line 108 of file MultiWriteBagOStuff.php.

◆ $cacheIndexes

int [] Wikimedia\ObjectCache\MultiWriteBagOStuff::$cacheIndexes = []

protected

List of all backing cache indexes.

Definition at line 110 of file MultiWriteBagOStuff.php.

◆ $caches

BagOStuff [] Wikimedia\ObjectCache\MultiWriteBagOStuff::$caches

protected

Backing cache stores in order of highest to lowest tier.

Definition at line 105 of file MultiWriteBagOStuff.php.

The documentation for this class was generated from the following file:

includes/libs/objectcache/MultiWriteBagOStuff.php

Public Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ add()

◆ changeTTL()

◆ changeTTLMulti()

◆ delete()

◆ deleteMulti()

◆ deleteObjectsExpiringBefore()

◆ get()

◆ getMulti()

◆ incrWithInit()

◆ lock()

◆ merge()

◆ set()

◆ setMockTime()

◆ setMulti()

◆ unlock()

Member Data Documentation

◆ $asyncWrites

◆ $cacheIndexes

◆ $caches