Multi-datacenter aware caching interface. More...

Inheritance diagram for WANObjectCache:

Collaboration diagram for WANObjectCache:

[legend]

Public Member Functions
	__construct (array $params)

	adaptiveTTL ( $mtime, $maxTTL, $minTTL=30, $factor=0.2)
	Get a TTL that is higher for objects that have not changed recently. More...

	clearLastError ()
	Clear the "last error" registry. More...

	clearProcessCache ()
	Clear the in-process caches; useful for testing. More...

	delete ( $key, $ttl=self::HOLDOFF_TTL)
	Purge a key from all datacenters. More...

	get ( $key, &$curTTL=null, array $checkKeys=[], &$asOf=null)
	Fetch the value of a key from cache. More...

	getCheckKeyTime ( $key)
	Fetch the value of a timestamp "check" key. More...

	getLastError ()
	Get the "last error" registered; clearLastError() should be called manually. More...

	getMulti (array $keys, &$curTTLs=[], array $checkKeys=[], array &$asOfs=[])
	Fetch the value of several keys from cache. More...

	getMultiWithSetCallback (ArrayIterator $keyedIds, $ttl, callable $callback, array $opts=[])
	Method to fetch multiple cache keys at once with regeneration. More...

	getMultiWithUnionSetCallback (ArrayIterator $keyedIds, $ttl, callable $callback, array $opts=[])
	Method to fetch/regenerate multiple cache keys at once. More...

	getQoS ( $flag)

	getWarmupKeyMisses ()

	getWithSetCallback ( $key, $ttl, $callback, array $opts=[])
	Method to fetch/regenerate cache keys. More...

	makeGlobalKey ()

	makeKey ()

	makeMultiKeys (array $entities, callable $keyFunc)

	reap ( $key, $purgeTimestamp, &$isStale=false)
	Locally set a key to expire soon if it is stale based on $purgeTimestamp. More...

	reapCheckKey ( $key, $purgeTimestamp, &$isStale=false)
	Locally set a "check" key to expire soon if it is stale based on $purgeTimestamp. More...

	resetCheckKey ( $key)
	Delete a "check" key from all datacenters, invalidating keys that use it. More...

	set ( $key, $value, $ttl=0, array $opts=[])
	Set the value of a key in cache. More...

	setLogger (LoggerInterface $logger)

	touchCheckKey ( $key, $holdoff=self::HOLDOFF_TTL)
	Purge a "check" key from all datacenters, invalidating keys that use it. More...

Static Public Member Functions
static	newEmpty ()
	Get an instance that wraps EmptyBagOStuff. More...

Public Attributes
const	AGE_NEW = 60
	Never consider performing "popularity" refreshes until a key reaches this age. More...

const	CHECK_KEY_TTL = self::TTL_YEAR
	Seconds to keep dependency purge keys around. More...

const	DEFAULT_PURGE_CHANNEL = 'wancache-purge'

const	ERR_NO_RESPONSE = 1

const	ERR_NONE = 0

const	ERR_RELAY = 4

const	ERR_UNEXPECTED = 3

const	ERR_UNREACHABLE = 2

const	FLD_FLAGS = 4

const	FLD_HOLDOFF = 5

const	FLD_TIME = 3

const	FLD_TTL = 2

const	FLD_VALUE = 1

const	FLD_VERSION = 0

const	HIT_RATE_HIGH = 1
	Hits/second for a refresh to be expected within the "popularity" window. More...

const	HOLDOFF_NONE = 0
	Idiom for delete() for "no hold-off". More...

const	HOLDOFF_TTL = 11
	Seconds to tombstone keys on delete() More...

const	HOT_TTR = 900
	The time length of the "popularity" refresh window for hot keys. More...

const	INTERIM_KEY_PREFIX = 'WANCache:i:'

const	LOCK_TSE = 1
	Default time-since-expiry on a miss that makes a key "hot". More...

const	LOCK_TTL = 10
	Seconds to keep lock keys around. More...

const	LOW_TTL = 30
	Default remaining TTL at which to consider pre-emptive regeneration. More...

const	MAX_COMMIT_DELAY = 3
	Max time expected to pass between delete() and DB commit finishing. More...

const	MAX_READ_LAG = 7
	Max replication+snapshot lag before applying TTL_LAGGED or disallowing set() More...

const	MIN_TIMESTAMP_NONE = 0.0
	Idiom for getWithSetCallback() for "no minimum required as-of timestamp". More...

const	MUTEX_KEY_PREFIX = 'WANCache:m:'

const	PC_PRIMARY = 'primary:1000'

const	PURGE_VAL_PREFIX = 'PURGED:'

const	RAMPUP_TTL = 30
	Seconds to ramp up to the "popularity" refresh chance after a key is no longer new. More...

const	TIME_KEY_PREFIX = 'WANCache:t:'

const	TINY_NEGATIVE = -0.000001
	Tiny negative float to use when CTL comes up >= 0 due to clock skew. More...

const	TSE_NONE = -1
	Idiom for getWithSetCallback() callbacks to 'lockTSE' logic. More...

const	TTL_LAGGED = 30
	Max TTL to store keys when a data sourced is lagged. More...

const	TTL_UNCACHEABLE = -1
	Idiom for getWithSetCallback() callbacks to avoid calling set() More...

const	VALUE_KEY_PREFIX = 'WANCache:v:'

const	VERSION = 1
	Cache format version number. More...

const	VFLD_DATA = 'WOC:d'

const	VFLD_VERSION = 'WOC:v'

Public Attributes inherited from IExpiringStore
const	ATTR_EMULATION = 1

const	ATTR_SYNCWRITES = 2

const	QOS_EMULATION_SQL = 1

const	QOS_SYNCWRITES_BE = 2

const	QOS_SYNCWRITES_NONE = 1

const	QOS_SYNCWRITES_QC = 3

const	QOS_SYNCWRITES_SS = 4

const	QOS_UNKNOWN = INF

const	TTL_DAY = 86400

const	TTL_HOUR = 3600

const	TTL_INDEFINITE = 0

const	TTL_MINUTE = 60

const	TTL_MONTH = 2592000

const	TTL_PROC_LONG = 30

const	TTL_PROC_SHORT = 3

const	TTL_WEEK = 604800

const	TTL_YEAR = 31536000

Protected Member Functions
	doGetWithSetCallback ( $key, $ttl, $callback, array $opts, &$asOf=null)
	Do the actual I/O for getWithSetCallback() when needed. More...

	getProcessCache ( $group)

	isValid ( $value, $versioned, $asOf, $minTime)
	Check whether $value is appropriately versioned and not older than $minTime (if set) More...

	makePurgeValue ( $timestamp, $holdoff)

	relayDelete ( $key)
	Do the actual async bus delete of a key. More...

	relayPurge ( $key, $ttl, $holdoff)
	Do the actual async bus purge of a key. More...

	unwrap ( $wrapped, $now)
	Do not use this method outside WANObjectCache. More...

	worthRefreshExpiring ( $curTTL, $lowTTL)
	Check if a key should be regenerated (using random probability) More...

	worthRefreshPopular ( $asOf, $ageNew, $timeTillRefresh, $now)
	Check if a key is due for randomized regeneration due to its popularity. More...

	wrap ( $value, $ttl, $now)
	Do not use this method outside WANObjectCache. More...

Static Protected Member Functions
static	parsePurgeValue ( $value)

static	prefixCacheKeys (array $keys, $prefix)

Protected Attributes
BagOStuff	$cache
	The local datacenter cache. More...

int	$lastRelayError = self::ERR_NONE
	ERR_* constant for the "last error" registry. More...

LoggerInterface	$logger

HashBagOStuff[]	$processCaches = []
	Map of group PHP instance caches. More...

string	$purgeChannel
	Purge channel name. More...

EventRelayer	$purgeRelayer
	Bus that handles purge broadcasts. More...

Private Member Functions
	getNonProcessCachedKeys (array $keys, array $opts)

	getRawKeysForWarmup (array $keys, array $checkKeys)

	processCheckKeys (array $timeKeys, array $wrappedValues, $now)

Private Attributes
int	$callbackDepth = 0
	Callback stack depth for getWithSetCallback() More...

mixed[]	$warmupCache = []
	Temporary warm-up cache. More...

int	$warmupKeyMisses = 0
	Key fetched. More...

Detailed Description

Multi-datacenter aware caching interface.

All operations go to the local datacenter cache, except for delete(), touchCheckKey(), and resetCheckKey(), which broadcast to all datacenters.

This class is intended for caching data from primary stores. If the get() method does not return a value, then the caller should query the new value and backfill the cache using set(). The preferred way to do this logic is through getWithSetCallback(). When querying the store on cache miss, the closest DB replica should be used. Try to avoid heavyweight DB master or quorum reads. When the source data changes, a purge method should be called. Since purges are expensive, they should be avoided. One can do so if:

a) The object cached is immutable; or
b) Validity is checked against the source after get(); or
c) Using a modest TTL is reasonably correct and performant

The simplest purge method is delete().

There are three supported ways to handle broadcasted operations:

a) Configure the 'purge' EventRelayer to point to a valid PubSub endpoint that has subscribed listeners on the cache servers applying the cache updates.
b) Ignore the 'purge' EventRelayer configuration (default is NullEventRelayer) and set up mcrouter as the underlying cache backend, using one of the memcached BagOStuff classes as 'cache'. Use OperationSelectorRoute in the mcrouter settings to configure 'set' and 'delete' operations to go to all DCs via AllAsyncRoute and configure other operations to go to the local DC via PoolRoute (for reference, see https://github.com/facebook/mcrouter/wiki/List-of-Route-Handles).
c) Ignore the 'purge' EventRelayer configuration (default is NullEventRelayer) and set up dynomite as cache middleware between the web servers and either memcached or redis. This will also broadcast all key setting operations, not just purges, which can be useful for cache warming. Writes are eventually consistent via the Dynamo replication model (see https://github.com/Netflix/dynomite).

Broadcasted operations like delete() and touchCheckKey() are done asynchronously in all datacenters this way, though the local one should likely be near immediate.

This means that callers in all datacenters may see older values for however many milliseconds that the purge took to reach that datacenter. As with any cache, this should not be relied on for cases where reads are used to determine writes to source (e.g. non-cache) data stores, except when reading immutable data.

All values are wrapped in metadata arrays. Keys use a "WANCache:" prefix to avoid collisions with keys that are not wrapped as metadata arrays. The prefixes are as follows:

a) "WANCache:v" : used for regular value keys
b) "WANCache:i" : used for temporarily storing values of tombstoned keys
c) "WANCache:t" : used for storing timestamp "check" keys
d) "WANCache:m" : used for temporary mutex keys to avoid cache stampedes

Since: 1.26

Definition at line 80 of file WANObjectCache.php.

Constructor & Destructor Documentation

◆ __construct()

WANObjectCache::__construct ( array $params )

Parameters

array

$params

cache : BagOStuff object for a persistent cache
channels : Map of (action => channel string). Actions include "purge".
relayers : Map of (action => EventRelayer object). Actions include "purge".
logger : LoggerInterface object

Definition at line 181 of file WANObjectCache.php.

References $params, cache, DEFAULT_PURGE_CHANNEL, and setLogger().

Member Function Documentation

◆ adaptiveTTL()

WANObjectCache::adaptiveTTL	(	$mtime,
		$maxTTL,
		$minTTL = `30`,
		$factor = `0.2`
	)

Get a TTL that is higher for objects that have not changed recently.

This is useful for keys that get explicit purges and DB or purge relay lag is a potential concern (especially how it interacts with CDN cache)

Example usage:

// Last-modified time of page
$mtime = wfTimestamp( TS_UNIX, $page->getTimestamp() );
// Get adjusted TTL. If $mtime is 3600 seconds ago and $minTTL/$factor left at
// defaults, then $ttl is 3600 * .2 = 720. If $minTTL was greater than 720, then
// $ttl would be $minTTL. If $maxTTL was smaller than 720, $ttl would be $maxTTL.
$ttl = $cache->adaptiveTTL( $mtime, $cache::TTL_DAY );

Parameters

int \| float	$mtime	UNIX timestamp
int	$maxTTL	Maximum TTL (seconds)
int	$minTTL	Minimum TTL (seconds); Default: 30
float	$factor	Value in the range (0,1); Default: .2

Returns: int Adaptive TTL

Since: 1.28

Definition at line 1427 of file WANObjectCache.php.

◆ clearLastError()

WANObjectCache::clearLastError ( )

final

Clear the "last error" registry.

Definition at line 1381 of file WANObjectCache.php.

References cache, and ERR_NONE.

◆ clearProcessCache()

WANObjectCache::clearProcessCache ( )

Clear the in-process caches; useful for testing.

Since: 1.27

Definition at line 1391 of file WANObjectCache.php.

◆ delete()

WANObjectCache::delete	(	$key,
		$ttl = `self::HOLDOFF_TTL`
	)

final

Purge a key from all datacenters.

This should only be called when the underlying data (being cached) changes in a significant way. This deletes the key and starts a hold-off period where the key cannot be written to for a few seconds (HOLDOFF_TTL). This is done to avoid the following race condition:

a) Some DB data changes and delete() is called on a corresponding key
b) A request refills the key with a stale value from a lagged DB
c) The stale value is stuck there until the key is expired/evicted

This is implemented by storing a special "tombstone" value at the cache key that this class recognizes; get() calls will return false for the key and any set() calls will refuse to replace tombstone values at the key. For this to always avoid stale value writes, the following must hold:

a) Replication lag is bounded to being less than HOLDOFF_TTL; or
b) If lag is higher, the DB will have gone into read-only mode already

Note that set() can also be lag-aware and lower the TTL if it's high.

When using potentially long-running ACID transactions, a good pattern is to use a pre-commit hook to issue the delete. This means that immediately after commit, callers will see the tombstone in cache upon purge relay. It also avoids the following race condition:

a) T1 begins, changes a row, and calls delete()
b) The HOLDOFF_TTL passes, expiring the delete() tombstone
c) T2 starts, reads the row and calls set() due to a cache miss
d) T1 finally commits
e) Stale value is stuck in cache

Example usage:

$dbw->startAtomic( __METHOD__ ); // start of request
... <execute some stuff> ...
// Update the row in the DB
$dbw->update( ... );
$key = $cache->makeKey( 'homes', $homeId );
// Purge the corresponding cache entry just before committing
$dbw->onTransactionPreCommitOrIdle( function() use ( $cache, $key ) {
    $cache->delete( $key );
} );
... <execute some stuff> ...
$dbw->endAtomic( __METHOD__ ); // end of request

The $ttl parameter can be used when purging values that have not actually changed recently. For example, a cleanup script to purge cache entries does not really need a hold-off period, so it can use HOLDOFF_NONE. Likewise for user-requested purge. Note that $ttl limits the effective range of 'lockTSE' for getWithSetCallback().

If called twice on the same key, then the last hold-off TTL takes precedence. For idempotence, the $ttl should not vary for different delete() calls on the same key.

Parameters

string	$key	Cache key
int	$ttl	Tombstone TTL; Default: WANObjectCache::HOLDOFF_TTL

Returns: bool True if the item was purged or not found, false on failure

Definition at line 543 of file WANObjectCache.php.

References relayDelete(), and relayPurge().

Referenced by WANObjectCacheTest\testBusyValue(), WANObjectCacheTest\testGetMultiWithSetCallback(), WANObjectCacheTest\testGetMultiWithUnionSetCallback(), WANObjectCacheTest\testGetWithSetCallback(), WANObjectCacheTest\testGetWithSetCallback_versions(), WANObjectCacheTest\testLockTSE(), and WANObjectCacheTest\testLockTSESlow().

◆ doGetWithSetCallback()

WANObjectCache::doGetWithSetCallback	(		$key,
			$ttl,
			$callback,
		array	$opts,
		&	$asOf = `null`
	)

protected

Do the actual I/O for getWithSetCallback() when needed.

See also: WANObjectCache::getWithSetCallback()

Parameters

string	$key
int	$ttl
callback	$callback
array	$opts	Options map for getWithSetCallback()
float	&$asOf	Cache generation timestamp of returned value [returned]

Returns: mixed

Note: Callable type hints are not used to avoid class-autoloading

Definition at line 939 of file WANObjectCache.php.

References $callbackDepth, $value, AGE_NEW, cache, false, FLD_TIME, HOT_TTR, isValid(), list, MIN_TIMESTAMP_NONE, TSE_NONE, unwrap(), use, worthRefreshExpiring(), worthRefreshPopular(), and wrap().

Referenced by getWithSetCallback().

◆ get()

WANObjectCache::get	(		$key,
		&	$curTTL = `null`,
		array	$checkKeys = `[]`,
		&	$asOf = `null`
	)

final

Fetch the value of a key from cache.

If supplied, $curTTL is set to the remaining TTL (current time left):

a) INF; if $key exists, has no TTL, and is not expired by $checkKeys
b) float (>=0); if $key exists, has a TTL, and is not expired by $checkKeys
c) float (<0); if $key is tombstoned, stale, or existing but expired by $checkKeys
d) null; if $key does not exist and is not tombstoned

If a key is tombstoned, $curTTL will reflect the time since delete().

The timestamp of $key will be checked against the last-purge timestamp of each of $checkKeys. Those $checkKeys not in cache will have the last-purge initialized to the current timestamp. If any of $checkKeys have a timestamp greater than that of $key, then $curTTL will reflect how long ago $key became invalid. Callers can use $curTTL to know when the value is stale. The $checkKeys parameter allow mass invalidations by updating a single key:

a) Each "check" key represents "last purged" of some source data
b) Callers pass in relevant "check" keys as $checkKeys in get()
c) When the source data that "check" keys represent changes, the touchCheckKey() method is called on them

Source data entities might exists in a DB that uses snapshot isolation (e.g. the default REPEATABLE-READ in innoDB). Even for mutable data, that isolation can largely be maintained by doing the following:

a) Calling delete() on entity change and creation, before DB commit
b) Keeping transaction duration shorter than delete() hold-off TTL

However, pre-snapshot values might still be seen if an update was made in a remote datacenter but the purge from delete() didn't relay yet.

Consider using getWithSetCallback() instead of get() and set() cycles. That method has cache slam avoiding features for hot/expensive keys.

Parameters

string	$key	Cache key
mixed	&$curTTL	Approximate TTL left on the key if present/tombstoned [returned]
array	$checkKeys	List of "check" keys
float	&$asOf	UNIX timestamp of cached value; null on failure [returned]

Returns: mixed Value of cache key or false on failure

Definition at line 248 of file WANObjectCache.php.

References getMulti().

Referenced by JobQueueDB\doDeduplicateRootJob(), WANObjectCacheTest\testCheckKeyInitHoldoff(), WANObjectCacheTest\testGetMultiWithSetCallback(), WANObjectCacheTest\testGetMultiWithUnionSetCallback(), WANObjectCacheTest\testGetWithSetCallback(), and WANObjectCacheTest\testLockTSESlow().

◆ getCheckKeyTime()

WANObjectCache::getCheckKeyTime ( $key )

final

Fetch the value of a timestamp "check" key.

The key will be initialized to the current time if not set, so only call this method if this behavior is actually desired

The timestamp can be used to check whether a cached value is valid. Callers should not assume that this returns the same timestamp in all datacenters due to relay delays.

The level of staleness can roughly be estimated from this key, but if the key was evicted from cache, such calculations may show the time since expiry as ~0 seconds.

Note that "check" keys won't collide with other regular keys.

Parameters

string $key

Returns: float UNIX timestamp of the check key

Definition at line 576 of file WANObjectCache.php.

References $time, cache, FLD_TIME, makePurgeValue(), parsePurgeValue(), and string.

Referenced by WANObjectCacheTest\testGetMulti(), WANObjectCacheTest\testGetMultiWithSetCallback(), WANObjectCacheTest\testGetMultiWithUnionSetCallback(), and WANObjectCacheTest\testGetWithSetCallback().

◆ getLastError()

WANObjectCache::getLastError ( )

final

Get the "last error" registered; clearLastError() should be called manually.

Returns: int ERR_* class constant for the "last error" registry

Definition at line 1355 of file WANObjectCache.php.

References $code, $lastRelayError, cache, BagOStuff\ERR_NO_RESPONSE, ERR_NO_RESPONSE, BagOStuff\ERR_NONE, ERR_NONE, ERR_UNEXPECTED, BagOStuff\ERR_UNREACHABLE, and ERR_UNREACHABLE.

◆ getMulti()

WANObjectCache::getMulti	(	array	$keys,
		&	$curTTLs = `[]`,
		array	$checkKeys = `[]`,
		array &	$asOfs = `[]`
	)

final

Fetch the value of several keys from cache.

See also: WANObjectCache::get()

Parameters

array	$keys	List of cache keys
array	&$curTTLs	Map of (key => approximate TTL left) for existing keys [returned]
array	$checkKeys	List of check keys to apply to all $keys. May also apply "check" keys to specific cache keys only by using cache keys as keys in the $checkKeys array.
float[]	&$asOfs	Map of (key => UNIX timestamp of cached value; null on failure)

Returns: array Map of (key => value) for keys that exist and are not tombstoned

Definition at line 270 of file WANObjectCache.php.

References $keys, $value, array(), as, cache, captcha-old\count, false, FLD_HOLDOFF, FLD_TIME, list, prefixCacheKeys(), processCheckKeys(), and unwrap().

Referenced by get(), getMultiWithUnionSetCallback(), WANObjectCacheTest\testGetMulti(), and WANObjectCacheTest\testGetMultiCheckKeys().

◆ getMultiWithSetCallback()

WANObjectCache::getMultiWithSetCallback	(	ArrayIterator	$keyedIds,
			$ttl,
		callable	$callback,
		array	$opts = `[]`
	)

final

Method to fetch multiple cache keys at once with regeneration.

This works the same as getWithSetCallback() except:

a) The $keys argument expects the result of WANObjectCache::makeMultiKeys()
b) The $callback argument expects a callback taking the following arguments:
- $id: ID of an entity to query
- $oldValue : the prior cache value or false if none was present
- &$ttl : a reference to the new value TTL in seconds
- &$setOpts : a reference to options for set() which can be altered
- $oldAsOf : generation UNIX timestamp of $oldValue or null if not present Aside from the additional $id argument, the other arguments function the same way they do in getWithSetCallback().
c) The return value is a map of (cache key => value) in the order of $keyedIds

See also: WANObjectCache::getWithSetCallback(); WANObjectCache::getMultiWithUnionSetCallback()

Example usage:

$rows = $cache->getMultiWithSetCallback(
    // Map of cache keys to entity IDs
    $cache->makeMultiKeys(
        $this->fileVersionIds(),
        function ( $id, WANObjectCache $cache ) {
            return $cache->makeKey( 'file-version', $id );
        }
    ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( $id, $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Load the row for this file
        $row = $dbr->selectRow( 'file', '*', [ 'id' => $id ], __METHOD__ );
 
        return $row ? (array)$row : false;
    },
    [
        // Process cache for 30 seconds
        'pcTTL' => 30,
        // Use a dedicated 500 item cache (initialized on-the-fly)
        'pcGroup' => 'file-versions:500'
    ]
);
$files = array_map( [ __CLASS__, 'newFromRow' ], $rows );

Parameters

ArrayIterator	$keyedIds	Result of WANObjectCache::makeMultiKeys()
int	$ttl	Seconds to live for key updates
callable	$callback	Callback the yields entity regeneration callbacks
array	$opts	Options map

Returns: array Map of (cache key => value) in the same order as $keyedIds

Since: 1.28

Definition at line 1107 of file WANObjectCache.php.

References as, getNonProcessCachedKeys(), getRawKeysForWarmup(), getWithSetCallback(), and use.

Referenced by WANObjectCacheTest\testGetMultiWithSetCallback().

◆ getMultiWithUnionSetCallback()

WANObjectCache::getMultiWithUnionSetCallback	(	ArrayIterator	$keyedIds,
			$ttl,
		callable	$callback,
		array	$opts = `[]`
	)

final

Method to fetch/regenerate multiple cache keys at once.

This works the same as getWithSetCallback() except:

a) The $keys argument expects the result of WANObjectCache::makeMultiKeys()
b) The $callback argument expects a callback returning a map of (ID => new value) for all entity IDs in $regenById and it takes the following arguments:
- $ids: a list of entity IDs to regenerate
- &$ttls: a reference to the (entity ID => new TTL) map
- &$setOpts: a reference to options for set() which can be altered
c) The return value is a map of (cache key => value) in the order of $keyedIds
d) The "lockTSE" and "busyValue" options are ignored

See also: WANObjectCache::getWithSetCallback(); WANObjectCache::getMultiWithSetCallback()

Example usage:

$rows = $cache->getMultiWithUnionSetCallback(
    // Map of cache keys to entity IDs
    $cache->makeMultiKeys(
        $this->fileVersionIds(),
        function ( $id, WANObjectCache $cache ) {
            return $cache->makeKey( 'file-version', $id );
        }
    ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( array $ids, array &$ttls, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Load the rows for these files
        $rows = [];
        $res = $dbr->select( 'file', '*', [ 'id' => $ids ], __METHOD__ );
        foreach ( $res as $row ) {
            $rows[$row->id] = $row;
            $mtime = wfTimestamp( TS_UNIX, $row->timestamp );
            $ttls[$row->id] = $this->adaptiveTTL( $mtime, $ttls[$row->id] );
        }
 
        return $rows;
    },
    ]
);
$files = array_map( [ __CLASS__, 'newFromRow' ], $rows );

Parameters

ArrayIterator	$keyedIds	Result of WANObjectCache::makeMultiKeys()
int	$ttl	Seconds to live for key updates
callable	$callback	Callback the yields entity regeneration callbacks
array	$opts	Options map

Returns: array Map of (cache key => value) in the same order as $keyedIds

Since: 1.30

Definition at line 1193 of file WANObjectCache.php.

References as, getMulti(), getNonProcessCachedKeys(), getRawKeysForWarmup(), getWithSetCallback(), and use.

Referenced by WANObjectCacheTest\testGetMultiWithUnionSetCallback().

◆ getNonProcessCachedKeys()

WANObjectCache::getNonProcessCachedKeys	(	array	$keys,
		array	$opts
	)

private

Parameters

array	$keys
array	$opts

Returns: array List of keys

Definition at line 1712 of file WANObjectCache.php.

References $keys, as, false, getProcessCache(), and PC_PRIMARY.

Referenced by getMultiWithSetCallback(), and getMultiWithUnionSetCallback().

◆ getProcessCache()

WANObjectCache::getProcessCache ( $group )

protected

Parameters

string $group

Returns: HashBagOStuff

Definition at line 1698 of file WANObjectCache.php.

References list.

Referenced by getNonProcessCachedKeys(), and getWithSetCallback().

◆ getQoS()

WANObjectCache::getQoS ( $flag )

Parameters

int $flag ATTR_* class constant

Returns: int QOS_* class constant

Since: 1.28

Definition at line 1400 of file WANObjectCache.php.

References cache.

◆ getRawKeysForWarmup()

WANObjectCache::getRawKeysForWarmup	(	array	$keys,
		array	$checkKeys
	)

private

Parameters

array	$keys
array	$checkKeys

Returns: array Map of (cache key => mixed)

Definition at line 1732 of file WANObjectCache.php.

References $keys, $warmupCache, as, and cache.

Referenced by getMultiWithSetCallback(), and getMultiWithUnionSetCallback().

◆ getWarmupKeyMisses()

WANObjectCache::getWarmupKeyMisses ( )

Returns: int Number of warmup key cache misses last round

Since: 1.30

Definition at line 1445 of file WANObjectCache.php.

References $warmupKeyMisses.

Referenced by WANObjectCacheTest\testGetMultiWithSetCallback(), and WANObjectCacheTest\testGetMultiWithUnionSetCallback().

◆ getWithSetCallback()

WANObjectCache::getWithSetCallback	(		$key,
			$ttl,
			$callback,
		array	$opts = `[]`
	)

final

Method to fetch/regenerate cache keys.

On cache miss, the key will be set to the callback result via set() (unless the callback returns false) and that result will be returned. The arguments supplied to the callback are:

$oldValue : current cache value or false if not present
&$ttl : a reference to the TTL which can be altered
&$setOpts : a reference to options for set() which can be altered
$oldAsOf : generation UNIX timestamp of $oldValue or null if not present (since 1.28)

It is strongly recommended to set the 'lag' and 'since' fields to avoid race conditions that can cause stale values to get stuck at keys. Usually, callbacks ignore the current value, but it can be used to maintain "most recent X" values that come from time or sequence based source data, provided that the "as of" id/time is tracked. Note that preemptive regeneration and $checkKeys can result in a non-false current value.

Usage of $checkKeys is similar to get() and getMulti(). However, rather than the caller having to inspect a "current time left" variable (e.g. $curTTL, $curTTLs), a cache regeneration will automatically be triggered using the callback.

The simplest way to avoid stampedes for hot keys is to use the 'lockTSE' option in $opts. If cache purges are needed, also:

a) Pass $key into $checkKeys
b) Use touchCheckKey( $key ) instead of delete( $key )

Example usage (typical key):

$catInfo = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-attributes', $catId ),
    // Time-to-live (in seconds)
    $cache::TTL_MINUTE,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return $dbr->selectRow( ... );
   }
);

Example usage (key that is expensive and hot):

$catConfig = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'site-cat-config' ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return CatConfig::newFromRow( $dbr->selectRow( ... ) );
    },
    [
        // Calling touchCheckKey() on this key invalidates the cache
        'checkKeys' => [ $cache->makeKey( 'site-cat-config' ) ],
        // Try to only let one datacenter thread manage cache updates at a time
        'lockTSE' => 30,
        // Avoid querying cache servers multiple times in a web request
        'pcTTL' => $cache::TTL_PROC_LONG
    ]
);

Example usage (key with dynamic dependencies):

$catState = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-state', $cat->getId() ),
    // Time-to-live (seconds)
    $cache::TTL_HOUR,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        // Determine new value from the DB
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return CatState::newFromResults( $dbr->select( ... ) );
    },
    [
         // The "check" keys that represent things the value depends on;
         // Calling touchCheckKey() on any of them invalidates the cache
        'checkKeys' => [
            $cache->makeKey( 'sustenance-bowls', $cat->getRoomId() ),
            $cache->makeKey( 'people-present', $cat->getHouseId() ),
            $cache->makeKey( 'cat-laws', $cat->getCityId() ),
        ]
    ]
);

Example usage (hot key holding most recent 100 events):

$lastCatActions = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-last-actions', 100 ),
    // Time-to-live (in seconds)
    10,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Start off with the last cached list
        $list = $oldValue ?: [];
        // Fetch the last 100 relevant rows in descending order;
        // only fetch rows newer than $list[0] to reduce scanning
        $rows = iterator_to_array( $dbr->select( ... ) );
        // Merge them and get the new "last 100" rows
        return array_slice( array_merge( $new, $list ), 0, 100 );
   },
   [
        // Try to only let one datacenter thread manage cache updates at a time
        'lockTSE' => 30,
        // Use a magic value when no cache value is ready rather than stampeding
        'busyValue' => 'computing'
   ]
);

See also: WANObjectCache::get(); WANObjectCache::set()

Parameters

string	$key	Cache key
int	$ttl	Seconds to live for key updates. Special values are: WANObjectCache::TTL_INDEFINITE: Cache forever WANObjectCache::TTL_UNCACHEABLE: Do not cache at all
callable	$callback	Value generation function
array	$opts	Options map: checkKeys: List of "check" keys. The key at $key will be seen as invalid when either touchCheckKey() or resetCheckKey() is called on any of these keys. Default: []. lockTSE: If the key is tombstoned or expired (by checkKeys) less than this many seconds ago, then try to have a single thread handle cache regeneration at any given time. Other threads will try to use stale values if possible. If, on miss, the time since expiration is low, the assumption is that the key is hot and that a stampede is worth avoiding. Setting this above WANObjectCache::HOLDOFF_TTL makes no difference. The higher this is set, the higher the worst-case staleness can be. Use WANObjectCache::TSE_NONE to disable this logic. Default: WANObjectCache::TSE_NONE. busyValue: If no value exists and another thread is currently regenerating it, use this as a fallback value (or a callback to generate such a value). This assures that cache stampedes cannot happen if the value falls out of cache. This can be used as insurance against cache regeneration becoming very slow for some reason (greater than the TTL). Default: null. pcTTL: Process cache the value in this PHP instance for this many seconds. This avoids network I/O when a key is read several times. This will not cache when the callback returns false, however. Note that any purges will not be seen while process cached; since the callback should use replica DBs and they may be lagged or have snapshot isolation anyway, this should not typically matter. Default: WANObjectCache::TTL_UNCACHEABLE. pcGroup: Process cache group to use instead of the primary one. If set, this must be of the format ALPHANUMERIC_NAME:MAX_KEY_SIZE, e.g. "mydata:10". Use this for storing large values, small yet numerous values, or some values with a high cost of eviction. It is generally preferable to use a class constant when setting this value. This has no effect unless pcTTL is used. Default: WANObjectCache::PC_PRIMARY. version: Integer version number. This allows for callers to make breaking changes to how values are stored while maintaining compatability and correct cache purges. New versions are stored alongside older versions concurrently. Avoid storing class objects however, as this reduces compatibility (due to serialization). Default: null. minAsOf: Reject values if they were generated before this UNIX timestamp. This is useful if the source of a key is suspected of having possibly changed recently, and the caller wants any such changes to be reflected. Default: WANObjectCache::MIN_TIMESTAMP_NONE. hotTTR: Expected time-till-refresh for keys that average ~1 hit/second. This should be greater than "ageNew". Keys with higher hit rates will regenerate more often. This is useful when a popular key is changed but the cache purge was delayed or lost. Seldom used keys are rarely affected by this setting, unless an extremely low "hotTTR" value is passed in. Default: WANObjectCache::HOT_TTR. lowTTL: Consider pre-emptive updates when the current TTL (seconds) of the key is less than this. It becomes more likely over time, becoming certain once the key is expired. Default: WANObjectCache::LOW_TTL. ageNew: Consider popularity refreshes only once a key reaches this age in seconds. Default: WANObjectCache::AGE_NEW.

Returns: mixed Value found or written to the key

Note: Options added in 1.28: version, busyValue, hotTTR, ageNew, pcGroup, minAsOf; Callable type hints are not used to avoid class-autoloading

Definition at line 857 of file WANObjectCache.php.

References $value, doGetWithSetCallback(), getProcessCache(), PC_PRIMARY, TTL_UNCACHEABLE, use, and VFLD_DATA.

Referenced by getMultiWithSetCallback(), getMultiWithUnionSetCallback(), WANObjectCacheTest\testBusyValue(), WANObjectCacheTest\testGetWithSetCallback(), WANObjectCacheTest\testGetWithSetCallback_versions(), WANObjectCacheTest\testLockTSE(), and WANObjectCacheTest\testLockTSESlow().

◆ isValid()

WANObjectCache::isValid	(	$value,
		$versioned,
		$asOf,
		$minTime
	)

protected

Check whether $value is appropriately versioned and not older than $minTime (if set)

Parameters

array	$value
bool	$versioned
float	$asOf	The time $value was generated
float	$minTime	The last time the main value was generated (0.0 if unknown)

Returns: bool

Definition at line 1579 of file WANObjectCache.php.

References $value.

Referenced by doGetWithSetCallback().

◆ makeGlobalKey()

WANObjectCache::makeGlobalKey ( )

See also: BagOStuff::makeGlobalKey()

Parameters

string $keys,... Key component

Returns: string

Since: 1.27

Definition at line 1332 of file WANObjectCache.php.

References cache.

◆ makeKey()

WANObjectCache::makeKey ( )

See also: BagOStuff::makeKey()

Parameters

string $keys,... Key component

Returns: string

Since: 1.27

Definition at line 1322 of file WANObjectCache.php.

References cache.

◆ makeMultiKeys()

WANObjectCache::makeMultiKeys	(	array	$entities,
		callable	$keyFunc
	)

Parameters

array	$entities	List of entity IDs
callable	$keyFunc	Callback yielding a key from (entity ID, this WANObjectCache)

Returns: ArrayIterator Iterator yielding (cache key => entity ID) in $entities order

Since: 1.28

Definition at line 1342 of file WANObjectCache.php.

References as.

Referenced by WANObjectCacheTest\testGetMultiWithSetCallback(), and WANObjectCacheTest\testGetMultiWithUnionSetCallback().

◆ makePurgeValue()

WANObjectCache::makePurgeValue	(	$timestamp,
		$holdoff
	)

protected

Parameters

float	$timestamp
int	$holdoff	In seconds

Returns: string Wrapped purge value

Definition at line 1690 of file WANObjectCache.php.

Referenced by getCheckKeyTime(), processCheckKeys(), and relayPurge().

◆ newEmpty()

static WANObjectCache::newEmpty ( )

static

Get an instance that wraps EmptyBagOStuff.

Returns: WANObjectCache

Definition at line 201 of file WANObjectCache.php.

Referenced by FileBackendStore\__construct(), Wikimedia\Rdbms\LBFactory\__construct(), Wikimedia\Rdbms\LoadBalancer\__construct(), ClassicInterwikiLookupTest\testArrayStorage(), ClassicInterwikiLookupTest\testCDBStorage(), ClassicInterwikiLookupTest\testDatabaseStorage(), and ClassicInterwikiLookupTest\testGetAllPrefixes().

◆ parsePurgeValue()

static WANObjectCache::parsePurgeValue ( $value )

staticprotected

Parameters

string $value Wrapped value like "PURGED:<timestamp>:<holdoff>"

Returns: array|bool Array containing a UNIX timestamp (float) and holdoff period (integer), or false if value isn't a valid purge value

Definition at line 1665 of file WANObjectCache.php.

References $value, and HOLDOFF_TTL.

Referenced by getCheckKeyTime(), processCheckKeys(), reapCheckKey(), and unwrap().

◆ prefixCacheKeys()

static WANObjectCache::prefixCacheKeys	(	array	$keys,
			$prefix
	)

staticprotected

Parameters

array	$keys
string	$prefix

Returns: string[]

Definition at line 1651 of file WANObjectCache.php.

References $keys, $res, and as.

Referenced by getMulti().

◆ processCheckKeys()

WANObjectCache::processCheckKeys	(	array	$timeKeys,
		array	$wrappedValues,
			$now
	)

private

Since: 1.27

Parameters

array	$timeKeys	List of prefixed time check keys
array	$wrappedValues
float	$now

Returns: array List of purge value arrays

Definition at line 361 of file WANObjectCache.php.

References as, cache, makePurgeValue(), and parsePurgeValue().

Referenced by getMulti().

◆ reap()

WANObjectCache::reap	(		$key,
			$purgeTimestamp,
		&	$isStale = `false`
	)

Locally set a key to expire soon if it is stale based on $purgeTimestamp.

This sets stale keys' time-to-live at HOLDOFF_TTL seconds, which both avoids broadcasting in mcrouter setups and also avoids races with new tombstones.

Parameters

string	$key	Cache key
int	$purgeTimestamp	UNIX timestamp of purge
bool	&$isStale	Whether the key is stale

Returns: bool Success

Since: 1.28

Definition at line 1269 of file WANObjectCache.php.

References cache, and HOLDOFF_TTL.

◆ reapCheckKey()

WANObjectCache::reapCheckKey	(		$key,
			$purgeTimestamp,
		&	$isStale = `false`
	)

Locally set a "check" key to expire soon if it is stale based on $purgeTimestamp.

Parameters

string	$key	Cache key
int	$purgeTimestamp	UNIX timestamp of purge
bool	&$isStale	Whether the key is stale

Returns: bool Success

Since: 1.28

Definition at line 1298 of file WANObjectCache.php.

References cache, and parsePurgeValue().

◆ relayDelete()

WANObjectCache::relayDelete ( $key )

protected

Do the actual async bus delete of a key.

Parameters

string $key Cache key

Returns: bool Success

Definition at line 1490 of file WANObjectCache.php.

References cache, and ERR_RELAY.

Referenced by delete(), and resetCheckKey().

◆ relayPurge()

WANObjectCache::relayPurge	(	$key,
		$ttl,
		$holdoff
	)

protected

Do the actual async bus purge of a key.

This must set the key to "PURGED:<UNIX timestamp>:<holdoff>"

Parameters

string	$key	Cache key
int	$ttl	How long to keep the tombstone [seconds]
int	$holdoff	HOLDOFF_* constant controlling how long to ignore sets for this key

Returns: bool Success

Definition at line 1459 of file WANObjectCache.php.

References cache, ERR_RELAY, and makePurgeValue().

Referenced by delete(), and touchCheckKey().

◆ resetCheckKey()

WANObjectCache::resetCheckKey ( $key )

final

Delete a "check" key from all datacenters, invalidating keys that use it.

This is similar to touchCheckKey() in that keys using it via get(), getMulti(), or getWithSetCallback() will be invalidated. The differences are:

a) The "check" key will be deleted from all caches and lazily re-initialized when accessed (rather than set everywhere)
b) Thus, dependent keys will be known to be invalid, but not for how long (they are treated as "just" purged), which effects any lockTSE logic in getWithSetCallback()
c) Since "check" keys are initialized only on the server the key hashes to, any temporary ejection of that server will cause the value to be seen as purged as a new server will initialize the "check" key.

The advantage is that this does not place high TTL keys on every cache server, making it better for code that will cache many different keys and either does not use lockTSE or uses a low enough TTL anyway.

This is typically useful for keys with dynamically generated names where a high number of combinations exist.

Note that "check" keys won't collide with other regular keys.

See also: WANObjectCache::get(); WANObjectCache::getWithSetCallback(); WANObjectCache::touchCheckKey()

Parameters

string $key Cache key

Returns: bool True if the item was purged or not found, false on failure

Definition at line 663 of file WANObjectCache.php.

References relayDelete().

◆ set()

WANObjectCache::set	(		$key,
			$value,
			$ttl = `0`,
		array	$opts = `[]`
	)

final

Set the value of a key in cache.

Simply calling this method when source data changes is not valid because the changes do not replicate to the other WAN sites. In that case, delete() should be used instead. This method is intended for use on cache misses.

If the data was read from a snapshot-isolated transactions (e.g. the default REPEATABLE-READ in innoDB), use 'since' to avoid the following race condition:

a) T1 starts
b) T2 updates a row, calls delete(), and commits
c) The HOLDOFF_TTL passes, expiring the delete() tombstone
d) T1 reads the row and calls set() due to a cache miss
e) Stale value is stuck in cache

Setting 'lag' and 'since' help avoids keys getting stuck in stale states.

Example usage:

$dbr = wfGetDB( DB_REPLICA );
$setOpts = Database::getCacheSetOptions( $dbr );
// Fetch the row from the DB
$row = $dbr->selectRow( ... );
$key = $cache->makeKey( 'building', $buildingId );
$cache->set( $key, $row, $cache::TTL_DAY, $setOpts );

Parameters

string	$key	Cache key
mixed	$value
int	$ttl	Seconds to live. Special values are: WANObjectCache::TTL_INDEFINITE: Cache forever
array	$opts	Options map: lag : Seconds of replica DB lag. Typically, this is either the replica DB lag before the data was read or, if applicable, the replica DB lag before the snapshot-isolated transaction the data was read from started. Use false to indicate that replication is not running. Default: 0 seconds since : UNIX timestamp of the data in $value. Typically, this is either the current time the data was read or (if applicable) the time when the snapshot-isolated transaction the data was read from started. Default: 0 seconds pending : Whether this data is possibly from an uncommitted write transaction. Generally, other threads should not see values from the future and they certainly should not see ones that ended up getting rolled back. Default: false lockTSE : if excessive replication/snapshot lag is detected, then store the value with this TTL and flag it as stale. This is only useful if the reads for this key use getWithSetCallback() with "lockTSE" set. Default: WANObjectCache::TSE_NONE staleTTL : Seconds to keep the key around if it is stale. The get()/getMulti() methods return such stale values with a $curTTL of 0, and getWithSetCallback() will call the regeneration callback in such cases, passing in the old value and its as-of time to the callback. This is useful if adaptiveTTL() is used on the old value's as-of time when it is verified as still being correct. Default: 0.

Note: Options added in 1.28: staleTTL

Returns: bool Success

Definition at line 436 of file WANObjectCache.php.

References $cache, cache, FLD_FLAGS, TSE_NONE, TTL_LAGGED, use, and wrap().

Referenced by JobQueueDB\doDeduplicateRootJob(), WANObjectCacheTest\testCheckKeyInitHoldoff(), WANObjectCacheTest\testGetMulti(), and WANObjectCacheTest\testGetMultiCheckKeys().

◆ setLogger()

WANObjectCache::setLogger ( LoggerInterface $logger )

Definition at line 192 of file WANObjectCache.php.

References $logger.

Referenced by __construct().

◆ touchCheckKey()

WANObjectCache::touchCheckKey	(	$key,
		$holdoff = `self::HOLDOFF_TTL`
	)

final

Purge a "check" key from all datacenters, invalidating keys that use it.

This should only be called when the underlying data (being cached) changes in a significant way, and it is impractical to call delete() on all keys that should be changed. When get() is called on those keys, the relevant "check" keys must be supplied for this to work.

The "check" key essentially represents a last-modified field. When touched, the field will be updated on all cache servers. Keys using it via get(), getMulti(), or getWithSetCallback() will be invalidated. It is treated as being HOLDOFF_TTL seconds in the future by those methods to avoid race conditions where dependent keys get updated with stale values (e.g. from a DB replica DB).

This is typically useful for keys with hardcoded names or in some cases dynamically generated names where a low number of combinations exist. When a few important keys get a large number of hits, a high cache time is usually desired as well as "lockTSE" logic. The resetCheckKey() method is less appropriate in such cases since the "time since expiry" cannot be inferred, causing any get() after the reset to treat the key as being "hot", resulting in more stale value usage.

Note that "check" keys won't collide with other regular keys.

See also: WANObjectCache::get(); WANObjectCache::getWithSetCallback(); WANObjectCache::resetCheckKey()

Parameters

string	$key	Cache key
int	$holdoff	HOLDOFF_TTL or HOLDOFF_NONE constant

Returns: bool True if the item was purged or not found, false on failure

Definition at line 628 of file WANObjectCache.php.

References relayPurge().

Referenced by WANObjectCacheTest\testGetMultiCheckKeys().

◆ unwrap()

WANObjectCache::unwrap	(	$wrapped,
		$now
	)

protected

Do not use this method outside WANObjectCache.

Parameters

array \| string \| bool	$wrapped
float	$now	Unix Current timestamp (preferrably pre-query)

Returns: array (mixed; false if absent/tombstoned/invalid, current time left)

Definition at line 1613 of file WANObjectCache.php.

References $flags, FLD_FLAGS, FLD_TIME, FLD_VALUE, and parsePurgeValue().

Referenced by doGetWithSetCallback(), and getMulti().

◆ worthRefreshExpiring()

WANObjectCache::worthRefreshExpiring	(	$curTTL,
		$lowTTL
	)

protected

Check if a key should be regenerated (using random probability)

This returns false if $curTTL >= $lowTTL. Otherwise, the chance of returning true increases steadily from 0% to 100% as the $curTTL moves from $lowTTL to 0 seconds. This handles widely varying levels of cache access traffic.

Parameters

float	$curTTL	Approximate TTL left on the key if present
float	$lowTTL	Consider a refresh when $curTTL is less than this

Returns: bool

Definition at line 1521 of file WANObjectCache.php.

Referenced by doGetWithSetCallback().

◆ worthRefreshPopular()

WANObjectCache::worthRefreshPopular	(	$asOf,
		$ageNew,
		$timeTillRefresh,
		$now
	)

protected

Check if a key is due for randomized regeneration due to its popularity.

This is used so that popular keys can preemptively refresh themselves for higher consistency (especially in the case of purge loss/delay). Unpopular keys can remain in cache with their high nominal TTL. This means popular keys keep good consistency, whether the data changes frequently or not, and long-tail keys get to stay in cache and get hits too. Similar to worthRefreshExpiring(), randomization is used.

Parameters

float	$asOf	UNIX timestamp of the value
int	$ageNew	Age of key when this might recommend refreshing (seconds)
int	$timeTillRefresh	Age of key when it should be refreshed if popular (seconds)
float	$now	The current UNIX timestamp

Returns: bool

Definition at line 1548 of file WANObjectCache.php.

References RAMPUP_TTL.

Referenced by doGetWithSetCallback().

◆ wrap()

WANObjectCache::wrap	(	$value,
		$ttl,
		$now
	)

protected

Do not use this method outside WANObjectCache.

Parameters

mixed	$value
int	$ttl	[0=forever]
float	$now	Unix Current timestamp just before calling set()

Returns: array

Definition at line 1597 of file WANObjectCache.php.

References $value, and VERSION.

Referenced by doGetWithSetCallback(), and set().

Member Data Documentation

◆ $cache

BagOStuff WANObjectCache::$cache

protected

The local datacenter cache.

Definition at line 82 of file WANObjectCache.php.

Referenced by set().

◆ $callbackDepth

int WANObjectCache::$callbackDepth = 0

private

Callback stack depth for getWithSetCallback()

Definition at line 96 of file WANObjectCache.php.

Referenced by doGetWithSetCallback().

◆ $lastRelayError

int WANObjectCache::$lastRelayError = self::ERR_NONE

protected

ERR_* constant for the "last error" registry.

Definition at line 93 of file WANObjectCache.php.

Referenced by getLastError().

◆ $logger

LoggerInterface WANObjectCache::$logger

protected

Definition at line 90 of file WANObjectCache.php.

Referenced by setLogger().

◆ $processCaches

HashBagOStuff [] WANObjectCache::$processCaches = []

protected

Map of group PHP instance caches.

Definition at line 84 of file WANObjectCache.php.

◆ $purgeChannel

string WANObjectCache::$purgeChannel

protected

Purge channel name.

Definition at line 86 of file WANObjectCache.php.

◆ $purgeRelayer

EventRelayer WANObjectCache::$purgeRelayer

protected

Bus that handles purge broadcasts.

Definition at line 88 of file WANObjectCache.php.

◆ $warmupCache

mixed [] WANObjectCache::$warmupCache = []

private

Temporary warm-up cache.

Definition at line 98 of file WANObjectCache.php.

Referenced by getRawKeysForWarmup().

◆ $warmupKeyMisses

int WANObjectCache::$warmupKeyMisses = 0

private

Key fetched.

Definition at line 100 of file WANObjectCache.php.

Referenced by getWarmupKeyMisses().

◆ AGE_NEW

const WANObjectCache::AGE_NEW = 60

Never consider performing "popularity" refreshes until a key reaches this age.

Definition at line 119 of file WANObjectCache.php.

Referenced by doGetWithSetCallback().

◆ CHECK_KEY_TTL

const WANObjectCache::CHECK_KEY_TTL = self::TTL_YEAR

Seconds to keep dependency purge keys around.

Definition at line 110 of file WANObjectCache.php.

◆ DEFAULT_PURGE_CHANNEL

const WANObjectCache::DEFAULT_PURGE_CHANNEL = 'wancache-purge'

Definition at line 172 of file WANObjectCache.php.

Referenced by __construct().

◆ ERR_NO_RESPONSE

const WANObjectCache::ERR_NO_RESPONSE = 1

Definition at line 155 of file WANObjectCache.php.

Referenced by getLastError().

◆ ERR_NONE

const WANObjectCache::ERR_NONE = 0

Definition at line 154 of file WANObjectCache.php.

Referenced by clearLastError(), and getLastError().

◆ ERR_RELAY

const WANObjectCache::ERR_RELAY = 4

Definition at line 158 of file WANObjectCache.php.

Referenced by relayDelete(), and relayPurge().

◆ ERR_UNEXPECTED

const WANObjectCache::ERR_UNEXPECTED = 3

Definition at line 157 of file WANObjectCache.php.

Referenced by getLastError().

◆ ERR_UNREACHABLE

const WANObjectCache::ERR_UNREACHABLE = 2

Definition at line 156 of file WANObjectCache.php.

Referenced by getLastError().

◆ FLD_FLAGS

const WANObjectCache::FLD_FLAGS = 4

Definition at line 148 of file WANObjectCache.php.

Referenced by set(), and unwrap().

◆ FLD_HOLDOFF

const WANObjectCache::FLD_HOLDOFF = 5

Definition at line 149 of file WANObjectCache.php.

Referenced by getMulti().

◆ FLD_TIME

const WANObjectCache::FLD_TIME = 3

Definition at line 147 of file WANObjectCache.php.

Referenced by doGetWithSetCallback(), getCheckKeyTime(), getMulti(), WANObjectCacheTest\testReap(), and unwrap().

◆ FLD_TTL

const WANObjectCache::FLD_TTL = 2

Definition at line 146 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testReap().

◆ FLD_VALUE

const WANObjectCache::FLD_VALUE = 1

Definition at line 145 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testReap(), and unwrap().

◆ FLD_VERSION

const WANObjectCache::FLD_VERSION = 0

Definition at line 144 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testReap().

◆ HIT_RATE_HIGH

const WANObjectCache::HIT_RATE_HIGH = 1

Hits/second for a refresh to be expected within the "popularity" window.

Definition at line 123 of file WANObjectCache.php.

◆ HOLDOFF_NONE

const WANObjectCache::HOLDOFF_NONE = 0

Idiom for delete() for "no hold-off".

Definition at line 134 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testDelete(), and WANObjectCacheTest\testGetMultiCheckKeys().

◆ HOLDOFF_TTL

const WANObjectCache::HOLDOFF_TTL = 11

Seconds to tombstone keys on delete()

Definition at line 107 of file WANObjectCache.php.

Referenced by WANObjectCacheReaper\invoke(), MediaWikiGadgetsDefinitionRepo\loadGadgets(), parsePurgeValue(), and reap().

◆ HOT_TTR

const WANObjectCache::HOT_TTR = 900

The time length of the "popularity" refresh window for hot keys.

Definition at line 121 of file WANObjectCache.php.

Referenced by doGetWithSetCallback().

◆ INTERIM_KEY_PREFIX

const WANObjectCache::INTERIM_KEY_PREFIX = 'WANCache:i:'

Definition at line 161 of file WANObjectCache.php.

◆ LOCK_TSE

const WANObjectCache::LOCK_TSE = 1

Default time-since-expiry on a miss that makes a key "hot".

Definition at line 116 of file WANObjectCache.php.

◆ LOCK_TTL

const WANObjectCache::LOCK_TTL = 10

Seconds to keep lock keys around.

Definition at line 112 of file WANObjectCache.php.

◆ LOW_TTL

const WANObjectCache::LOW_TTL = 30

Default remaining TTL at which to consider pre-emptive regeneration.

Definition at line 114 of file WANObjectCache.php.

◆ MAX_COMMIT_DELAY

const WANObjectCache::MAX_COMMIT_DELAY = 3

Max time expected to pass between delete() and DB commit finishing.

Definition at line 103 of file WANObjectCache.php.

Referenced by CiteHooks\onLinksUpdateComplete().

◆ MAX_READ_LAG

const WANObjectCache::MAX_READ_LAG = 7

Max replication+snapshot lag before applying TTL_LAGGED or disallowing set()

Definition at line 105 of file WANObjectCache.php.

◆ MIN_TIMESTAMP_NONE

const WANObjectCache::MIN_TIMESTAMP_NONE = 0.0

Idiom for getWithSetCallback() for "no minimum required as-of timestamp".

Definition at line 136 of file WANObjectCache.php.

Referenced by doGetWithSetCallback().

◆ MUTEX_KEY_PREFIX

const WANObjectCache::MUTEX_KEY_PREFIX = 'WANCache:m:'

Definition at line 163 of file WANObjectCache.php.

◆ PC_PRIMARY

const WANObjectCache::PC_PRIMARY = 'primary:1000'

Definition at line 170 of file WANObjectCache.php.

Referenced by getNonProcessCachedKeys(), and getWithSetCallback().

◆ PURGE_VAL_PREFIX

const WANObjectCache::PURGE_VAL_PREFIX = 'PURGED:'

Definition at line 165 of file WANObjectCache.php.

Referenced by MessageBlobStoreTest\setUp(), WANObjectCacheTest\testGetWithSeveralCheckKeys(), and WANObjectCacheTest\testReap().

◆ RAMPUP_TTL

const WANObjectCache::RAMPUP_TTL = 30

Seconds to ramp up to the "popularity" refresh chance after a key is no longer new.

Definition at line 125 of file WANObjectCache.php.

Referenced by worthRefreshPopular().

◆ TIME_KEY_PREFIX

const WANObjectCache::TIME_KEY_PREFIX = 'WANCache:t:'

Definition at line 162 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testGetWithSeveralCheckKeys(), and WANObjectCacheTest\testReap().

◆ TINY_NEGATIVE

const WANObjectCache::TINY_NEGATIVE = -0.000001

Tiny negative float to use when CTL comes up >= 0 due to clock skew.

Definition at line 139 of file WANObjectCache.php.

◆ TSE_NONE

const WANObjectCache::TSE_NONE = -1

Idiom for getWithSetCallback() callbacks to 'lockTSE' logic.

Definition at line 130 of file WANObjectCache.php.

Referenced by doGetWithSetCallback(), and set().

◆ TTL_LAGGED

const WANObjectCache::TTL_LAGGED = 30

Max TTL to store keys when a data sourced is lagged.

Definition at line 132 of file WANObjectCache.php.

Referenced by set().

◆ TTL_UNCACHEABLE

const WANObjectCache::TTL_UNCACHEABLE = -1

Idiom for getWithSetCallback() callbacks to avoid calling set()

Definition at line 128 of file WANObjectCache.php.

Referenced by ForeignDBFile\getDescriptionText(), File\getDescriptionText(), GadgetDefinitionNamespaceRepo\getGadget(), getWithSetCallback(), and MediaWikiGadgetsDefinitionRepo\loadGadgets().

◆ VALUE_KEY_PREFIX

const WANObjectCache::VALUE_KEY_PREFIX = 'WANCache:v:'

Definition at line 160 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testGetMultiWithSetCallback(), and WANObjectCacheTest\testReap().

◆ VERSION

const WANObjectCache::VERSION = 1

Cache format version number.

Definition at line 142 of file WANObjectCache.php.

Referenced by WANObjectCacheTest\testReap(), and wrap().

◆ VFLD_DATA

const WANObjectCache::VFLD_DATA = 'WOC:d'

Definition at line 167 of file WANObjectCache.php.

Referenced by getWithSetCallback().

◆ VFLD_VERSION

const WANObjectCache::VFLD_VERSION = 'WOC:v'

Definition at line 168 of file WANObjectCache.php.

The documentation for this class was generated from the following file:

includes/libs/objectcache/WANObjectCache.php

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ adaptiveTTL()

◆ clearLastError()

◆ clearProcessCache()

◆ delete()

◆ doGetWithSetCallback()

◆ get()

◆ getCheckKeyTime()

◆ getLastError()

◆ getMulti()

◆ getMultiWithSetCallback()

◆ getMultiWithUnionSetCallback()

◆ getNonProcessCachedKeys()

◆ getProcessCache()

◆ getQoS()

◆ getRawKeysForWarmup()

◆ getWarmupKeyMisses()

◆ getWithSetCallback()

◆ isValid()

◆ makeGlobalKey()

◆ makeKey()

◆ makeMultiKeys()

◆ makePurgeValue()

◆ newEmpty()

◆ parsePurgeValue()

◆ prefixCacheKeys()

◆ processCheckKeys()

◆ reap()

◆ reapCheckKey()

◆ relayDelete()

◆ relayPurge()

◆ resetCheckKey()

◆ set()

◆ setLogger()

◆ touchCheckKey()

◆ unwrap()

◆ worthRefreshExpiring()

◆ worthRefreshPopular()

◆ wrap()

Member Data Documentation

◆ $cache

◆ $callbackDepth

◆ $lastRelayError

◆ $logger

◆ $processCaches

◆ $purgeChannel

◆ $purgeRelayer

◆ $warmupCache

◆ $warmupKeyMisses

◆ AGE_NEW

◆ CHECK_KEY_TTL

◆ DEFAULT_PURGE_CHANNEL

◆ ERR_NO_RESPONSE

◆ ERR_NONE

◆ ERR_RELAY

◆ ERR_UNEXPECTED

◆ ERR_UNREACHABLE

◆ FLD_FLAGS

◆ FLD_HOLDOFF

◆ FLD_TIME

◆ FLD_TTL

◆ FLD_VALUE

◆ FLD_VERSION

◆ HIT_RATE_HIGH

◆ HOLDOFF_NONE

◆ HOLDOFF_TTL

◆ HOT_TTR

◆ INTERIM_KEY_PREFIX

◆ LOCK_TSE