Multi-datacenter aware caching interface. More...

Inheritance diagram for WANObjectCache:

[legend]

Collaboration diagram for WANObjectCache:

[legend]

Public Member Functions
	__construct (array $params)

	adaptiveTTL ( $mtime, $maxTTL, $minTTL=30, $factor=0.2)
	Get a TTL that is higher for objects that have not changed recently. More...

	clearLastError ()
	Clear the "last error" registry. More...

	clearProcessCache ()
	Clear the in-process caches; useful for testing. More...

	delete ( $key, $ttl=self::HOLDOFF_TTL)
	Purge a key from all datacenters. More...

	get ( $key, &$curTTL=null, array $checkKeys=[], &$info=null)
	Fetch the value of a key from cache. More...

	getCheckKeyTime ( $key)
	Fetch the value of a timestamp "check" key. More...

	getLastError ()
	Get the "last error" registered; clearLastError() should be called manually. More...

	getMulti (array $keys, &$curTTLs=[], array $checkKeys=[], &$info=null)
	Fetch the value of several keys from cache. More...

	getMultiCheckKeyTime (array $keys)
	Fetch the values of each timestamp "check" key. More...

	getMultiWithSetCallback (ArrayIterator $keyedIds, $ttl, callable $callback, array $opts=[])
	Method to fetch multiple cache keys at once with regeneration. More...

	getMultiWithUnionSetCallback (ArrayIterator $keyedIds, $ttl, callable $callback, array $opts=[])
	Method to fetch/regenerate multiple cache keys at once. More...

	getQoS ( $flag)

	getWarmupKeyMisses ()

	getWithSetCallback ( $key, $ttl, $callback, array $opts=[])
	Method to fetch/regenerate cache keys. More...

	hash256 ( $component)
	Hash a possibly long string into a suitable component for makeKey()/makeGlobalKey() More...

	makeGlobalKey ( $class,... $components)

	makeKey ( $class,... $components)

	makeMultiKeys (array $ids, $keyCallback)
	Get an iterator of (cache key => entity ID) for a list of entity IDs. More...

	multiRemap (array $ids, array $res)
	Get an (ID => value) map from (i) a non-unique list of entity IDs, and (ii) the list of corresponding entity values by first appearance of each ID in the entity ID list. More...

	reap ( $key, $purgeTimestamp, &$isStale=false)
	Set a key to soon expire in the local cluster if it pre-dates $purgeTimestamp. More...

	reapCheckKey ( $key, $purgeTimestamp, &$isStale=false)
	Set a "check" key to soon expire in the local cluster if it pre-dates $purgeTimestamp. More...

	resetCheckKey ( $key)
	Delete a "check" key from all datacenters, invalidating keys that use it. More...

	set ( $key, $value, $ttl=self::TTL_INDEFINITE, array $opts=[])
	Set the value of a key in cache. More...

	setLogger (LoggerInterface $logger)

	setMockTime (&$time)

	touchCheckKey ( $key, $holdoff=self::HOLDOFF_TTL)
	Purge a "check" key from all datacenters, invalidating keys that use it. More...

	useInterimHoldOffCaching ( $enabled)
	Enable or disable the use of brief caching for tombstoned keys. More...

Static Public Member Functions
static	newEmpty ()
	Get an instance that wraps EmptyBagOStuff. More...

Protected Member Functions
	getCurrentTime ()

	isValid ( $value, $asOf, $minAsOf, $purgeTime=null)
	Check if $value is not false, versioned (if needed), and not older than $minTime (if set) More...

	relayDelete ( $key)
	Do the actual async bus delete of a key. More...

	relayPurge ( $key, $ttl, $holdoff)
	Do the actual async bus purge of a key. More...

	worthRefreshExpiring ( $curTTL, $lowTTL)
	Check if a key is nearing expiration and thus due for randomized regeneration. More...

	worthRefreshPopular ( $asOf, $ageNew, $timeTillRefresh, $now)
	Check if a key is due for randomized regeneration due to its popularity. More...

Static Protected Member Functions
static	prefixCacheKeys (array $keys, $prefix)

Protected Attributes
callable null	$asyncHandler
	Function that takes a WAN cache callback and runs it later. More...

BagOStuff	$cache
	The local datacenter cache. More...

string	$cluster
	Cache cluster name for mcrouter use. More...

float	$epoch
	Unix timestamp of the oldest possible valid values. More...

LoggerInterface	$logger

bool	$mcrouterAware
	Whether to use mcrouter key prefixing for routing. More...

MapCacheLRU[]	$processCaches = []
	Map of group PHP instance caches. More...

string	$region
	Physical region for mcrouter use. More...

string	$secret
	Stable secret used for hasing long strings into key components. More...

StatsdDataFactoryInterface	$stats

bool	$useInterimHoldOffCaching = true
	Whether to use "interim" caching while keys are tombstoned. More...

Private Member Functions
	checkAndSetCooloff ( $key, $kClass, $elapsed, $lockTSE, $hasLock)

	claimStampedeLock ( $key)

	determineKeyClassForStats ( $key)

	fetchOrRegenerate ( $key, $ttl, $callback, array $opts)
	Do the actual I/O for getWithSetCallback() when needed. More...

	getInterimValue ( $key, $minAsOf)

	getNonProcessCachedMultiKeys (ArrayIterator $keys, array $opts)

	getProcessCache ( $group)

	getProcessCacheKey ( $key, $version)

	getRawKeysForWarmup (array $keys, array $checkKeys)

	isAliveOrInGracePeriod ( $curTTL, $graceTTL)
	Check if a key is fresh or in the grace window and thus due for randomized reuse. More...

	isVolatileValueAgeNegligible ( $age)

	makePurgeValue ( $timestamp, $holdoff)

	parsePurgeValue ( $value)

	processCheckKeys (array $timeKeys, array $wrappedValues, $now)

	resolveBusyValue ( $busyValue)

	resolveCTL ( $value, $curTTL, $curInfo, $touchedCallback)

	resolveTouched ( $value, $lastPurge, $touchedCallback)

	scheduleAsyncRefresh ( $key, $ttl, $callback, $opts)

	setInterimValue ( $key, $value, $ttl, $version, $walltime)

	unwrap ( $wrapped, $now)

	wrap ( $value, $ttl, $version, $now, $walltime)

	yieldStampedeLock ( $key, $hasLock)

Private Attributes
int	$callbackDepth = 0
	Callback stack depth for getWithSetCallback() More...

float null	$wallClockOverride

mixed[]	$warmupCache = []
	Temporary warm-up cache. More...

int	$warmupKeyMisses = 0
	Key fetched. More...

Static Private Attributes
static int	$CHECK_KEY_TTL = self::TTL_YEAR
	Seconds to keep dependency purge keys around. More...

static	$COOLOFF_KEY_PREFIX = 'WANCache:c:'

static int	$COOLOFF_TTL = 1
	Seconds to no-op key set() calls to avoid large blob I/O stampedes. More...

static int	$FLD_FLAGS = 4
	@noinspection PhpUnusedPrivateFieldInspection More...

static int	$FLD_FORMAT_VERSION = 0
	Key to WAN cache version number. More...

static int	$FLD_GENERATION_TIME = 6
	Key to how long it took to generate the value. More...

static int	$FLD_TIME = 3
	Key to the cache timestamp. More...

static int	$FLD_TTL = 2
	Key to the original TTL. More...

static int	$FLD_VALUE = 1
	Key to the cached value. More...

static int	$FLD_VALUE_VERSION = 5
	Key to collection cache version number. More...

static int	$GENERATION_SLOW_SEC = 3
	Consider value generation slow if it takes more than this many seconds. More...

static	$INTERIM_KEY_PREFIX = 'WANCache:i:'

static int	$INTERIM_KEY_TTL = 1
	Seconds to keep interim value keys for tombstoned keys around. More...

static int	$LOCK_TTL = 10
	Seconds to keep lock keys around. More...

static	$MUTEX_KEY_PREFIX = 'WANCache:m:'

static int	$PURGE_HOLDOFF = 1
	Key to the tombstone entry hold-off TTL. More...

static int	$PURGE_TIME = 0
	Key to the tombstone entry timestamp. More...

static	$PURGE_VAL_PREFIX = 'PURGED:'

static int	$RAMPUP_TTL = 30
	Seconds to ramp up the chance of regeneration due to expected time-till-refresh. More...

static int	$RECENT_SET_HIGH_MS = 100
	Max millisecond set() backoff during hold-off (far less than INTERIM_KEY_TTL) More...

static int	$RECENT_SET_LOW_MS = 50
	Min millisecond set() backoff during hold-off (far less than INTERIM_KEY_TTL) More...

static int	$SET_DELAY_HIGH_MS = 50
	Milliseconds of key fetch/validate/regenerate delay prone to set() stampedes. More...

static	$TIME_KEY_PREFIX = 'WANCache:t:'

static float	$TINY_NEGATIVE = -0.000001
	Tiny negative float to use when CTL comes up >= 0 due to clock skew. More...

static float	$TINY_POSTIVE = 0.000001
	Tiny positive float to use when using "minTime" to assert an inequality. More...

static	$VALUE_KEY_PREFIX = 'WANCache:v:'

static int	$VERSION = 1
	Cache format version number. More...

Additional Inherited Members
Public Attributes inherited from IExpiringStore
const	ATTR_EMULATION = 1

const	ATTR_SYNCWRITES = 2

const	ERR_NO_RESPONSE = 1

const	ERR_NONE = 0

const	ERR_UNEXPECTED = 3

const	ERR_UNREACHABLE = 2

const	QOS_EMULATION_SQL = 1

const	QOS_SYNCWRITES_BE = 2

const	QOS_SYNCWRITES_NONE = 1

const	QOS_SYNCWRITES_QC = 3

const	QOS_SYNCWRITES_SS = 4

const	QOS_UNKNOWN = INF

const	TTL_DAY = 86400

const	TTL_HOUR = 3600

const	TTL_INDEFINITE = 0

const	TTL_MINUTE = 60

const	TTL_MONTH = 2592000

const	TTL_PROC_LONG = 30

const	TTL_PROC_SHORT = 3

const	TTL_SECOND = 1

const	TTL_WEEK = 604800

const	TTL_YEAR = 31536000

Detailed Description

Multi-datacenter aware caching interface.

Using WANObjectCache

All operations go to the local datacenter cache, except for delete(), touchCheckKey(), and resetCheckKey(), which broadcast to all datacenters.

This class is intended for caching data from primary stores. If the get() method does not return a value, then the caller should query the new value and backfill the cache using set(). The preferred way to do this logic is through getWithSetCallback(). When querying the store on cache miss, the closest DB replica should be used. Try to avoid heavyweight DB master or quorum reads.

To ensure consumers of the cache see new values in a timely manner, you either need to follow either the validation strategy, or the purge strategy.

The validation strategy refers to the natural avoidance of stale data by one of the following means:

A) The cached value is immutable. If the consumer has access to an identifier that uniquely describes a value, cached value need not change. Instead, the key can change. This also allows all servers to access their perceived current version. This is important in context of multiple deployed versions of your application and/or cross-dc database replication, to ensure deterministic values without oscillation.
B) Validity is checked against the source after get(). This is the inverse of A. The unique identifier is embedded inside the value and validated after on retreival. If outdated, the value is recomputed.
C) The value is cached with a modest TTL (without validation). If value recomputation is reasonably performant, and the value is allowed to be stale, one should consider using TTL only – using the value's age as method of validation.

The purge strategy refers to the the approach whereby your application knows that source data has changed and can react by purging the relevant cache keys. As purges are expensive, this strategy should be avoided if possible. The simplest purge method is delete().

No matter which strategy you choose, callers must not rely on updates or purges being immediately visible to other servers. It should be treated similarly as one would a database replica.

The need for immediate updates should be avoided. If needed, solutions must be sought outside WANObjectCache.

Deploying WANObjectCache

There are two supported ways to set up broadcasted operations:

A) Set up mcrouter as the underlying cache backend, using a memcached BagOStuff class for the 'cache' parameter. The 'region' and 'cluster' parameters must be provided and 'mcrouterAware' must be set to true. Configure mcrouter as follows:
- 1) Use Route Prefixing based on region (datacenter) and cache cluster. See https://github.com/facebook/mcrouter/wiki/Routing-Prefix and https://github.com/facebook/mcrouter/wiki/Multi-cluster-broadcast-setup.
- 2) To increase the consistency of delete() and touchCheckKey() during cache server membership changes, you can use the OperationSelectorRoute to configure 'set' and 'delete' operations to go to all servers in the cache cluster, instead of just one server determined by hashing. See https://github.com/facebook/mcrouter/wiki/List-of-Route-Handles.
B) Set up dynomite as a cache middleware between the web servers and either memcached or redis and use it as the underlying cache backend, using a memcached BagOStuff class for the 'cache' parameter. This will broadcast all key setting operations, not just purges, which can be useful for cache warming. Writes are eventually consistent via the Dynamo replication model. See https://github.com/Netflix/dynomite.

Broadcasted operations like delete() and touchCheckKey() are intended to run immediately in the local datacenter and asynchronously in remote datacenters.

This means that callers in all datacenters may see older values for however many milliseconds that the purge took to reach that datacenter. As with any cache, this should not be relied on for cases where reads are used to determine writes to source (e.g. non-cache) data stores, except when reading immutable data.

All values are wrapped in metadata arrays. Keys use a "WANCache:" prefix to avoid collisions with keys that are not wrapped as metadata arrays. The prefixes are as follows:

a) "WANCache:v" : used for regular value keys
b) "WANCache:i" : used for temporarily storing values of tombstoned keys
c) "WANCache:t" : used for storing timestamp "check" keys
d) "WANCache:m" : used for temporary mutex keys to avoid cache stampedes

Since: 1.26

Definition at line 116 of file WANObjectCache.php.

Constructor & Destructor Documentation

◆ __construct()

WANObjectCache::__construct ( array $params )

Parameters

array

$params

cache : BagOStuff object for a persistent cache
logger : LoggerInterface object
stats : StatsdDataFactoryInterface object
asyncHandler : A function that takes a callback and runs it later. If supplied, whenever a preemptive refresh would be triggered in getWithSetCallback(), the current cache value is still used instead. However, the async-handler function receives a WAN cache callback that, when run, will execute the value generation callback supplied by the getWithSetCallback() caller. The result will be saved as normal. The handler is expected to call the WAN cache callback at an opportune time (e.g. HTTP post-send), though generally within a few 100ms. [optional]
region: the current physical region. This is required when using mcrouter as the backing store proxy. [optional]
cluster: name of the cache cluster used by this WAN cache. The name must be the same in all datacenters; the ("region","cluster") tuple is what distinguishes the counterpart cache clusters among all the datacenter. The contents of https://github.com/facebook/mcrouter/wiki/Config-Files give background on this. This is required when using mcrouter as the backing store proxy. [optional]
mcrouterAware: set as true if mcrouter is the backing store proxy and mcrouter is configured to interpret /<region>/<cluster>/ key prefixes as routes. This requires that "region" and "cluster" are both set above. [optional]
epoch: lowest UNIX timestamp a value/tombstone must have to be valid. [optional]
secret: stable secret used for hashing long strings into key components. [optional]

Definition at line 275 of file WANObjectCache.php.

References setLogger().

Member Function Documentation

◆ adaptiveTTL()

WANObjectCache::adaptiveTTL	(	$mtime,
		$maxTTL,
		$minTTL = `30`,
		$factor = `0.2`
	)

Get a TTL that is higher for objects that have not changed recently.

This is useful for keys that get explicit purges and DB or purge relay lag is a potential concern (especially how it interacts with CDN cache)

Example usage:

// Last-modified time of page
$mtime = wfTimestamp( TS_UNIX, $page->getTimestamp() );
// Get adjusted TTL. If $mtime is 3600 seconds ago and $minTTL/$factor left at
// defaults, then $ttl is 3600 * .2 = 720. If $minTTL was greater than 720, then
// $ttl would be $minTTL. If $maxTTL was smaller than 720, $ttl would be $maxTTL.
$ttl = $cache->adaptiveTTL( $mtime, $cache::TTL_DAY );

Another use case is when there are no applicable "last modified" fields in the DB, and there are too many dependencies for explicit purges to be viable, and the rate of change to relevant content is unstable, and it is highly valued to have the cached value be as up-to-date as possible.

Example usage:

$query = "<some complex query>";
$idListFromComplexQuery = $cache->getWithSetCallback(
    $cache->makeKey( 'complex-graph-query', $hashOfQuery ),
    GraphQueryClass::STARTING_TTL,
    function ( $oldValue, &$ttl, array &$setOpts, $oldAsOf ) use ( $query, $cache ) {
        $gdb = $this->getReplicaGraphDbConnection();
        // Account for any snapshot/replica DB lag
        $setOpts += GraphDatabase::getCacheSetOptions( $gdb );
 
        $newList = iterator_to_array( $gdb->query( $query ) );
        sort( $newList, SORT_NUMERIC ); // normalize
 
        $minTTL = GraphQueryClass::MIN_TTL;
        $maxTTL = GraphQueryClass::MAX_TTL;
        if ( $oldValue !== false ) {
            // Note that $oldAsOf is the last time this callback ran
            $ttl = ( $newList === $oldValue )
                // No change: cache for 150% of the age of $oldValue
                ? $cache->adaptiveTTL( $oldAsOf, $maxTTL, $minTTL, 1.5 )
                // Changed: cache for 50% of the age of $oldValue
                : $cache->adaptiveTTL( $oldAsOf, $maxTTL, $minTTL, .5 );
        }
 
        return $newList;
   },
   [
        // Keep stale values around for doing comparisons for TTL calculations.
        // High values improve long-tail keys hit-rates, though might waste space.
        'staleTTL' => GraphQueryClass::GRACE_TTL
   ]
);

Parameters

int \| float	$mtime	UNIX timestamp
int	$maxTTL	Maximum TTL (seconds)
int	$minTTL	Minimum TTL (seconds); Default: 30
float	$factor	Value in the range (0,1); Default: .2

Returns: int Adaptive TTL

Since: 1.28

Definition at line 2180 of file WANObjectCache.php.

References getCurrentTime().

◆ checkAndSetCooloff()

WANObjectCache::checkAndSetCooloff	(	$key,
		$kClass,
		$elapsed,
		$lockTSE,
		$hasLock
	)

private

Parameters

string	$key
string	$kClass
float	$elapsed	Seconds spent regenerating the value
float	$lockTSE
bool	$hasLock

Returns: bool Whether it is OK to proceed with a key set operation

Definition at line 1508 of file WANObjectCache.php.

References IExpiringStore\ERR_NONE.

Referenced by fetchOrRegenerate().

◆ claimStampedeLock()

WANObjectCache::claimStampedeLock ( $key )

private

Parameters

string $key

Returns: bool Success

Definition at line 1474 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ clearLastError()

WANObjectCache::clearLastError ( )

final

Clear the "last error" registry.

Definition at line 2071 of file WANObjectCache.php.

◆ clearProcessCache()

WANObjectCache::clearProcessCache ( )

Clear the in-process caches; useful for testing.

Since: 1.27

Definition at line 2080 of file WANObjectCache.php.

◆ delete()

WANObjectCache::delete	(	$key,
		$ttl = `self::HOLDOFF_TTL`
	)

final

Purge a key from all datacenters.

This should only be called when the underlying data (being cached) changes in a significant way. This deletes the key and starts a hold-off period where the key cannot be written to for a few seconds (HOLDOFF_TTL). This is done to avoid the following race condition:

a) Some DB data changes and delete() is called on a corresponding key
b) A request refills the key with a stale value from a lagged DB
c) The stale value is stuck there until the key is expired/evicted

This is implemented by storing a special "tombstone" value at the cache key that this class recognizes; get() calls will return false for the key and any set() calls will refuse to replace tombstone values at the key. For this to always avoid stale value writes, the following must hold:

a) Replication lag is bounded to being less than HOLDOFF_TTL; or
b) If lag is higher, the DB will have gone into read-only mode already

Note that set() can also be lag-aware and lower the TTL if it's high.

Be aware that this does not clear the process cache. Even if it did, callbacks used by getWithSetCallback() might still return stale data in the case of either uncommitted or not-yet-replicated changes (callback generally use replica DBs).

When using potentially long-running ACID transactions, a good pattern is to use a pre-commit hook to issue the delete. This means that immediately after commit, callers will see the tombstone in cache upon purge relay. It also avoids the following race condition:

a) T1 begins, changes a row, and calls delete()
b) The HOLDOFF_TTL passes, expiring the delete() tombstone
c) T2 starts, reads the row and calls set() due to a cache miss
d) T1 finally commits
e) Stale value is stuck in cache

Example usage:

$dbw->startAtomic( __METHOD__ ); // start of request
... <execute some stuff> ...
// Update the row in the DB
$dbw->update( ... );
$key = $cache->makeKey( 'homes', $homeId );
// Purge the corresponding cache entry just before committing
$dbw->onTransactionPreCommitOrIdle( function() use ( $cache, $key ) {
    $cache->delete( $key );
} );
... <execute some stuff> ...
$dbw->endAtomic( __METHOD__ ); // end of request

The $ttl parameter can be used when purging values that have not actually changed recently. For example, a cleanup script to purge cache entries does not really need a hold-off period, so it can use HOLDOFF_TTL_NONE. Likewise for user-requested purge. Note that $ttl limits the effective range of 'lockTSE' for getWithSetCallback().

If called twice on the same key, then the last hold-off TTL takes precedence. For idempotence, the $ttl should not vary for different delete() calls on the same key.

Parameters

string	$key	Cache key
int	$ttl	Tombstone TTL; Default: WANObjectCache::HOLDOFF_TTL

Returns: bool True if the item was purged or not found, false on failure

Definition at line 745 of file WANObjectCache.php.

References determineKeyClassForStats(), relayDelete(), and relayPurge().

◆ determineKeyClassForStats()

WANObjectCache::determineKeyClassForStats ( $key )

private

Parameters

string $key String of the format <scope>:<class>[:<class or="" variable>="">]...

Returns: string A collection name to describe this class of key

Definition at line 2489 of file WANObjectCache.php.

Referenced by delete(), fetchOrRegenerate(), resetCheckKey(), and touchCheckKey().

◆ fetchOrRegenerate()

WANObjectCache::fetchOrRegenerate	(		$key,
			$ttl,
			$callback,
		array	$opts
	)

private

Do the actual I/O for getWithSetCallback() when needed.

See also: WANObjectCache::getWithSetCallback()

Parameters

string	$key
int	$ttl
callable	$callback
array	$opts	Options map for getWithSetCallback()

Returns

array Ordered list of the following:

Cached or regenerated value
Cached or regenerated value version number or null if not versioned
Timestamp of the current cached value at the key or null if there is no value

Note: Callable type hints are not used to avoid class-autoloading PhanTypeArraySuspicious

Definition at line 1316 of file WANObjectCache.php.

References $callbackDepth, checkAndSetCooloff(), claimStampedeLock(), determineKeyClassForStats(), getCurrentTime(), getInterimValue(), isAliveOrInGracePeriod(), isValid(), isVolatileValueAgeNegligible(), resolveBusyValue(), resolveCTL(), resolveTouched(), scheduleAsyncRefresh(), setInterimValue(), worthRefreshExpiring(), worthRefreshPopular(), and yieldStampedeLock().

Referenced by getWithSetCallback(), and scheduleAsyncRefresh().

◆ get()

WANObjectCache::get	(		$key,
		&	$curTTL = `null`,
		array	$checkKeys = `[]`,
		&	$info = `null`
	)

final

Fetch the value of a key from cache.

If supplied, $curTTL is set to the remaining TTL (current time left):

a) INF; if $key exists, has no TTL, and is not invalidated by $checkKeys
b) float (>=0); if $key exists, has a TTL, and is not invalidated by $checkKeys
c) float (<0); if $key is tombstoned, stale, or existing but invalidated by $checkKeys
d) null; if $key does not exist and is not tombstoned

If a key is tombstoned, $curTTL will reflect the time since delete().

The timestamp of $key will be checked against the last-purge timestamp of each of $checkKeys. Those $checkKeys not in cache will have the last-purge initialized to the current timestamp. If any of $checkKeys have a timestamp greater than that of $key, then $curTTL will reflect how long ago $key became invalid. Callers can use $curTTL to know when the value is stale. The $checkKeys parameter allow mass invalidations by updating a single key:

a) Each "check" key represents "last purged" of some source data
b) Callers pass in relevant "check" keys as $checkKeys in get()
c) When the source data that "check" keys represent changes, the touchCheckKey() method is called on them

Source data entities might exists in a DB that uses snapshot isolation (e.g. the default REPEATABLE-READ in innoDB). Even for mutable data, that isolation can largely be maintained by doing the following:

a) Calling delete() on entity change and creation, before DB commit
b) Keeping transaction duration shorter than the delete() hold-off TTL
c) Disabling interim key caching via useInterimHoldOffCaching() before get() calls

However, pre-snapshot values might still be seen if an update was made in a remote datacenter but the purge from delete() didn't relay yet.

Consider using getWithSetCallback() instead of get() and set() cycles. That method has cache slam avoiding features for hot/expensive keys.

Pass $info as WANObjectCache::PASS_BY_REF to transform it into a cache key metadata map. This map includes the following metadata:

asOf: UNIX timestamp of the value or null if the key is nonexistant
tombAsOf: UNIX timestamp of the tombstone or null if the key is not tombstoned
lastCKPurge: UNIX timestamp of the highest check key or null if none provided
version: cached value version number or null if the key is nonexistant

Otherwise, $info will transform into the cached value timestamp.

Parameters

string	$key	Cache key made from makeKey() or makeGlobalKey()
mixed \| null	&$curTTL	Approximate TTL left on the key if present/tombstoned [returned]
string[]	$checkKeys	The "check" keys used to validate the value
mixed \| null	&$info	Key info if WANObjectCache::PASS_BY_REF [returned]

Returns: mixed Value of cache key or false on failure

Definition at line 354 of file WANObjectCache.php.

References getMulti().

◆ getCheckKeyTime()

WANObjectCache::getCheckKeyTime ( $key )

final

Fetch the value of a timestamp "check" key.

The key will be initialized to the current time if not set, so only call this method if this behavior is actually desired

The timestamp can be used to check whether a cached value is valid. Callers should not assume that this returns the same timestamp in all datacenters due to relay delays.

The level of staleness can roughly be estimated from this key, but if the key was evicted from cache, such calculations may show the time since expiry as ~0 seconds.

Note that "check" keys won't collide with other regular keys.

Parameters

string $key

Returns: float UNIX timestamp

Definition at line 779 of file WANObjectCache.php.

References getMultiCheckKeyTime().

◆ getCurrentTime()

WANObjectCache::getCurrentTime ( )

protected

Returns: float UNIX timestamp

Definition at line 2626 of file WANObjectCache.php.

References $wallClockOverride.

Referenced by adaptiveTTL(), fetchOrRegenerate(), getInterimValue(), getMulti(), getMultiCheckKeyTime(), relayPurge(), set(), setInterimValue(), and yieldStampedeLock().

◆ getInterimValue()

WANObjectCache::getInterimValue	(	$key,
		$minAsOf
	)

private

Parameters

string	$key
float	$minAsOf	Minimum acceptable "as of" timestamp

Returns: array (cached value or false, cache key metadata map)

Definition at line 1576 of file WANObjectCache.php.

References getCurrentTime(), isValid(), unwrap(), and useInterimHoldOffCaching().

Referenced by fetchOrRegenerate().

◆ getLastError()

WANObjectCache::getLastError ( )

final

Get the "last error" registered; clearLastError() should be called manually.

Returns: int ERR_* class constant for the "last error" registry

Definition at line 2054 of file WANObjectCache.php.

References IExpiringStore\ERR_NO_RESPONSE, IExpiringStore\ERR_NONE, IExpiringStore\ERR_UNEXPECTED, and IExpiringStore\ERR_UNREACHABLE.

◆ getMulti()

WANObjectCache::getMulti	(	array	$keys,
		&	$curTTLs = `[]`,
		array	$checkKeys = `[]`,
		&	$info = `null`
	)

final

Fetch the value of several keys from cache.

Pass $info as WANObjectCache::PASS_BY_REF to transform it into a map of cache keys to cache key metadata maps, each having the same style as those of WANObjectCache::get(). All the cache keys listed in $keys will have an entry.

Othwerwise, $info will transform into a map of (cache key => cached value timestamp). Only the cache keys listed in $keys that exists or are tombstoned will have an entry.

$checkKeys holds the "check" keys used to validate values of applicable keys. The integer indexes hold "check" keys that apply to all of $keys while the string indexes hold "check" keys that only apply to the cache key with that name.

See also: WANObjectCache::get()

Parameters

string[]	$keys	List of cache keys made from makeKey() or makeGlobalKey()
mixed \| null	&$curTTLs	Map of (key => TTL left) for existing/tombstoned keys [returned]
string[] \| string[][]	$checkKeys	Map of (integer or cache key => "check" key(s))
mixed \| null	&$info	Map of (key => info) if WANObjectCache::PASS_BY_REF [returned]

Returns: mixed[] Map of (key => value) for existing values; order of $keys is preserved

Definition at line 397 of file WANObjectCache.php.

References $keys, $PURGE_HOLDOFF, $PURGE_TIME, getCurrentTime(), prefixCacheKeys(), processCheckKeys(), and unwrap().

Referenced by get(), and getMultiWithUnionSetCallback().

◆ getMultiCheckKeyTime()

WANObjectCache::getMultiCheckKeyTime ( array $keys )

final

Fetch the values of each timestamp "check" key.

This works like getCheckKeyTime() except it takes a list of keys and returns a map of timestamps instead of just that of one key

This might be useful if both:

a) a class of entities each depend on hundreds of other entities
b) these other entities are depended upon by millions of entities

The later entities can each use a "check" key to invalidate their dependee entities. However, it is expensive for the former entities to verify against all of the relevant "check" keys during each getWithSetCallback() call. A less expensive approach is to do these verifications only after a "time-till-verify" (TTV) has passed. This is a middle ground between using blind TTLs and using constant verification. The adaptiveTTL() method can be used to dynamically adjust the TTV. Also, the initial TTV can make use of the last-modified times of the dependant entities (either from the DB or the "check" keys).

Example usage:

$value = $cache->getWithSetCallback(
    $cache->makeGlobalKey( 'wikibase-item', $id ),
    self::INITIAL_TTV, // initial time-till-verify
    function ( $oldValue, &$ttv, &$setOpts, $oldAsOf ) use ( $checkKeys, $cache ) {
        $now = microtime( true );
        // Use $oldValue if it passes max ultimate age and "check" key comparisons
        if ( $oldValue &&
            $oldAsOf > max( $cache->getMultiCheckKeyTime( $checkKeys ) ) &&
            ( $now - $oldValue['ctime'] ) <= self::MAX_CACHE_AGE
        ) {
            // Increase time-till-verify by 50% of last time to reduce overhead
            $ttv = $cache->adaptiveTTL( $oldAsOf, self::MAX_TTV, self::MIN_TTV, 1.5 );
            // Unlike $oldAsOf, "ctime" is the ultimate age of the cached data
            return $oldValue;
        }
 
        $mtimes = []; // dependency last-modified times; passed by reference
        $value = [ 'data' => $this->fetchEntityData( $mtimes ), 'ctime' => $now ];
        // Guess time-till-change among the dependencies, e.g. 1/(total change rate)
        $ttc = 1 / array_sum( array_map(
            function ( $mtime ) use ( $now ) {
                return 1 / ( $mtime ? ( $now - $mtime ) : 900 );
            },
            $mtimes
        ) );
        // The time-to-verify should not be overly pessimistic nor optimistic
        $ttv = min( max( $ttc, self::MIN_TTV ), self::MAX_TTV );
 
        return $value;
    },
    [ 'staleTTL' => $cache::TTL_DAY ] // keep around to verify and re-save
);

See also: WANObjectCache::getCheckKeyTime(); WANObjectCache::getWithSetCallback()

Parameters

string[] $keys

Returns: float[] Map of (key => UNIX timestamp)

Since: 1.31

Definition at line 844 of file WANObjectCache.php.

References $keys, $PURGE_TIME, getCurrentTime(), makePurgeValue(), and parsePurgeValue().

Referenced by getCheckKeyTime().

◆ getMultiWithSetCallback()

WANObjectCache::getMultiWithSetCallback	(	ArrayIterator	$keyedIds,
			$ttl,
		callable	$callback,
		array	$opts = `[]`
	)

final

Method to fetch multiple cache keys at once with regeneration.

This works the same as getWithSetCallback() except:

a) The $keys argument expects the result of WANObjectCache::makeMultiKeys()
b) The $callback argument expects a callback taking the following arguments:
- $id: ID of an entity to query
- $oldValue : the prior cache value or false if none was present
- &$ttl : a reference to the new value TTL in seconds
- &$setOpts : a reference to options for set() which can be altered
- $oldAsOf : generation UNIX timestamp of $oldValue or null if not present Aside from the additional $id argument, the other arguments function the same way they do in getWithSetCallback().
c) The return value is a map of (cache key => value) in the order of $keyedIds

See also: WANObjectCache::getWithSetCallback(); WANObjectCache::getMultiWithUnionSetCallback()

Example usage:

$rows = $cache->getMultiWithSetCallback(
    // Map of cache keys to entity IDs
    $cache->makeMultiKeys(
        $this->fileVersionIds(),
        function ( $id ) use ( $cache ) {
            return $cache->makeKey( 'file-version', $id );
        }
    ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( $id, $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Load the row for this file
        $queryInfo = File::getQueryInfo();
        $row = $dbr->selectRow(
            $queryInfo['tables'],
            $queryInfo['fields'],
            [ 'id' => $id ],
            __METHOD__,
            [],
            $queryInfo['joins']
        );
 
        return $row ? (array)$row : false;
    },
    [
        // Process cache for 30 seconds
        'pcTTL' => 30,
        // Use a dedicated 500 item cache (initialized on-the-fly)
        'pcGroup' => 'file-versions:500'
    ]
);
$files = array_map( [ __CLASS__, 'newFromRow' ], $rows );

Parameters

ArrayIterator	$keyedIds	Result of WANObjectCache::makeMultiKeys()
int	$ttl	Seconds to live for key updates
callable	$callback	Callback the yields entity regeneration callbacks
array	$opts	Options map

Returns: mixed[] Map of (cache key => value) in the same order as $keyedIds

Since: 1.28

Definition at line 1686 of file WANObjectCache.php.

References getNonProcessCachedMultiKeys(), getRawKeysForWarmup(), and getWithSetCallback().

◆ getMultiWithUnionSetCallback()

WANObjectCache::getMultiWithUnionSetCallback	(	ArrayIterator	$keyedIds,
			$ttl,
		callable	$callback,
		array	$opts = `[]`
	)

final

Method to fetch/regenerate multiple cache keys at once.

This works the same as getWithSetCallback() except:

a) The $keys argument expects the result of WANObjectCache::makeMultiKeys()
b) The $callback argument expects a callback returning a map of (ID => new value) for all entity IDs in $ids and it takes the following arguments:
- $ids: a list of entity IDs that require cache regeneration
- &$ttls: a reference to the (entity ID => new TTL) map
- &$setOpts: a reference to options for set() which can be altered
c) The return value is a map of (cache key => value) in the order of $keyedIds
d) The "lockTSE" and "busyValue" options are ignored

See also: WANObjectCache::getWithSetCallback(); WANObjectCache::getMultiWithSetCallback()

Example usage:

$rows = $cache->getMultiWithUnionSetCallback(
    // Map of cache keys to entity IDs
    $cache->makeMultiKeys(
        $this->fileVersionIds(),
        function ( $id ) use ( $cache ) {
            return $cache->makeKey( 'file-version', $id );
        }
    ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( array $ids, array &$ttls, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Load the rows for these files
        $rows = [];
        $queryInfo = File::getQueryInfo();
        $res = $dbr->select(
            $queryInfo['tables'],
            $queryInfo['fields'],
            [ 'id' => $ids ],
            __METHOD__,
            [],
            $queryInfo['joins']
        );
        foreach ( $res as $row ) {
            $rows[$row->id] = $row;
            $mtime = wfTimestamp( TS_UNIX, $row->timestamp );
            $ttls[$row->id] = $this->adaptiveTTL( $mtime, $ttls[$row->id] );
        }
 
        return $rows;
    },
    ]
);
$files = array_map( [ __CLASS__, 'newFromRow' ], $rows );

Parameters

ArrayIterator	$keyedIds	Result of WANObjectCache::makeMultiKeys()
int	$ttl	Seconds to live for key updates
callable	$callback	Callback the yields entity regeneration callbacks
array	$opts	Options map

Returns: mixed[] Map of (cache key => value) in the same order as $keyedIds

Since: 1.30

Definition at line 1777 of file WANObjectCache.php.

References getMulti(), getNonProcessCachedMultiKeys(), getRawKeysForWarmup(), and getWithSetCallback().

◆ getNonProcessCachedMultiKeys()

WANObjectCache::getNonProcessCachedMultiKeys	(	ArrayIterator	$keys,
		array	$opts
	)

private

Parameters

ArrayIterator	$keys
array	$opts

Returns: string[] Map of (ID => cache key)

Definition at line 2570 of file WANObjectCache.php.

References $keys, and getProcessCache().

Referenced by getMultiWithSetCallback(), and getMultiWithUnionSetCallback().

◆ getProcessCache()

WANObjectCache::getProcessCache ( $group )

private

Parameters

string $group

Returns: MapCacheLRU

Definition at line 2544 of file WANObjectCache.php.

Referenced by getNonProcessCachedMultiKeys(), and getWithSetCallback().

◆ getProcessCacheKey()

WANObjectCache::getProcessCacheKey	(	$key,
		$version
	)

private

Parameters

string	$key
int	$version

Returns: string

Definition at line 2561 of file WANObjectCache.php.

Referenced by getWithSetCallback().

◆ getQoS()

WANObjectCache::getQoS ( $flag )

Parameters

int $flag ATTR_* class constant

Returns: int QOS_* class constant

Since: 1.28

Definition at line 2113 of file WANObjectCache.php.

Referenced by MWLBFactory\injectObjectCaches().

◆ getRawKeysForWarmup()

WANObjectCache::getRawKeysForWarmup	(	array	$keys,
		array	$checkKeys
	)

private

Parameters

string[]	$keys
string[] \| string[][]	$checkKeys

Returns: string[] List of cache keys

Definition at line 2592 of file WANObjectCache.php.

References $keys, and $warmupCache.

Referenced by getMultiWithSetCallback(), and getMultiWithUnionSetCallback().

◆ getWarmupKeyMisses()

WANObjectCache::getWarmupKeyMisses ( )

final

Returns: int Number of warmup key cache misses last round

Since: 1.30

Definition at line 2198 of file WANObjectCache.php.

References $warmupKeyMisses.

◆ getWithSetCallback()

WANObjectCache::getWithSetCallback	(		$key,
			$ttl,
			$callback,
		array	$opts = `[]`
	)

final

Method to fetch/regenerate cache keys.

On cache miss, the key will be set to the callback result via set() (unless the callback returns false) and that result will be returned. The arguments supplied to the callback are:

$oldValue : current cache value or false if not present
&$ttl : a reference to the TTL which can be altered
&$setOpts : a reference to options for set() which can be altered
$oldAsOf : generation UNIX timestamp of $oldValue or null if not present (since 1.28)

It is strongly recommended to set the 'lag' and 'since' fields to avoid race conditions that can cause stale values to get stuck at keys. Usually, callbacks ignore the current value, but it can be used to maintain "most recent X" values that come from time or sequence based source data, provided that the "as of" id/time is tracked. Note that preemptive regeneration and $checkKeys can result in a non-false current value.

Usage of $checkKeys is similar to get() and getMulti(). However, rather than the caller having to inspect a "current time left" variable (e.g. $curTTL, $curTTLs), a cache regeneration will automatically be triggered using the callback.

The $ttl argument and "hotTTR" option (in $opts) use time-dependant randomization to avoid stampedes. Keys that are slow to regenerate and either heavily used or subject to explicit (unpredictable) purges, may need additional mechanisms. The simplest way to avoid stampedes for such keys is to use 'lockTSE' (in $opts). If explicit purges are needed, also:

a) Pass $key into $checkKeys
b) Use touchCheckKey( $key ) instead of delete( $key )

Example usage (typical key):

$catInfo = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-attributes', $catId ),
    // Time-to-live (in seconds)
    $cache::TTL_MINUTE,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return $dbr->selectRow( ... );
   }
);

Example usage (key that is expensive and hot):

$catConfig = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'site-cat-config' ),
    // Time-to-live (in seconds)
    $cache::TTL_DAY,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return CatConfig::newFromRow( $dbr->selectRow( ... ) );
    },
    [
        // Calling touchCheckKey() on this key invalidates the cache
        'checkKeys' => [ $cache->makeKey( 'site-cat-config' ) ],
        // Try to only let one datacenter thread manage cache updates at a time
        'lockTSE' => 30,
        // Avoid querying cache servers multiple times in a web request
        'pcTTL' => $cache::TTL_PROC_LONG
    ]
);

Example usage (key with dynamic dependencies):

$catState = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-state', $cat->getId() ),
    // Time-to-live (seconds)
    $cache::TTL_HOUR,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        // Determine new value from the DB
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return CatState::newFromResults( $dbr->select( ... ) );
    },
    [
         // The "check" keys that represent things the value depends on;
         // Calling touchCheckKey() on any of them invalidates the cache
        'checkKeys' => [
            $cache->makeKey( 'sustenance-bowls', $cat->getRoomId() ),
            $cache->makeKey( 'people-present', $cat->getHouseId() ),
            $cache->makeKey( 'cat-laws', $cat->getCityId() ),
        ]
    ]
);

Example usage (key that is expensive with too many DB dependencies for "check keys"):

$catToys = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-toys', $catId ),
    // Time-to-live (seconds)
    $cache::TTL_HOUR,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        // Determine new value from the DB
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        return CatToys::newFromResults( $dbr->select( ... ) );
    },
    [
         // Get the highest timestamp of any of the cat's toys
        'touchedCallback' => function ( $value ) use ( $catId ) {
            $dbr = wfGetDB( DB_REPLICA );
            $ts = $dbr->selectField( 'cat_toys', 'MAX(ct_touched)', ... );
 
            return wfTimestampOrNull( TS_UNIX, $ts );
        },
        // Avoid DB queries for repeated access
        'pcTTL' => $cache::TTL_PROC_SHORT
    ]
);

Example usage (hot key holding most recent 100 events):

$lastCatActions = $cache->getWithSetCallback(
    // Key to store the cached value under
    $cache->makeKey( 'cat-last-actions', 100 ),
    // Time-to-live (in seconds)
    10,
    // Function that derives the new key value
    function ( $oldValue, &$ttl, array &$setOpts ) {
        $dbr = wfGetDB( DB_REPLICA );
        // Account for any snapshot/replica DB lag
        $setOpts += Database::getCacheSetOptions( $dbr );
 
        // Start off with the last cached list
        $list = $oldValue ?: [];
        // Fetch the last 100 relevant rows in descending order;
        // only fetch rows newer than $list[0] to reduce scanning
        $rows = iterator_to_array( $dbr->select( ... ) );
        // Merge them and get the new "last 100" rows
        return array_slice( array_merge( $new, $list ), 0, 100 );
   },
   [
        // Try to only let one datacenter thread manage cache updates at a time
        'lockTSE' => 30,
        // Use a magic value when no cache value is ready rather than stampeding
        'busyValue' => 'computing'
   ]
);

Example usage (key holding an LRU subkey:value map; this can avoid flooding cache with keys for an unlimited set of (constraint,situation) pairs, thereby avoiding elevated cache evictions and wasted memory):

$catSituationTolerabilityCache = $this->cache->getWithSetCallback(
    // Group by constraint ID/hash, cat family ID/hash, or something else useful
    $this->cache->makeKey( 'cat-situation-tolerability-checks', $groupKey ),
    WANObjectCache::TTL_DAY, // rarely used groups should fade away
    // The $scenarioKey format is $constraintId:<ID/hash of $situation>
    function ( $cacheMap ) use ( $scenarioKey, $constraintId, $situation ) {
        $lruCache = MapCacheLRU::newFromArray( $cacheMap ?: [], self::CACHE_SIZE );
        $result = $lruCache->get( $scenarioKey ); // triggers LRU bump if present
        if ( $result === null || $this->isScenarioResultExpired( $result ) ) {
            $result = $this->checkScenarioTolerability( $constraintId, $situation );
            $lruCache->set( $scenarioKey, $result, 3 / 8 );
        }
        // Save the new LRU cache map and reset the map's TTL
        return $lruCache->toArray();
    },
    [
        // Once map is > 1 sec old, consider refreshing
        'ageNew' => 1,
        // Update within 5 seconds after "ageNew" given a 1hz cache check rate
        'hotTTR' => 5,
        // Avoid querying cache servers multiple times in a request; this also means
        // that a request can only alter the value of any given constraint key once
        'pcTTL' => WANObjectCache::TTL_PROC_LONG
    ]
);
$tolerability = isset( $catSituationTolerabilityCache[$scenarioKey] )
    ? $catSituationTolerabilityCache[$scenarioKey]
    : $this->checkScenarioTolerability( $constraintId, $situation );

See also: WANObjectCache::get(); WANObjectCache::set()

Parameters

string	$key	Cache key made from makeKey() or makeGlobalKey()
int	$ttl	Seconds to live for key updates. Special values are: WANObjectCache::TTL_INDEFINITE: Cache forever (subject to LRU-style evictions) WANObjectCache::TTL_UNCACHEABLE: Do not cache (if the key exists, it is not deleted)
callable	$callback	Value generation function
array	$opts	Options map: checkKeys: List of "check" keys. The key at $key will be seen as stale when either touchCheckKey() or resetCheckKey() is called on any of the keys in this list. This is useful if thousands or millions of keys depend on the same entity. The entity can simply have its "check" key updated whenever the entity is modified. Default: []. graceTTL: If the key is invalidated (by "checkKeys"/"touchedCallback") less than this many seconds ago, consider reusing the stale value. The odds of a refresh becomes more likely over time, becoming certain once the grace period is reached. This can reduce traffic spikes when millions of keys are compared to the same "check" key and touchCheckKey() or resetCheckKey() is called on that "check" key. This option is not useful for avoiding traffic spikes in the case of the key simply expiring on account of its TTL (use "lowTTL" instead). Default: WANObjectCache::GRACE_TTL_NONE. lockTSE: If the key is tombstoned or invalidated (by "checkKeys"/"touchedCallback") less than this many seconds ago, try to have a single thread handle cache regeneration at any given time. Other threads will use stale values if possible. If, on miss, the time since expiration is low, the assumption is that the key is hot and that a stampede is worth avoiding. Note that if the key falls out of cache then concurrent threads will all run the callback on cache miss until the value is saved in cache. The only stampede protection in that case is from duplicate cache sets when the callback takes longer than WANObjectCache::SET_DELAY_HIGH_MS milliseconds; consider using "busyValue" if such stampedes are a problem. Note that the higher "lockTSE" is set, the higher the worst-case staleness of returned values can be. Also note that this option does not by itself handle the case of the key simply expiring on account of its TTL, so make sure that "lowTTL" is not disabled when using this option. Avoid combining this option with delete() as it can always cause a stampede due to their being no stale value available until after a thread completes the callback. Use WANObjectCache::TSE_NONE to disable this logic. Default: WANObjectCache::TSE_NONE. busyValue: Specify a placeholder value to use when no value exists and another thread is currently regenerating it. This assures that cache stampedes cannot happen if the value falls out of cache. This also mitigates stampedes when value regeneration becomes very slow (greater than $ttl/"lowTTL"). If this is a closure, then it will be invoked to get the placeholder when needed. Default: null. pcTTL: Process cache the value in this PHP instance for this many seconds. This avoids network I/O when a key is read several times. This will not cache when the callback returns false, however. Note that any purges will not be seen while process cached; since the callback should use replica DBs and they may be lagged or have snapshot isolation anyway, this should not typically matter. Default: WANObjectCache::TTL_UNCACHEABLE. pcGroup: Process cache group to use instead of the primary one. If set, this must be of the format ALPHANUMERIC_NAME:MAX_KEY_SIZE, e.g. "mydata:10". Use this for storing large values, small yet numerous values, or some values with a high cost of eviction. It is generally preferable to use a class constant when setting this value. This has no effect unless pcTTL is used. Default: WANObjectCache::PC_PRIMARY. version: Integer version number. This lets callers make breaking changes to the format of cached values without causing problems for sites that use non-instantaneous code deployments. Old and new code will recognize incompatible versions and purges from both old and new code will been seen by each other. When this method encounters an incompatibly versioned value at the provided key, a "variant key" will be used for reading from and saving to cache. The variant key is specific to the key and version number provided to this method. If the variant key value is older than that of the provided key, or the provided key is non-existant, then the variant key will be seen as non-existant. Therefore, delete() calls invalidate the provided key's variant keys. The "checkKeys" and "touchedCallback" options still apply to variant keys as usual. Avoid storing class objects, as this reduces compatibility (due to serialization). Default: null. minAsOf: Reject values if they were generated before this UNIX timestamp. This is useful if the source of a key is suspected of having possibly changed recently, and the caller wants any such changes to be reflected. Default: WANObjectCache::MIN_TIMESTAMP_NONE. hotTTR: Expected time-till-refresh (TTR) in seconds for keys that average ~1 hit per second (e.g. 1Hz). Keys with a hit rate higher than 1Hz will refresh sooner than this TTR and vise versa. Such refreshes won't happen until keys are "ageNew" seconds old. This uses randomization to avoid triggering cache stampedes. The TTR is useful at reducing the impact of missed cache purges, since the effect of a heavily referenced key being stale is worse than that of a rarely referenced key. Unlike simply lowering $ttl, seldomly used keys are largely unaffected by this option, which makes it possible to have a high hit rate for the "long-tail" of less-used keys. Default: WANObjectCache::HOT_TTR. lowTTL: Consider pre-emptive updates when the current TTL (seconds) of the key is less than this. It becomes more likely over time, becoming certain once the key is expired. This helps avoid cache stampedes that might be triggered due to the key expiring. Default: WANObjectCache::LOW_TTL. ageNew: Consider popularity refreshes only once a key reaches this age in seconds. Default: WANObjectCache::AGE_NEW. staleTTL: Seconds to keep the key around if it is stale. This means that on cache miss the callback may get $oldValue/$oldAsOf values for keys that have already been expired for this specified time. This is useful if adaptiveTTL() is used on the old value's as-of time when it is verified as still being correct. Default: WANObjectCache::STALE_TTL_NONE touchedCallback: A callback that takes the current value and returns a UNIX timestamp indicating the last time a dynamic dependency changed. Null can be returned if there are no relevant dependency changes to check. This can be used to check against things like last-modified times of files or DB timestamp fields. This should generally not be used for small and easily queried values in a DB if the callback itself ends up doing a similarly expensive DB query to check a timestamp. Usages of this option makes the most sense for values that are moderately to highly expensive to regenerate and easy to query for dependency timestamps. The use of "pcTTL" reduces timestamp queries. Default: null.

-param array{checkKeys?:string[],graceTTL?:int,lockTSE?:int,busyValue?:mixed,pcTTL?:int,pcGroup?:string,version?:int,minAsOf?:int,hotTTR?:int,lowTTL?:int,ageNew?:int,staleTTL?:int,touchedCallback?:callable} $opts

Returns: mixed Value found or written to the key

Note: Options added in 1.28: version, busyValue, hotTTR, ageNew, pcGroup, minAsOf; Options added in 1.31: staleTTL, graceTTL; Options added in 1.33: touchedCallback; Callable type hints are not used to avoid class-autoloading

Definition at line 1261 of file WANObjectCache.php.

References $res, fetchOrRegenerate(), getProcessCache(), getProcessCacheKey(), and makeGlobalKey().

Referenced by getMultiWithSetCallback(), and getMultiWithUnionSetCallback().

◆ hash256()

WANObjectCache::hash256 ( $component )

Hash a possibly long string into a suitable component for makeKey()/makeGlobalKey()

Parameters

string $component A raw component used in building a cache key

Returns: string 64 character HMAC using a stable secret for public collision resistance

Since: 1.34

Definition at line 1927 of file WANObjectCache.php.

◆ isAliveOrInGracePeriod()

WANObjectCache::isAliveOrInGracePeriod	(	$curTTL,
		$graceTTL
	)

private

Check if a key is fresh or in the grace window and thus due for randomized reuse.

If $curTTL > 0 (e.g. not expired) this returns true. Otherwise, the chance of returning true decrease steadily from 100% to 0% as the |$curTTL| moves from 0 to $graceTTL seconds. This handles widely varying levels of cache access traffic.

If $curTTL <= -$graceTTL (e.g. already expired), then this returns false.

Parameters

float	$curTTL	Approximate TTL left on the key if present
int	$graceTTL	Consider using stale values if $curTTL is greater than this

Returns: bool

Definition at line 2287 of file WANObjectCache.php.

References worthRefreshExpiring().

Referenced by fetchOrRegenerate().

◆ isValid()

WANObjectCache::isValid	(	$value,
		$asOf,
		$minAsOf,
		$purgeTime = `null`
	)

protected

Check if $value is not false, versioned (if needed), and not older than $minTime (if set)

Parameters

array \| bool	$value
float	$asOf	The time $value was generated
float	$minAsOf	Minimum acceptable "as of" timestamp
float \| null	$purgeTime	The last time the value was invalidated

Returns: bool

Definition at line 2384 of file WANObjectCache.php.

Referenced by fetchOrRegenerate(), and getInterimValue().

◆ isVolatileValueAgeNegligible()

WANObjectCache::isVolatileValueAgeNegligible ( $age )

private

Parameters

float $age Age of volatile/interim key in seconds

Returns: bool Whether the age of a volatile value is negligible

Definition at line 1496 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ makeGlobalKey()

WANObjectCache::makeGlobalKey	(	$class,
		$components
	)

See also: BagOStuff::makeGlobalKey()

Parameters

string	$class	Key class
	string	...$components Key components (starting with a key collection name)

Returns: string Colon-delimited list of $keyspace followed by escaped components

Since: 1.27

Implements IStoreKeyEncoder.

Definition at line 1916 of file WANObjectCache.php.

Referenced by getWithSetCallback().

◆ makeKey()

WANObjectCache::makeKey	(	$class,
		$components
	)

See also: BagOStuff::makeKey()

Parameters

string	$class	Key class
	string	...$components Key components (starting with a key collection name)

Returns: string Colon-delimited list of $keyspace followed by escaped components

Since: 1.27

Implements IStoreKeyEncoder.

Definition at line 1905 of file WANObjectCache.php.

◆ makeMultiKeys()

WANObjectCache::makeMultiKeys	(	array	$ids,
			$keyCallback
	)

final

Get an iterator of (cache key => entity ID) for a list of entity IDs.

The callback takes an ID string and returns a key via makeKey()/makeGlobalKey(). There should be no network nor filesystem I/O used in the callback. The entity ID/key mapping must be 1:1 or an exception will be thrown. If hashing is needed, then use the hash256() method.

Example usage for the default keyspace:

$keyedIds = $cache->makeMultiKeys(
    $modules,
    function ( $module ) use ( $cache ) {
        return $cache->makeKey( 'module-info', $module );
    }
);

Example usage for mixed default and global keyspace:

$keyedIds = $cache->makeMultiKeys(
    $filters,
    function ( $filter ) use ( $cache ) {
        return ( strpos( $filter, 'central:' ) === 0 )
            ? $cache->makeGlobalKey( 'regex-filter', $filter )
            : $cache->makeKey( 'regex-filter', $filter )
    }
);

Example usage with hashing:

$keyedIds = $cache->makeMultiKeys(
    $urls,
    function ( $url ) use ( $cache ) {
        return $cache->makeKey( 'url-info', $cache->hash256( $url ) );
    }
);

See also: WANObjectCache::makeKey(); WANObjectCache::makeGlobalKey(); WANObjectCache::hash256()

Parameters

string[] \| int[]	$ids	List of entity IDs
callable	$keyCallback	Function returning makeKey()/makeGlobalKey() on the input ID

Returns: ArrayIterator Iterator of (cache key => ID); order of $ids is preserved

Exceptions

UnexpectedValueException

Since: 1.28

Definition at line 1981 of file WANObjectCache.php.

◆ makePurgeValue()

WANObjectCache::makePurgeValue	(	$timestamp,
		$holdoff
	)

private

Parameters

float	$timestamp
int	$holdoff	In seconds

Returns: string Wrapped purge value

Definition at line 2536 of file WANObjectCache.php.

Referenced by getMultiCheckKeyTime(), processCheckKeys(), and relayPurge().

◆ multiRemap()

WANObjectCache::multiRemap	(	array	$ids,
		array	$res
	)

final

Get an (ID => value) map from (i) a non-unique list of entity IDs, and (ii) the list of corresponding entity values by first appearance of each ID in the entity ID list.

For use with getMultiWithSetCallback() and getMultiWithUnionSetCallback().

Only use this method if the entity ID/key mapping is trivially 1:1 without exception. Key generation method must utitilize the full entity ID in the key (not a hash of it).

Example usage:

$poems = $cache->getMultiWithSetCallback(
    $cache->makeMultiKeys(
        $uuids,
        function ( $uuid ) use ( $cache ) {
            return $cache->makeKey( 'poem', $uuid );
        }
    ),
    $cache::TTL_DAY,
    function ( $uuid ) use ( $url ) {
        return $this->http->run( [ 'method' => 'GET', 'url' => "$url/$uuid" ] );
    }
);
$poemsByUUID = $cache->multiRemap( $uuids, $poems );

See also: WANObjectCache::makeMultiKeys(); WANObjectCache::getMultiWithSetCallback(); WANObjectCache::getMultiWithUnionSetCallback()

Parameters

string[] \| int[]	$ids	Entity ID list makeMultiKeys()
mixed[]	$res	Result of getMultiWithSetCallback()/getMultiWithUnionSetCallback()

Returns: mixed[] Map of (ID => value); order of $ids is preserved

Since: 1.34

Definition at line 2037 of file WANObjectCache.php.

References $res.

◆ newEmpty()

static WANObjectCache::newEmpty ( )

static

Get an instance that wraps EmptyBagOStuff.

Returns: WANObjectCache

Definition at line 300 of file WANObjectCache.php.

Referenced by JobQueue\__construct(), FileBackendStore\__construct(), Wikimedia\Rdbms\LBFactory\__construct(), FileRepo\__construct(), and Wikimedia\Rdbms\LoadBalancer\__construct().

◆ parsePurgeValue()

WANObjectCache::parsePurgeValue ( $value )

private

Parameters

string | array | bool $value Possible string of the form "PURGED:<timestamp>:<holdoff>"

Returns: array|bool Array containing a UNIX timestamp (float) and holdoff period (integer), or false if value isn't a valid purge value

Definition at line 2501 of file WANObjectCache.php.

Referenced by getMultiCheckKeyTime(), processCheckKeys(), reapCheckKey(), and unwrap().

◆ prefixCacheKeys()

static WANObjectCache::prefixCacheKeys	(	array	$keys,
			$prefix
	)

staticprotected

Parameters

string[]	$keys
string	$prefix

Returns: string[] Prefix keys; the order of $keys is preserved

Definition at line 2476 of file WANObjectCache.php.

References $keys, and $res.

Referenced by getMulti().

◆ processCheckKeys()

WANObjectCache::processCheckKeys	(	array	$timeKeys,
		array	$wrappedValues,
			$now
	)

private

Since: 1.27

Parameters

string[]	$timeKeys	List of prefixed time check keys
mixed[]	$wrappedValues
float	$now

Returns: array[] List of purge value arrays

Definition at line 495 of file WANObjectCache.php.

References makePurgeValue(), and parsePurgeValue().

Referenced by getMulti().

◆ reap()

WANObjectCache::reap	(		$key,
			$purgeTimestamp,
		&	$isStale = `false`
	)

final

Set a key to soon expire in the local cluster if it pre-dates $purgeTimestamp.

This sets stale keys' time-to-live at HOLDOFF_TTL seconds, which both avoids broadcasting in mcrouter setups and also avoids races with new tombstones.

Parameters

string	$key	Cache key
int	$purgeTimestamp	UNIX timestamp of purge
bool	&$isStale	Whether the key is stale

Returns: bool Success

Since: 1.28

Definition at line 1851 of file WANObjectCache.php.

◆ reapCheckKey()

WANObjectCache::reapCheckKey	(		$key,
			$purgeTimestamp,
		&	$isStale = `false`
	)

final

Set a "check" key to soon expire in the local cluster if it pre-dates $purgeTimestamp.

Parameters

string	$key	Cache key
int	$purgeTimestamp	UNIX timestamp of purge
bool	&$isStale	Whether the key is stale

Returns: bool Success

Since: 1.28

Definition at line 1880 of file WANObjectCache.php.

References parsePurgeValue().

◆ relayDelete()

WANObjectCache::relayDelete ( $key )

protected

Do the actual async bus delete of a key.

Parameters

string $key Cache key

Returns: bool Success

Definition at line 2239 of file WANObjectCache.php.

Referenced by delete(), and resetCheckKey().

◆ relayPurge()

WANObjectCache::relayPurge	(	$key,
		$ttl,
		$holdoff
	)

protected

Do the actual async bus purge of a key.

This must set the key to "PURGED:<UNIX timestamp>:<holdoff>"

Parameters

string	$key	Cache key
int	$ttl	Seconds to keep the tombstone around
int	$holdoff	HOLDOFF_* constant controlling how long to ignore sets for this key

Returns: bool Success

Definition at line 2212 of file WANObjectCache.php.

References getCurrentTime(), and makePurgeValue().

Referenced by delete(), and touchCheckKey().

◆ resetCheckKey()

WANObjectCache::resetCheckKey ( $key )

final

Delete a "check" key from all datacenters, invalidating keys that use it.

This is similar to touchCheckKey() in that keys using it via get(), getMulti(), or getWithSetCallback() will be invalidated. The differences are:

a) The "check" key will be deleted from all caches and lazily re-initialized when accessed (rather than set everywhere)
b) Thus, dependent keys will be known to be stale, but not for how long (they are treated as "just" purged), which effects any lockTSE logic in getWithSetCallback()
c) Since "check" keys are initialized only on the server the key hashes to, any temporary ejection of that server will cause the value to be seen as purged as a new server will initialize the "check" key.

The advantage here is that the "check" keys, which have high TTLs, will only be created when a get*() method actually uses that key. This is better when a large number of "check" keys are invalided in a short period of time.

Note that "check" keys won't collide with other regular keys.

See also: WANObjectCache::get(); WANObjectCache::getWithSetCallback(); WANObjectCache::touchCheckKey()

Parameters

string $key Cache key

Returns: bool True if the item was purged or not found, false on failure

Definition at line 946 of file WANObjectCache.php.

References determineKeyClassForStats(), and relayDelete().

◆ resolveBusyValue()

WANObjectCache::resolveBusyValue ( $busyValue )

private

Parameters

mixed $busyValue

Returns: mixed

Definition at line 1616 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ resolveCTL()

WANObjectCache::resolveCTL	(	$value,
		$curTTL,
		$curInfo,
		$touchedCallback
	)

private

Parameters

mixed	$value
float \| null	$curTTL
array	$curInfo
callable \| null	$touchedCallback

Returns: array (current time left or null, UNIX timestamp of last purge or null)

Note: Callable type hints are not used to avoid class-autoloading

Definition at line 1545 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ resolveTouched()

WANObjectCache::resolveTouched	(	$value,
		$lastPurge,
		$touchedCallback
	)

private

Parameters

mixed	$value
float \| null	$lastPurge
callable \| null	$touchedCallback

Returns: float|null UNIX timestamp of last purge or null

Note: Callable type hints are not used to avoid class-autoloading

Definition at line 1565 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ scheduleAsyncRefresh()

WANObjectCache::scheduleAsyncRefresh	(	$key,
		$ttl,
		$callback,
		$opts
	)

private

Parameters

string	$key
int	$ttl	Seconds to live
callable	$callback
array	$opts

Returns: bool Success

Note: Callable type hints are not used to avoid class-autoloading

Definition at line 2260 of file WANObjectCache.php.

References $asyncHandler, and fetchOrRegenerate().

Referenced by fetchOrRegenerate().

◆ set()

WANObjectCache::set	(		$key,
			$value,
			$ttl = `self::TTL_INDEFINITE`,
		array	$opts = `[]`
	)

final

Set the value of a key in cache.

Simply calling this method when source data changes is not valid because the changes do not replicate to the other WAN sites. In that case, delete() should be used instead. This method is intended for use on cache misses.

If the data was read from a snapshot-isolated transactions (e.g. the default REPEATABLE-READ in innoDB), use 'since' to avoid the following race condition:

a) T1 starts
b) T2 updates a row, calls delete(), and commits
c) The HOLDOFF_TTL passes, expiring the delete() tombstone
d) T1 reads the row and calls set() due to a cache miss
e) Stale value is stuck in cache

Setting 'lag' and 'since' help avoids keys getting stuck in stale states.

Be aware that this does not update the process cache for getWithSetCallback() callers. Keys accessed via that method are not generally meant to also be set using this primitive method.

Do not use this method on versioned keys accessed via getWithSetCallback().

Example usage:

$dbr = wfGetDB( DB_REPLICA );
$setOpts = Database::getCacheSetOptions( $dbr );
// Fetch the row from the DB
$row = $dbr->selectRow( ... );
$key = $cache->makeKey( 'building', $buildingId );
$cache->set( $key, $row, $cache::TTL_DAY, $setOpts );

Parameters

string	$key	Cache key
mixed	$value
int	$ttl	Seconds to live. Special values are: WANObjectCache::TTL_INDEFINITE: Cache forever (default) WANObjectCache::TTL_UNCACHEABLE: Do not cache (if the key exists, it is not deleted)
array	$opts	Options map: lag: Seconds of replica DB lag. Typically, this is either the replica DB lag before the data was read or, if applicable, the replica DB lag before the snapshot-isolated transaction the data was read from started. Use false to indicate that replication is not running. Default: 0 seconds since: UNIX timestamp of the data in $value. Typically, this is either the current time the data was read or (if applicable) the time when the snapshot-isolated transaction the data was read from started. Default: 0 seconds pending: Whether this data is possibly from an uncommitted write transaction. Generally, other threads should not see values from the future and they certainly should not see ones that ended up getting rolled back. Default: false lockTSE: If excessive replication/snapshot lag is detected, then store the value with this TTL and flag it as stale. This is only useful if the reads for this key use getWithSetCallback() with "lockTSE" set. Note that if "staleTTL" is set then it will still add on to this TTL in the excessive lag scenario. Default: WANObjectCache::TSE_NONE staleTTL: Seconds to keep the key around if it is stale. The get()/getMulti() methods return such stale values with a $curTTL of 0, and getWithSetCallback() will call the regeneration callback in such cases, passing in the old value and its as-of time to the callback. This is useful if adaptiveTTL() is used on the old value's as-of time when it is verified as still being correct. Default: WANObjectCache::STALE_TTL_NONE creating: Optimize for the case where the key does not already exist. Default: false version: Integer version number signifiying the format of the value. Default: null walltime: How long the value took to generate in seconds. Default: 0.0

-param array{lag?:int,since?:int,pending?:bool,lockTSE?:int,staleTTL?:int,creating?:bool,version?:?string,walltime?:int|float} $opts

Note: Options added in 1.28: staleTTL; Options added in 1.33: creating; Options added in 1.34: version, walltime

Returns: bool Success

Definition at line 589 of file WANObjectCache.php.

References $cache, getCurrentTime(), IExpiringStore\TTL_SECOND, and wrap().

◆ setInterimValue()

WANObjectCache::setInterimValue	(	$key,
		$value,
		$ttl,
		$version,
		$walltime
	)

private

Parameters

string	$key
mixed	$value
int	$ttl
int \| null	$version	Value version number
float	$walltime	How long it took to generate the value in seconds

Definition at line 1598 of file WANObjectCache.php.

References getCurrentTime(), and wrap().

Referenced by fetchOrRegenerate().

◆ setLogger()

WANObjectCache::setLogger ( LoggerInterface $logger )

Parameters

LoggerInterface $logger

Definition at line 291 of file WANObjectCache.php.

References $logger.

Referenced by __construct().

◆ setMockTime()

WANObjectCache::setMockTime ( & $time )

Parameters

float | null &$time Mock UNIX timestamp for testing

Definition at line 2644 of file WANObjectCache.php.

◆ touchCheckKey()

WANObjectCache::touchCheckKey	(	$key,
		$holdoff = `self::HOLDOFF_TTL`
	)

final

Purge a "check" key from all datacenters, invalidating keys that use it.

This should only be called when the underlying data (being cached) changes in a significant way, and it is impractical to call delete() on all keys that should be changed. When get() is called on those keys, the relevant "check" keys must be supplied for this to work.

The "check" key essentially represents a last-modified time of an entity. When the key is touched, the timestamp will be updated to the current time. Keys using the "check" key via get(), getMulti(), or getWithSetCallback() will be invalidated. This approach is useful if many keys depend on a single entity.

The timestamp of the "check" key is treated as being HOLDOFF_TTL seconds in the future by get*() methods in order to avoid race conditions where keys are updated with stale values (e.g. from a lagged replica DB). A high TTL is set on the "check" key, making it possible to know the timestamp of the last change to the corresponding entities in most cases. This might use more cache space than resetCheckKey().

When a few important keys get a large number of hits, a high cache time is usually desired as well as "lockTSE" logic. The resetCheckKey() method is less appropriate in such cases since the "time since expiry" cannot be inferred, causing any get() after the reset to treat the key as being "hot", resulting in more stale value usage.

Note that "check" keys won't collide with other regular keys.

See also: WANObjectCache::get(); WANObjectCache::getWithSetCallback(); WANObjectCache::resetCheckKey()

Parameters

string	$key	Cache key
int	$holdoff	HOLDOFF_TTL or HOLDOFF_TTL_NONE constant

Returns: bool True if the item was purged or not found, false on failure

Definition at line 909 of file WANObjectCache.php.

References determineKeyClassForStats(), and relayPurge().

◆ unwrap()

WANObjectCache::unwrap	(	$wrapped,
		$now
	)

private

Parameters

array \| string \| bool	$wrapped	The entry at a cache key
float	$now	Unix Current timestamp (preferrably pre-query)

Returns

array (value or false if absent/tombstoned/malformed, value metadata map). The cache key metadata includes the following metadata:

asOf: UNIX timestamp of the value or null if there is no value
curTTL: remaining time-to-live (negative if tombstoned) or null if there is no value
version: value version number or null if the if there is no value
tombAsOf: UNIX timestamp of the tombstone or null if there is no tombstone -return array{0:mixed,1:array{asOf:?mixed,curTTL:?int|float,version:?mixed,tombAsOf:?mixed}}

Definition at line 2435 of file WANObjectCache.php.

References $FLD_TIME, $FLD_VALUE, $FLD_VALUE_VERSION, $PURGE_TIME, and parsePurgeValue().

Referenced by getInterimValue(), and getMulti().

◆ useInterimHoldOffCaching()

WANObjectCache::useInterimHoldOffCaching ( $enabled )

final

Enable or disable the use of brief caching for tombstoned keys.

When a key is purged via delete(), there normally is a period where caching is hold-off limited to an extremely short time. This method will disable that caching, forcing the callback to run for any of:

This is useful when both:

a) the database used by the callback is known to be up-to-date enough for some particular purpose (e.g. replica DB has applied transaction X)
b) the caller needs to exploit that fact, and therefore needs to avoid the use of inherently volatile and possibly stale interim keys

See also: WANObjectCache::delete()

Parameters

bool $enabled Whether to enable interim caching

Since: 1.31

Definition at line 2104 of file WANObjectCache.php.

Referenced by getInterimValue().

◆ worthRefreshExpiring()

WANObjectCache::worthRefreshExpiring	(	$curTTL,
		$lowTTL
	)

protected

Check if a key is nearing expiration and thus due for randomized regeneration.

This returns false if $curTTL >= $lowTTL. Otherwise, the chance of returning true increases steadily from 0% to 100% as the $curTTL moves from $lowTTL to 0 seconds. This handles widely varying levels of cache access traffic.

If $curTTL <= 0 (e.g. already expired), then this returns false.

Parameters

float	$curTTL	Approximate TTL left on the key if present
float	$lowTTL	Consider a refresh when $curTTL is less than this

Returns: bool

Definition at line 2317 of file WANObjectCache.php.

Referenced by fetchOrRegenerate(), and isAliveOrInGracePeriod().

◆ worthRefreshPopular()

WANObjectCache::worthRefreshPopular	(	$asOf,
		$ageNew,
		$timeTillRefresh,
		$now
	)

protected

Check if a key is due for randomized regeneration due to its popularity.

This is used so that popular keys can preemptively refresh themselves for higher consistency (especially in the case of purge loss/delay). Unpopular keys can remain in cache with their high nominal TTL. This means popular keys keep good consistency, whether the data changes frequently or not, and long-tail keys get to stay in cache and get hits too. Similar to worthRefreshExpiring(), randomization is used.

Parameters

float	$asOf	UNIX timestamp of the value
int	$ageNew	Age of key when this might recommend refreshing (seconds)
int	$timeTillRefresh	Age of key when it should be refreshed if popular (seconds)
float	$now	The current UNIX timestamp

Returns: bool

Definition at line 2347 of file WANObjectCache.php.

References $RAMPUP_TTL.

Referenced by fetchOrRegenerate().

◆ wrap()

WANObjectCache::wrap	(	$value,
		$ttl,
		$version,
		$now,
		$walltime
	)

private

Parameters

mixed	$value
int	$ttl	Seconds to live or zero for "indefinite"
int \| null	$version	Value version number or null if not versioned
float	$now	Unix Current timestamp just before calling set()
float	$walltime	How long it took to generate the value in seconds

Returns: array

Definition at line 2405 of file WANObjectCache.php.

References $FLD_GENERATION_TIME, $FLD_VALUE_VERSION, and $VERSION.

Referenced by set(), and setInterimValue().

◆ yieldStampedeLock()

WANObjectCache::yieldStampedeLock	(	$key,
		$hasLock
	)

private

Parameters

string	$key
bool	$hasLock

Definition at line 1483 of file WANObjectCache.php.

References getCurrentTime().

Referenced by fetchOrRegenerate().

Member Data Documentation

◆ $asyncHandler

callable null WANObjectCache::$asyncHandler

protected

Function that takes a WAN cache callback and runs it later.

Definition at line 126 of file WANObjectCache.php.

Referenced by scheduleAsyncRefresh().

◆ $cache

BagOStuff WANObjectCache::$cache

protected

The local datacenter cache.

Definition at line 118 of file WANObjectCache.php.

Referenced by set().

◆ $callbackDepth

int WANObjectCache::$callbackDepth = 0

private

Callback stack depth for getWithSetCallback()

Definition at line 142 of file WANObjectCache.php.

Referenced by fetchOrRegenerate().

◆ $CHECK_KEY_TTL

int WANObjectCache::$CHECK_KEY_TTL = self::TTL_YEAR

staticprivate

Seconds to keep dependency purge keys around.

Definition at line 193 of file WANObjectCache.php.

◆ $cluster

string WANObjectCache::$cluster

protected

Cache cluster name for mcrouter use.

Definition at line 133 of file WANObjectCache.php.

◆ $COOLOFF_KEY_PREFIX

WANObjectCache::$COOLOFF_KEY_PREFIX = 'WANCache:c:'

staticprivate

Definition at line 246 of file WANObjectCache.php.

◆ $COOLOFF_TTL

int WANObjectCache::$COOLOFF_TTL = 1

staticprivate

Seconds to no-op key set() calls to avoid large blob I/O stampedes.

Definition at line 200 of file WANObjectCache.php.

◆ $epoch

float WANObjectCache::$epoch

protected

Unix timestamp of the oldest possible valid values.

Definition at line 137 of file WANObjectCache.php.

◆ $FLD_FLAGS

int WANObjectCache::$FLD_FLAGS = 4

staticprivate

@noinspection PhpUnusedPrivateFieldInspection

Key to the flags bit field (reserved number)

Definition at line 236 of file WANObjectCache.php.

◆ $FLD_FORMAT_VERSION

int WANObjectCache::$FLD_FORMAT_VERSION = 0

staticprivate

Key to WAN cache version number.

Definition at line 228 of file WANObjectCache.php.

◆ $FLD_GENERATION_TIME

int WANObjectCache::$FLD_GENERATION_TIME = 6

staticprivate

Key to how long it took to generate the value.

Definition at line 240 of file WANObjectCache.php.

Referenced by wrap().

◆ $FLD_TIME

int WANObjectCache::$FLD_TIME = 3

staticprivate

Key to the cache timestamp.

Definition at line 234 of file WANObjectCache.php.

Referenced by unwrap().

◆ $FLD_TTL

int WANObjectCache::$FLD_TTL = 2

staticprivate

Key to the original TTL.

Definition at line 232 of file WANObjectCache.php.

◆ $FLD_VALUE

int WANObjectCache::$FLD_VALUE = 1

staticprivate

Key to the cached value.

Definition at line 230 of file WANObjectCache.php.

Referenced by unwrap().

◆ $FLD_VALUE_VERSION

int WANObjectCache::$FLD_VALUE_VERSION = 5

staticprivate

Key to collection cache version number.

Definition at line 238 of file WANObjectCache.php.

Referenced by unwrap(), and wrap().

◆ $GENERATION_SLOW_SEC

int WANObjectCache::$GENERATION_SLOW_SEC = 3

staticprivate

Consider value generation slow if it takes more than this many seconds.

Definition at line 217 of file WANObjectCache.php.

◆ $INTERIM_KEY_PREFIX

WANObjectCache::$INTERIM_KEY_PREFIX = 'WANCache:i:'

staticprivate

Definition at line 243 of file WANObjectCache.php.

◆ $INTERIM_KEY_TTL

int WANObjectCache::$INTERIM_KEY_TTL = 1

staticprivate

Seconds to keep interim value keys for tombstoned keys around.

Definition at line 195 of file WANObjectCache.php.

◆ $LOCK_TTL

int WANObjectCache::$LOCK_TTL = 10

staticprivate

Seconds to keep lock keys around.

Definition at line 198 of file WANObjectCache.php.

◆ $logger

LoggerInterface WANObjectCache::$logger

protected

Definition at line 122 of file WANObjectCache.php.

Referenced by setLogger().

◆ $mcrouterAware

bool WANObjectCache::$mcrouterAware

protected

Whether to use mcrouter key prefixing for routing.

Definition at line 129 of file WANObjectCache.php.

◆ $MUTEX_KEY_PREFIX

WANObjectCache::$MUTEX_KEY_PREFIX = 'WANCache:m:'

staticprivate

Definition at line 245 of file WANObjectCache.php.

◆ $processCaches

MapCacheLRU [] WANObjectCache::$processCaches = []

protected

Map of group PHP instance caches.

Definition at line 120 of file WANObjectCache.php.

◆ $PURGE_HOLDOFF

int WANObjectCache::$PURGE_HOLDOFF = 1

staticprivate

Key to the tombstone entry hold-off TTL.

Definition at line 222 of file WANObjectCache.php.

Referenced by getMulti().

◆ $PURGE_TIME

int WANObjectCache::$PURGE_TIME = 0

staticprivate

Key to the tombstone entry timestamp.

Definition at line 220 of file WANObjectCache.php.

Referenced by getMulti(), getMultiCheckKeyTime(), and unwrap().

◆ $PURGE_VAL_PREFIX

WANObjectCache::$PURGE_VAL_PREFIX = 'PURGED:'

staticprivate

Definition at line 248 of file WANObjectCache.php.

◆ $RAMPUP_TTL

int WANObjectCache::$RAMPUP_TTL = 30

staticprivate

Seconds to ramp up the chance of regeneration due to expected time-till-refresh.

Definition at line 202 of file WANObjectCache.php.

Referenced by worthRefreshPopular().

◆ $RECENT_SET_HIGH_MS

int WANObjectCache::$RECENT_SET_HIGH_MS = 100

staticprivate

Max millisecond set() backoff during hold-off (far less than INTERIM_KEY_TTL)

Definition at line 214 of file WANObjectCache.php.

◆ $RECENT_SET_LOW_MS

int WANObjectCache::$RECENT_SET_LOW_MS = 50

staticprivate

Min millisecond set() backoff during hold-off (far less than INTERIM_KEY_TTL)

Definition at line 212 of file WANObjectCache.php.

◆ $region

string WANObjectCache::$region

protected

Physical region for mcrouter use.

Definition at line 131 of file WANObjectCache.php.

◆ $secret

string WANObjectCache::$secret

protected

Stable secret used for hasing long strings into key components.

Definition at line 139 of file WANObjectCache.php.

◆ $SET_DELAY_HIGH_MS

int WANObjectCache::$SET_DELAY_HIGH_MS = 50

staticprivate

Milliseconds of key fetch/validate/regenerate delay prone to set() stampedes.

Definition at line 210 of file WANObjectCache.php.

◆ $stats

StatsdDataFactoryInterface WANObjectCache::$stats

protected

Definition at line 124 of file WANObjectCache.php.

◆ $TIME_KEY_PREFIX

WANObjectCache::$TIME_KEY_PREFIX = 'WANCache:t:'

staticprivate

Definition at line 244 of file WANObjectCache.php.

◆ $TINY_NEGATIVE

float WANObjectCache::$TINY_NEGATIVE = -0.000001

staticprivate

Tiny negative float to use when CTL comes up >= 0 due to clock skew.

Definition at line 205 of file WANObjectCache.php.

◆ $TINY_POSTIVE

float WANObjectCache::$TINY_POSTIVE = 0.000001

staticprivate

Tiny positive float to use when using "minTime" to assert an inequality.

Definition at line 207 of file WANObjectCache.php.

◆ $useInterimHoldOffCaching

bool WANObjectCache::$useInterimHoldOffCaching = true

protected

Whether to use "interim" caching while keys are tombstoned.

Definition at line 135 of file WANObjectCache.php.

◆ $VALUE_KEY_PREFIX

WANObjectCache::$VALUE_KEY_PREFIX = 'WANCache:v:'

staticprivate

Definition at line 242 of file WANObjectCache.php.

◆ $VERSION

int WANObjectCache::$VERSION = 1

staticprivate

Cache format version number.

Definition at line 225 of file WANObjectCache.php.

Referenced by wrap().

◆ $wallClockOverride

float null WANObjectCache::$wallClockOverride

private

Definition at line 149 of file WANObjectCache.php.

Referenced by getCurrentTime().

◆ $warmupCache

mixed [] WANObjectCache::$warmupCache = []

private

Temporary warm-up cache.

Definition at line 144 of file WANObjectCache.php.

Referenced by getRawKeysForWarmup().

◆ $warmupKeyMisses

int WANObjectCache::$warmupKeyMisses = 0

private

Key fetched.

Definition at line 146 of file WANObjectCache.php.

Referenced by getWarmupKeyMisses().

The documentation for this class was generated from the following file:

includes/libs/objectcache/wancache/WANObjectCache.php

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Detailed Description

Using WANObjectCache

Deploying WANObjectCache

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ adaptiveTTL()

◆ checkAndSetCooloff()

◆ claimStampedeLock()

◆ clearLastError()

◆ clearProcessCache()

◆ delete()

◆ determineKeyClassForStats()

◆ fetchOrRegenerate()

◆ get()

◆ getCheckKeyTime()

◆ getCurrentTime()

◆ getInterimValue()

◆ getLastError()

◆ getMulti()

◆ getMultiCheckKeyTime()

◆ getMultiWithSetCallback()

◆ getMultiWithUnionSetCallback()

◆ getNonProcessCachedMultiKeys()

◆ getProcessCache()

◆ getProcessCacheKey()

◆ getQoS()

◆ getRawKeysForWarmup()

◆ getWarmupKeyMisses()

◆ getWithSetCallback()

◆ hash256()

◆ isAliveOrInGracePeriod()

◆ isValid()

◆ isVolatileValueAgeNegligible()

◆ makeGlobalKey()

◆ makeKey()

◆ makeMultiKeys()

◆ makePurgeValue()

◆ multiRemap()

◆ newEmpty()

◆ parsePurgeValue()

◆ prefixCacheKeys()

◆ processCheckKeys()

◆ reap()

◆ reapCheckKey()

◆ relayDelete()

◆ relayPurge()

◆ resetCheckKey()

◆ resolveBusyValue()

◆ resolveCTL()

◆ resolveTouched()

◆ scheduleAsyncRefresh()

◆ set()

◆ setInterimValue()

◆ setLogger()

◆ setMockTime()

◆ touchCheckKey()

◆ unwrap()

◆ useInterimHoldOffCaching()

◆ worthRefreshExpiring()

◆ worthRefreshPopular()

◆ wrap()

◆ yieldStampedeLock()

Member Data Documentation

◆ $asyncHandler

◆ $cache

◆ $callbackDepth

◆ $CHECK_KEY_TTL

◆ $cluster

◆ $COOLOFF_KEY_PREFIX

◆ $COOLOFF_TTL