MediaWiki REL1_41
PoolCounter Class Reference

Semaphore semantics to restrict how many workers may concurrently perform a task. More...

Inherited by MediaWiki\PoolCounter\PoolCounterClient, PoolCounterNull, and PoolCounterRedis.

Public Member Functions

 __construct (array $conf, string $type, string $key)
 
 acquireForAnyone ( $timeout=null)
 I want to do this task, but if anyone else does it instead, it's also fine for me.
 
 acquireForMe ( $timeout=null)
 I want to do this task and I need to do it myself.
 
 getKey ()
 
 isFastStaleEnabled ()
 Is fast stale mode (T250248) enabled? This may be overridden by the PoolCounterWork subclass.
 
 release ()
 I have successfully finished my task.
 

Public Attributes

const DONE = 3
 
const ERROR = -1
 
const LOCK_HELD = -5
 
const LOCKED = 1
 
const NOT_LOCKED = -2
 
const QUEUE_FULL = -3
 
const RELEASED = 2
 
const TIMEOUT = -4
 

Protected Member Functions

 hashKeyIntoSlots ( $type, $key, $slots)
 Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range).
 
 onAcquire ()
 Update any lock tracking information when the lock is acquired.
 
 onRelease ()
 Update any lock tracking information when the lock is released.
 
 precheckAcquire ()
 Checks that the lock request is sensible.
 

Protected Attributes

string $key
 All workers with the same key share the lock.
 
int $maxqueue
 If this number of workers are already working/waiting, fail instead of wait.
 
int $slots = 0
 Maximum number of workers working on this task type, regardless of key.
 
int $timeout
 Maximum time in seconds to wait for the lock.
 
int $workers
 Maximum number of workers working on tasks with the same key simultaneously.
 

Detailed Description

Semaphore semantics to restrict how many workers may concurrently perform a task.

When you have many workers (threads/servers) in service, and a cached item expensive to produce expires, you may get several workers computing the same expensive item at the same time.

Given enough incoming requests and the item expiring quickly (non-cacheable, or lots of edits or other invalidation events) that single task can end up unfairly using most (or all) of the CPUs of the server cluster. This is also known as "Michael Jackson effect", as this scenario happened on the English Wikipedia in 2009 on the day Michael Jackson died. See also https://wikitech.wikimedia.org/wiki/Michael_Jackson_effect.

PoolCounter was created to provide semaphore semantics to restrict the number of workers that may be concurrently performing a given task. Only one key can be locked by any PoolCounter instance of a process, except for keys that start with "nowait:". However, only non-blocking requests (timeout=0) may be used with a "nowait:" key.

By default PoolCounterNull is used, which provides no locking. Install the poolcounterd service from https://gerrit.wikimedia.org/g/mediawiki/services/poolcounter to enable this feature.

Since
1.16
Stability: stable
to extend

Definition at line 51 of file PoolCounter.php.

Constructor & Destructor Documentation

◆ __construct()

PoolCounter::__construct ( array $conf,
string $type,
string $key )
Parameters
array$conf
string$typeThe class of actions to limit concurrency for (task type)
string$key

Definition at line 98 of file PoolCounter.php.

References $key, and hashKeyIntoSlots().

Member Function Documentation

◆ acquireForAnyone()

PoolCounter::acquireForAnyone ( $timeout = null)
abstract

I want to do this task, but if anyone else does it instead, it's also fine for me.

I will read its cached data.

Parameters
int | null$timeoutWait timeout, or null to use value passed to the constructor
Returns
Status Value is one of Locked/Done/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, PoolCounterNull, and PoolCounterRedis.

◆ acquireForMe()

PoolCounter::acquireForMe ( $timeout = null)
abstract

I want to do this task and I need to do it myself.

Parameters
int | null$timeoutWait timeout, or null to use value passed to the constructor
Returns
Status Value is one of Locked/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, PoolCounterNull, and PoolCounterRedis.

◆ getKey()

PoolCounter::getKey ( )
Returns
string

Definition at line 118 of file PoolCounter.php.

References $key.

◆ hashKeyIntoSlots()

PoolCounter::hashKeyIntoSlots ( $type,
$key,
$slots )
protected

Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range).

This is used for a global limit on the number of instances of a given type that can acquire a lock. The hashing is deterministic so that PoolCounter::$workers is always an upper limit of how many instances with the same key can acquire a lock.

Parameters
string$typeThe class of actions to limit concurrency for (task type)
string$keyPoolCounter instance key (any string)
int$slotsThe number of slots (max allowed value is 65536)
Returns
string Slot key with the type and slot number

Definition at line 204 of file PoolCounter.php.

References $key, and $slots.

Referenced by __construct().

◆ isFastStaleEnabled()

PoolCounter::isFastStaleEnabled ( )

Is fast stale mode (T250248) enabled? This may be overridden by the PoolCounterWork subclass.

Returns
bool

Definition at line 214 of file PoolCounter.php.

◆ onAcquire()

PoolCounter::onAcquire ( )
finalprotected

Update any lock tracking information when the lock is acquired.

Since
1.25

Definition at line 180 of file PoolCounter.php.

Referenced by PoolCounterRedis\waitForSlotOrNotif().

◆ onRelease()

PoolCounter::onRelease ( )
finalprotected

Update any lock tracking information when the lock is released.

Since
1.25

Definition at line 188 of file PoolCounter.php.

Referenced by PoolCounterRedis\release(), and PoolCounterRedis\waitForSlotOrNotif().

◆ precheckAcquire()

PoolCounter::precheckAcquire ( )
finalprotected

Checks that the lock request is sensible.

Returns
Status good for sensible requests, fatal for the not so sensible
Since
1.25

Definition at line 155 of file PoolCounter.php.

Referenced by PoolCounterRedis\acquireForAnyone(), and PoolCounterRedis\acquireForMe().

◆ release()

PoolCounter::release ( )
abstract

I have successfully finished my task.

Lets another one grab the lock, and returns the workers waiting on acquireForAnyone()

Returns
Status Value is one of Released/NotLocked/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, PoolCounterNull, and PoolCounterRedis.

Member Data Documentation

◆ $key

string PoolCounter::$key
protected

All workers with the same key share the lock.

Definition at line 64 of file PoolCounter.php.

Referenced by PoolCounterRedis\__construct(), __construct(), getKey(), and hashKeyIntoSlots().

◆ $maxqueue

int PoolCounter::$maxqueue
protected

If this number of workers are already working/waiting, fail instead of wait.

Definition at line 75 of file PoolCounter.php.

◆ $slots

int PoolCounter::$slots = 0
protected

Maximum number of workers working on this task type, regardless of key.

0 means unlimited. Max allowed value is 65536. The way the slot limit is enforced is overzealous - this option should be used with caution.

Definition at line 73 of file PoolCounter.php.

Referenced by hashKeyIntoSlots().

◆ $timeout

int PoolCounter::$timeout
protected

Maximum time in seconds to wait for the lock.

Definition at line 77 of file PoolCounter.php.

Referenced by PoolCounterRedis\acquireForAnyone(), PoolCounterRedis\acquireForMe(), and PoolCounterRedis\waitForSlotOrNotif().

◆ $workers

int PoolCounter::$workers
protected

Maximum number of workers working on tasks with the same key simultaneously.

Definition at line 66 of file PoolCounter.php.

◆ DONE

const PoolCounter::DONE = 3

Definition at line 55 of file PoolCounter.php.

Referenced by PoolCounterWork\execute(), and PoolCounterRedis\waitForSlotOrNotif().

◆ ERROR

const PoolCounter::ERROR = -1

Definition at line 57 of file PoolCounter.php.

Referenced by PoolCounterWork\execute().

◆ LOCK_HELD

const PoolCounter::LOCK_HELD = -5

Definition at line 61 of file PoolCounter.php.

Referenced by PoolCounterWork\execute(), and PoolCounterRedis\waitForSlotOrNotif().

◆ LOCKED

◆ NOT_LOCKED

const PoolCounter::NOT_LOCKED = -2

Definition at line 58 of file PoolCounter.php.

Referenced by PoolCounterRedis\release().

◆ QUEUE_FULL

const PoolCounter::QUEUE_FULL = -3

Definition at line 59 of file PoolCounter.php.

Referenced by PoolCounterWork\execute(), and PoolCounterRedis\waitForSlotOrNotif().

◆ RELEASED

const PoolCounter::RELEASED = 2

Definition at line 54 of file PoolCounter.php.

Referenced by PoolCounterNull\release(), and PoolCounterRedis\release().

◆ TIMEOUT

const PoolCounter::TIMEOUT = -4

Definition at line 60 of file PoolCounter.php.

Referenced by PoolCounterWork\execute(), and PoolCounterRedis\waitForSlotOrNotif().


The documentation for this class was generated from the following file: