Semaphore semantics to restrict how many workers may concurrently perform a task. More...

Inherits LoggerAwareInterface.

Inherited by MediaWiki\PoolCounter\PoolCounterClient, MediaWiki\PoolCounter\PoolCounterNull, and MediaWiki\PoolCounter\PoolCounterRedis.

Collaboration diagram for MediaWiki\PoolCounter\PoolCounter:

[legend]

Public Member Functions
	__construct (array $conf, string $type, string $key)

	acquireForAnyone ( $timeout=null)
	I want to do this task, but if anyone else does it instead, it's also fine for me.

	acquireForMe ( $timeout=null)
	I want to do this task and I need to do it myself.

	getKey ()

	getLogger ()

	isFastStaleEnabled ()
	Is fast stale mode (T250248) enabled? This may be overridden by the PoolCounterWork subclass.

	release ()
	I have successfully finished my task.

	setLogger (LoggerInterface $logger)

Public Attributes
const	DONE = 3

const	ERROR = -1

const	LOCK_HELD = -5

const	LOCKED = 1

const	NOT_LOCKED = -2

const	QUEUE_FULL = -3

const	RELEASED = 2

const	TIMEOUT = -4

Protected Member Functions
	hashKeyIntoSlots ( $type, $key, $slots)
	Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range).

	onAcquire ()
	Update any lock tracking information when the lock is acquired.

	onRelease ()
	Update any lock tracking information when the lock is released.

	precheckAcquire ()
	Checks that the lock request is sensible.

Protected Attributes
string	$key
	All workers with the same key share the lock.

LoggerInterface	$logger

int	$maxqueue
	If this number of workers are already working/waiting, fail instead of wait.

int	$slots = 0
	Maximum number of workers working on this task type, regardless of key.

int	$timeout
	Maximum time in seconds to wait for the lock.

int	$workers
	Maximum number of workers working on tasks with the same key simultaneously.

Detailed Description

Semaphore semantics to restrict how many workers may concurrently perform a task.

When you have many workers (threads/servers) in service, and a cached item expensive to produce expires, you may get several workers computing the same expensive item at the same time.

Given enough incoming requests and the item expiring quickly (non-cacheable, or lots of edits or other invalidation events) that single task can end up unfairly using most (or all) of the CPUs of the server cluster. This is also known as "Michael Jackson effect", as this scenario happened on the English Wikipedia in 2009 on the day Michael Jackson died. See also https://wikitech.wikimedia.org/wiki/Michael_Jackson_effect.

PoolCounter was created to provide semaphore semantics to restrict the number of workers that may be concurrently performing a given task. Only one key can be locked by any PoolCounter instance of a process, except for keys that start with "nowait:". However, only non-blocking requests (timeout=0) may be used with a "nowait:" key.

By default PoolCounterNull is used, which provides no locking. Install the poolcounterd service from https://gerrit.wikimedia.org/g/mediawiki/services/poolcounter to enable this feature.

Since: 1.16

Stability: stable: to extend

Definition at line 56 of file PoolCounter.php.

Constructor & Destructor Documentation

◆ __construct()

MediaWiki\PoolCounter\PoolCounter::__construct	(	array	$conf,
		string	$type,
		string	$key )

Parameters

array	$conf
string	$type	The class of actions to limit concurrency for (task type)
string	$key

Definition at line 104 of file PoolCounter.php.

References MediaWiki\PoolCounter\PoolCounter\$key, and MediaWiki\PoolCounter\PoolCounter\hashKeyIntoSlots().

Member Function Documentation

◆ acquireForAnyone()

MediaWiki\PoolCounter\PoolCounter::acquireForAnyone ( $timeout = null )

abstract

I want to do this task, but if anyone else does it instead, it's also fine for me.

I will read its cached data.

Parameters

int | null $timeout Wait timeout, or null to use value passed to the constructor

Returns: Status Value is one of Locked/Done/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, MediaWiki\PoolCounter\PoolCounterNull, and MediaWiki\PoolCounter\PoolCounterRedis.

◆ acquireForMe()

MediaWiki\PoolCounter\PoolCounter::acquireForMe ( $timeout = null )

abstract

I want to do this task and I need to do it myself.

Parameters

int | null $timeout Wait timeout, or null to use value passed to the constructor

Returns: Status Value is one of Locked/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, MediaWiki\PoolCounter\PoolCounterNull, and MediaWiki\PoolCounter\PoolCounterRedis.

◆ getKey()

MediaWiki\PoolCounter\PoolCounter::getKey ( )

Returns: string

Definition at line 125 of file PoolCounter.php.

References MediaWiki\PoolCounter\PoolCounter\$key.

◆ getLogger()

MediaWiki\PoolCounter\PoolCounter::getLogger ( )

Access: internal: For use in PoolCounterWork only

Returns: LoggerInterface

Definition at line 242 of file PoolCounter.php.

◆ hashKeyIntoSlots()

MediaWiki\PoolCounter\PoolCounter::hashKeyIntoSlots	(	$type,
		$key,
		$slots )

protected

Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range).

This is used for a global limit on the number of instances of a given type that can acquire a lock. The hashing is deterministic so that PoolCounter::$workers is always an upper limit of how many instances with the same key can acquire a lock.

Parameters

string	$type	The class of actions to limit concurrency for (task type)
string	$key	PoolCounter instance key (any string)
int	$slots	The number of slots (max allowed value is 65536)

Returns: string Slot key with the type and slot number

Definition at line 215 of file PoolCounter.php.

References MediaWiki\PoolCounter\PoolCounter\$key, and MediaWiki\PoolCounter\PoolCounter\$slots.

Referenced by MediaWiki\PoolCounter\PoolCounter\__construct().

◆ isFastStaleEnabled()

MediaWiki\PoolCounter\PoolCounter::isFastStaleEnabled ( )

Is fast stale mode (T250248) enabled? This may be overridden by the PoolCounterWork subclass.

Returns: bool

Definition at line 225 of file PoolCounter.php.

◆ onAcquire()

MediaWiki\PoolCounter\PoolCounter::onAcquire ( )

finalprotected

Update any lock tracking information when the lock is acquired.

Since: 1.25

Definition at line 191 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ onRelease()

MediaWiki\PoolCounter\PoolCounter::onRelease ( )

finalprotected

Update any lock tracking information when the lock is released.

Since: 1.25

Definition at line 199 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ precheckAcquire()

MediaWiki\PoolCounter\PoolCounter::precheckAcquire ( )

finalprotected

Checks that the lock request is sensible.

Returns: Status good for sensible requests, fatal for the not so sensible

Since: 1.25

Definition at line 162 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\acquireForAnyone(), and MediaWiki\PoolCounter\PoolCounterRedis\acquireForMe().

◆ release()

MediaWiki\PoolCounter\PoolCounter::release ( )

abstract

I have successfully finished my task.

Lets another one grab the lock, and returns the workers waiting on acquireForAnyone()

Returns: Status Value is one of Released/NotLocked/Error

Reimplemented in MediaWiki\PoolCounter\PoolCounterClient, MediaWiki\PoolCounter\PoolCounterNull, and MediaWiki\PoolCounter\PoolCounterRedis.

◆ setLogger()

MediaWiki\PoolCounter\PoolCounter::setLogger ( LoggerInterface $logger )

Since: 1.42

Parameters

LoggerInterface $logger

Returns: void

Definition at line 234 of file PoolCounter.php.

References MediaWiki\PoolCounter\PoolCounter\$logger.

Member Data Documentation

◆ $key

string MediaWiki\PoolCounter\PoolCounter::$key

protected

All workers with the same key share the lock.

Definition at line 69 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\__construct(), MediaWiki\PoolCounter\PoolCounter\__construct(), MediaWiki\PoolCounter\PoolCounter\getKey(), and MediaWiki\PoolCounter\PoolCounter\hashKeyIntoSlots().

◆ $logger

LoggerInterface MediaWiki\PoolCounter\PoolCounter::$logger

protected

Definition at line 83 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounter\setLogger().

◆ $maxqueue

int MediaWiki\PoolCounter\PoolCounter::$maxqueue

protected

If this number of workers are already working/waiting, fail instead of wait.

Definition at line 80 of file PoolCounter.php.

◆ $slots

int MediaWiki\PoolCounter\PoolCounter::$slots = 0

protected

Maximum number of workers working on this task type, regardless of key.

0 means unlimited. Max allowed value is 65536. The way the slot limit is enforced is overzealous - this option should be used with caution.

Definition at line 78 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounter\hashKeyIntoSlots().

◆ $timeout

int MediaWiki\PoolCounter\PoolCounter::$timeout

protected

Maximum time in seconds to wait for the lock.

Definition at line 82 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\acquireForAnyone(), MediaWiki\PoolCounter\PoolCounterRedis\acquireForMe(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ $workers

int MediaWiki\PoolCounter\PoolCounter::$workers

protected

Maximum number of workers working on tasks with the same key simultaneously.

Definition at line 71 of file PoolCounter.php.

◆ DONE

const MediaWiki\PoolCounter\PoolCounter::DONE = 3

Definition at line 60 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterWork\execute(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ ERROR

const MediaWiki\PoolCounter\PoolCounter::ERROR = -1

Definition at line 62 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterWork\execute().

◆ LOCK_HELD

const MediaWiki\PoolCounter\PoolCounter::LOCK_HELD = -5

Definition at line 66 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterWork\execute(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ LOCKED

const MediaWiki\PoolCounter\PoolCounter::LOCKED = 1

Definition at line 58 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterNull\acquireForAnyone(), MediaWiki\PoolCounter\PoolCounterNull\acquireForMe(), MediaWiki\PoolCounter\PoolCounterWork\execute(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ NOT_LOCKED

const MediaWiki\PoolCounter\PoolCounter::NOT_LOCKED = -2

Definition at line 63 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\release().

◆ QUEUE_FULL

const MediaWiki\PoolCounter\PoolCounter::QUEUE_FULL = -3

Definition at line 64 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterWork\execute(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ RELEASED

const MediaWiki\PoolCounter\PoolCounter::RELEASED = 2

Definition at line 59 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterNull\release(), and MediaWiki\PoolCounter\PoolCounterRedis\release().

◆ TIMEOUT

const MediaWiki\PoolCounter\PoolCounter::TIMEOUT = -4

Definition at line 65 of file PoolCounter.php.

Referenced by MediaWiki\PoolCounter\PoolCounterWork\execute(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

The documentation for this class was generated from the following file:

includes/poolcounter/PoolCounter.php

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ acquireForAnyone()

◆ acquireForMe()

◆ getKey()

◆ getLogger()

◆ hashKeyIntoSlots()

◆ isFastStaleEnabled()

◆ onAcquire()

◆ onRelease()

◆ precheckAcquire()

◆ release()

◆ setLogger()

Member Data Documentation

◆ $key

◆ $logger

◆ $maxqueue

◆ $slots

◆ $timeout

◆ $workers

◆ DONE

◆ ERROR

◆ LOCK_HELD

◆ LOCKED

◆ NOT_LOCKED

◆ QUEUE_FULL

◆ RELEASED

◆ TIMEOUT