MediaWiki master
Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Static Protected Attributes | List of all members
MediaWiki\PoolCounter\PoolCounterRedis Class Reference

Version of PoolCounter that uses Redis. More...

Inherits MediaWiki\PoolCounter\PoolCounter.

Collaboration diagram for MediaWiki\PoolCounter\PoolCounterRedis:
Collaboration graph
[legend]

Public Member Functions

 __construct ( $conf, $type, $key)
 
 acquireForAnyone ( $timeout=null)
 I want to do this task, but if anyone else does it instead, it's also fine for me.
 
 acquireForMe ( $timeout=null)
 I want to do this task and I need to do it myself.
 
 release ()
 I have successfully finished my task.
 
- Public Member Functions inherited from MediaWiki\PoolCounter\PoolCounter
 __construct (array $conf, string $type, string $key)
 
 getKey ()
 
 getLogger ()
 
 isFastStaleEnabled ()
 Is fast stale mode (T250248) enabled? This may be overridden by the PoolCounterWork subclass.
 
 setLogger (LoggerInterface $logger)
 

Static Public Member Functions

static releaseAll ()
 Try to make sure that locks get released (even with exceptions and fatals)
 

Protected Member Functions

 getConnection ()
 
 getSlotListKey ()
 
 getSlotRTimeSetKey ()
 
 getWaitSetKey ()
 
 getWakeupListKey ()
 
 initAndPopPoolSlotList (RedisConnRef $conn, $now)
 
 registerAcquisitionTime (RedisConnRef $conn, $slot, $now)
 
 waitForSlotOrNotif ( $doWakeup, $timeout=null)
 
- Protected Member Functions inherited from MediaWiki\PoolCounter\PoolCounter
 hashKeyIntoSlots ( $type, $key, $slots)
 Given a key (any string) and the number of lots, returns a slot key (a prefix with a suffix integer from the [0..($slots-1)] range).
 
 onAcquire ()
 Update any lock tracking information when the lock is acquired.
 
 onRelease ()
 Update any lock tracking information when the lock is released.
 
 precheckAcquire ()
 Checks that the lock request is sensible.
 

Protected Attributes

RedisConnRef $conn
 
string $keySha1
 SHA-1 of the key.
 
int $lockTTL
 TTL for locks to expire (work should finish in this time)
 
int null $onRelease
 AWAKE_* constant.
 
RedisConnectionPool $pool
 
HashRing $ring
 
array $serversByLabel
 (server label => host) map
 
string $session
 Unique string to identify this process.
 
string null $slot
 Pool slot value.
 
float null $slotTime
 UNIX timestamp.
 
- Protected Attributes inherited from MediaWiki\PoolCounter\PoolCounter
string $key
 All workers with the same key share the lock.
 
LoggerInterface $logger
 
int $maxqueue
 If this number of workers are already working/waiting, fail instead of wait.
 
int $slots = 0
 Maximum number of workers working on this task type, regardless of key.
 
int $timeout
 Maximum time in seconds to wait for the lock.
 
int $workers
 Maximum number of workers working on tasks with the same key simultaneously.
 

Static Protected Attributes

static PoolCounterRedis[] $active = null
 List of active PoolCounterRedis objects in this script.
 

Additional Inherited Members

- Public Attributes inherited from MediaWiki\PoolCounter\PoolCounter
const DONE = 3
 
const ERROR = -1
 
const LOCK_HELD = -5
 
const LOCKED = 1
 
const NOT_LOCKED = -2
 
const QUEUE_FULL = -3
 
const RELEASED = 2
 
const TIMEOUT = -4
 

Detailed Description

Version of PoolCounter that uses Redis.

There are four main redis keys used to track each pool counter key:

  • poolcounter:l-slots-* : A list of available slot IDs for a pool.
  • poolcounter:z-renewtime-* : A sorted set of (slot ID, UNIX timestamp as score) used for tracking the next time a slot should be released. This is -1 when a slot is created, and is set when released (expired), locked, and unlocked.
  • poolcounter:z-wait-* : A sorted set of (slot ID, UNIX timestamp as score) used for tracking waiting processes (and wait time).
  • poolcounter:l-wakeup-* : A list pushed to for the sake of waking up processes when a any process in the pool finishes (lasts for 1ms).

For a given pool key, all the redis keys start off non-existing and are deleted if not used for a while to prevent garbage from building up on the server. They are atomically re-initialized as needed. The "z-renewtime" key is used for detecting sessions which got slots but then disappeared. Stale entries from there have their timestamp updated and the corresponding slots freed up. The "z-wait" key is used for detecting processes registered as waiting but that disappeared. Stale entries from there are deleted and the corresponding slots are freed up. The worker count is included in all the redis key names as it does not vary within each $wgPoolCounterConf type and doing so handles configuration changes.

This class requires Redis 2.6 as it makes use Lua scripts for fast atomic operations. Also this should be on a server plenty of RAM for the working set to avoid evictions. Evictions could temporarily allow wait queues to double in size or temporarily cause pools to appear as full when they are not. Using volatile-ttl and bumping memory-samples in redis.conf can be helpful otherwise.

Since
1.23

Definition at line 62 of file PoolCounterRedis.php.

Constructor & Destructor Documentation

◆ __construct()

MediaWiki\PoolCounter\PoolCounterRedis::__construct ( $conf,
$type,
$key )

Definition at line 90 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounter\$key.

Member Function Documentation

◆ acquireForAnyone()

MediaWiki\PoolCounter\PoolCounterRedis::acquireForAnyone ( $timeout = null)

I want to do this task, but if anyone else does it instead, it's also fine for me.

I will read its cached data.

Parameters
int | null$timeoutWait timeout, or null to use value passed to the constructor
Returns
Status Value is one of Locked/Done/Error

Reimplemented from MediaWiki\PoolCounter\PoolCounter.

Definition at line 142 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounter\$timeout, MediaWiki\PoolCounter\PoolCounter\precheckAcquire(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ acquireForMe()

MediaWiki\PoolCounter\PoolCounterRedis::acquireForMe ( $timeout = null)

I want to do this task and I need to do it myself.

Parameters
int | null$timeoutWait timeout, or null to use value passed to the constructor
Returns
Status Value is one of Locked/Error

Reimplemented from MediaWiki\PoolCounter\PoolCounter.

Definition at line 133 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounter\$timeout, MediaWiki\PoolCounter\PoolCounter\precheckAcquire(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ getConnection()

MediaWiki\PoolCounter\PoolCounterRedis::getConnection ( )
protected
Returns
Status Uses RediConnRef as value on success

Definition at line 114 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounterRedis\$conn.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ getSlotListKey()

MediaWiki\PoolCounter\PoolCounterRedis::getSlotListKey ( )
protected
Returns
string

Definition at line 407 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\initAndPopPoolSlotList(), MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ getSlotRTimeSetKey()

MediaWiki\PoolCounter\PoolCounterRedis::getSlotRTimeSetKey ( )
protected
Returns
string

Definition at line 414 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\initAndPopPoolSlotList(), MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), and MediaWiki\PoolCounter\PoolCounterRedis\release().

◆ getWaitSetKey()

MediaWiki\PoolCounter\PoolCounterRedis::getWaitSetKey ( )
protected
Returns
string

Definition at line 421 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\initAndPopPoolSlotList(), MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ getWakeupListKey()

MediaWiki\PoolCounter\PoolCounterRedis::getWakeupListKey ( )
protected
Returns
string

Definition at line 428 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ initAndPopPoolSlotList()

MediaWiki\PoolCounter\PoolCounterRedis::initAndPopPoolSlotList ( RedisConnRef $conn,
$now )
protected
Parameters
RedisConnRef$conn
float$nowUNIX timestamp
Returns
string|bool False on failure

Definition at line 305 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounterRedis\$conn, MediaWiki\PoolCounter\PoolCounterRedis\getSlotListKey(), MediaWiki\PoolCounter\PoolCounterRedis\getSlotRTimeSetKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWaitSetKey(), and RedisConnRef\luaEval().

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ registerAcquisitionTime()

MediaWiki\PoolCounter\PoolCounterRedis::registerAcquisitionTime ( RedisConnRef $conn,
$slot,
$now )
protected
Parameters
RedisConnRef$conn
string$slot
float$now
Returns
int|bool False on failure

Definition at line 373 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounterRedis\$conn, MediaWiki\PoolCounter\PoolCounterRedis\$slot, MediaWiki\PoolCounter\PoolCounterRedis\getSlotListKey(), MediaWiki\PoolCounter\PoolCounterRedis\getSlotRTimeSetKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWaitSetKey(), and RedisConnRef\luaEval().

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ release()

MediaWiki\PoolCounter\PoolCounterRedis::release ( )

I have successfully finished my task.

Lets another one grab the lock, and returns the workers waiting on acquireForAnyone()

Returns
Status Value is one of Released/NotLocked/Error

Reimplemented from MediaWiki\PoolCounter\PoolCounter.

Definition at line 151 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounterRedis\$conn, MediaWiki\PoolCounter\PoolCounterRedis\getConnection(), MediaWiki\PoolCounter\PoolCounterRedis\getSlotListKey(), MediaWiki\PoolCounter\PoolCounterRedis\getSlotRTimeSetKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWaitSetKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWakeupListKey(), RedisConnRef\luaEval(), MediaWiki\PoolCounter\PoolCounter\NOT_LOCKED, MediaWiki\PoolCounter\PoolCounter\onRelease(), and MediaWiki\PoolCounter\PoolCounter\RELEASED.

◆ releaseAll()

static MediaWiki\PoolCounter\PoolCounterRedis::releaseAll ( )
static

Try to make sure that locks get released (even with exceptions and fatals)

Definition at line 435 of file PoolCounterRedis.php.

◆ waitForSlotOrNotif()

MediaWiki\PoolCounter\PoolCounterRedis::waitForSlotOrNotif ( $doWakeup,
$timeout = null )
protected
Parameters
int$doWakeupAWAKE_* constant
int | float | null$timeout
Returns
Status

Definition at line 241 of file PoolCounterRedis.php.

References MediaWiki\PoolCounter\PoolCounterRedis\$conn, MediaWiki\PoolCounter\PoolCounterRedis\$session, MediaWiki\PoolCounter\PoolCounterRedis\$slot, MediaWiki\PoolCounter\PoolCounterRedis\$slotTime, MediaWiki\PoolCounter\PoolCounter\$timeout, MediaWiki\PoolCounter\PoolCounter\DONE, MediaWiki\PoolCounter\PoolCounterRedis\getConnection(), MediaWiki\PoolCounter\PoolCounterRedis\getSlotListKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWaitSetKey(), MediaWiki\PoolCounter\PoolCounterRedis\getWakeupListKey(), MediaWiki\PoolCounter\PoolCounterRedis\initAndPopPoolSlotList(), MediaWiki\PoolCounter\PoolCounter\LOCK_HELD, MediaWiki\PoolCounter\PoolCounter\LOCKED, MediaWiki\PoolCounter\PoolCounter\onAcquire(), MediaWiki\PoolCounter\PoolCounter\onRelease(), MediaWiki\PoolCounter\PoolCounter\QUEUE_FULL, MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), and MediaWiki\PoolCounter\PoolCounter\TIMEOUT.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\acquireForAnyone(), and MediaWiki\PoolCounter\PoolCounterRedis\acquireForMe().

Member Data Documentation

◆ $active

PoolCounterRedis [] MediaWiki\PoolCounter\PoolCounterRedis::$active = null
staticprotected

List of active PoolCounterRedis objects in this script.

Definition at line 88 of file PoolCounterRedis.php.

◆ $conn

RedisConnRef MediaWiki\PoolCounter\PoolCounterRedis::$conn
protected

Definition at line 74 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\getConnection(), MediaWiki\PoolCounter\PoolCounterRedis\initAndPopPoolSlotList(), MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), MediaWiki\PoolCounter\PoolCounterRedis\release(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ $keySha1

string MediaWiki\PoolCounter\PoolCounterRedis::$keySha1
protected

SHA-1 of the key.

Definition at line 70 of file PoolCounterRedis.php.

◆ $lockTTL

int MediaWiki\PoolCounter\PoolCounterRedis::$lockTTL
protected

TTL for locks to expire (work should finish in this time)

Definition at line 72 of file PoolCounterRedis.php.

◆ $onRelease

int null MediaWiki\PoolCounter\PoolCounterRedis::$onRelease
protected

AWAKE_* constant.

Definition at line 78 of file PoolCounterRedis.php.

◆ $pool

RedisConnectionPool MediaWiki\PoolCounter\PoolCounterRedis::$pool
protected

Definition at line 66 of file PoolCounterRedis.php.

◆ $ring

HashRing MediaWiki\PoolCounter\PoolCounterRedis::$ring
protected

Definition at line 64 of file PoolCounterRedis.php.

◆ $serversByLabel

array MediaWiki\PoolCounter\PoolCounterRedis::$serversByLabel
protected

(server label => host) map

Definition at line 68 of file PoolCounterRedis.php.

◆ $session

string MediaWiki\PoolCounter\PoolCounterRedis::$session
protected

Unique string to identify this process.

Definition at line 80 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ $slot

string null MediaWiki\PoolCounter\PoolCounterRedis::$slot
protected

Pool slot value.

Definition at line 76 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\registerAcquisitionTime(), and MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

◆ $slotTime

float null MediaWiki\PoolCounter\PoolCounterRedis::$slotTime
protected

UNIX timestamp.

Definition at line 82 of file PoolCounterRedis.php.

Referenced by MediaWiki\PoolCounter\PoolCounterRedis\waitForSlotOrNotif().

The documentation for this class was generated from the following file: