Database-backed job queue storage. More...

Collaboration diagram for JobQueueDB:

[legend]

Public Member Functions
	doBatchPushInternal (IDatabase $dbw, array $jobs, $flags, $method)
	This function should not be called outside of JobQueueDB.

	getAllAbandonedJobs ()

	getAllAcquiredJobs ()

	getAllQueuedJobs ()

	getCoalesceLocationInternal ()
	Do not use this function outside of JobQueue/JobQueueGroup.

	recycleAndDeleteStaleJobs ()
	Recycle or destroy any jobs that have been claimed for too long.

Public Member Functions inherited from JobQueue
	ack (RunnableJob $job)
	Acknowledge that a job was completed.

	batchPush (array $jobs, $flags=0)
	Push a batch of jobs into the queue.

	deduplicateRootJob (IJobSpecification $job)
	Register the "root job" of a given job into the queue for de-duplication.

	delayedJobsEnabled ()

	delete ()
	Delete all unclaimed and delayed jobs from the queue.

	flushCaches ()
	Clear any process and persistent caches.

	getAbandonedCount ()
	Get the number of acquired jobs that can no longer be attempted.

	getAcquiredCount ()
	Get the number of acquired jobs (these are temporarily out of the queue).

	getAllDelayedJobs ()
	Get an iterator to traverse over all delayed jobs in this queue.

	getDelayedCount ()
	Get the number of delayed jobs (these are temporarily out of the queue).

	getDomain ()

	getOrder ()

	getReadOnlyReason ()

	getSiblingQueueSizes (array $types)
	Check the size of each of the given queues.

	getSiblingQueuesWithJobs (array $types)
	Check whether each of the given queues are empty.

	getSize ()
	Get the number of available (unacquired, non-delayed) jobs in the queue.

	getType ()

	isEmpty ()
	Quickly check if the queue has no available (unacquired, non-delayed) jobs.

	pop ()
	Pop a job off of the queue.

	push ( $jobs, $flags=0)
	Push one or more jobs into the queue.

	waitForBackups ()
	Wait for any replica DBs or backup servers to catch up.

Static Public Member Functions
static	selectFields ()
	Return the list of job fields that should be selected.

Static Public Member Functions inherited from JobQueue
static	factory (array $params)
	Get a job queue object of the specified type.

Protected Member Functions
	__construct (array $params)
	Additional parameters include:

	claimOldest ( $uuid)
	Reserve a row with a single UPDATE without holding row locks over RTTs...

	claimRandom ( $uuid, $rand, $gte)
	Reserve a row with a single UPDATE without holding row locks over RTTs...

	doAck (RunnableJob $job)

	doBatchPush (array $jobs, $flags)

	doDeduplicateRootJob (IJobSpecification $job)

	doDelete ()

	doFlushCaches ()

	doGetAbandonedCount ()

	doGetAcquiredCount ()

	doGetSiblingQueueSizes (array $types)

	doGetSiblingQueuesWithJobs (array $types)

	doGetSize ()

	doIsEmpty ()

	doPop ()

	doWaitForBackups ()

	getDB ( $index)

	getDBException (DBError $e)

	getJobIterator (array $conds)

	getPrimaryDB ()

	getReplicaDB ()

	insertFields (IJobSpecification $job, IReadableDatabase $db)

	jobFromRow ( $row)

	optimalOrder ()
	Get the default queue order to use if configuration does not specify one.

	supportedOrders ()
	Get the allowed queue orders for configuration validation.

Protected Member Functions inherited from JobQueue
	assertNotReadOnly ()

	doGetDelayedCount ()

	doIsRootJobOldDuplicate (IJobSpecification $job)

	factoryJob ( $command, $params)

	getRootJobCacheKey ( $signature, $type)

	incrStats ( $key, $type, $delta=1)
	Call StatsdDataFactoryInterface::updateCount() for the queue overall and for the queue type.

	isRootJobOldDuplicate (IJobSpecification $job)
	Check if the "root" job of a given job has been superseded by a newer one.

	supportsDelayedJobs ()
	Find out if delayed jobs are supported for configuration validation.

	supportsTypeAgnostic ()
	Subclasses should set this to true if they support type agnostic queues.

Static Protected Member Functions
static	makeBlob ( $params)

Protected Attributes
string null	$cluster
	Name of an external DB cluster or null for the local DB cluster.

IMaintainableDatabase DBError null	$conn

array null	$server
	Server configuration array.

Protected Attributes inherited from JobQueue
int	$claimTTL
	Time to live in seconds.

string	$domain
	DB domain ID.

GlobalIdGenerator	$idGenerator

int	$maxTries
	Maximum number of times to try a job.

string	$order
	Job priority for pop()

string false	$readOnlyReason
	Read only rationale (or false if r/w)

StatsdDataFactoryInterface	$stats

string	$type
	Job type.

bool	$typeAgnostic

WANObjectCache	$wanCache

const	QOS_ATOMIC = 1

const	ROOTJOB_TTL = 28 * 24 * 3600

Detailed Description

Database-backed job queue storage.

Since: 1.21

Definition at line 37 of file JobQueueDB.php.

Constructor & Destructor Documentation

◆ __construct()

JobQueueDB::__construct ( array $params )

protected

Additional parameters include:

server : Server configuration array for Database::factory. Overrides "cluster".
cluster : The name of an external cluster registered via LBFactory. If not specified, the primary DB cluster for the wiki will be used. This can be overridden with a custom cluster so that DB handles will be retrieved via LBFactory::getExternalLB() and getConnection().
Parameters

array $params

Reimplemented from JobQueue.

Definition at line 67 of file JobQueueDB.php.

References $params.

Member Function Documentation

◆ claimOldest()

JobQueueDB::claimOldest ( $uuid )

protected

Reserve a row with a single UPDATE without holding row locks over RTTs...

Parameters

string $uuid 32 char hex string

Returns: stdClass|false Row|false

Definition at line 450 of file JobQueueDB.php.

References getPrimaryDB(), and wfDebug().

Referenced by doPop().

◆ claimRandom()

JobQueueDB::claimRandom	(	$uuid,
		$rand,
		$gte )

protected

Reserve a row with a single UPDATE without holding row locks over RTTs...

Parameters

string	$uuid	32 char hex string
int	$rand	Random unsigned integer (31 bits)
bool	$gte	Search for job_random >= $random (otherwise job_random <= $random)

Returns: stdClass|false Row|false

Definition at line 360 of file JobQueueDB.php.

References getCacheKey(), and getPrimaryDB().

Referenced by doPop().

◆ doAck()

JobQueueDB::doAck ( RunnableJob $job )

protected

See also: JobQueue::doAck()

Parameters

RunnableJob $job

Exceptions

JobQueueConnectionError
JobQueueError

Reimplemented from JobQueue.

Definition at line 516 of file JobQueueDB.php.

References $job, getDBException(), getPrimaryDB(), and JobQueue\incrStats().

◆ doBatchPush()

JobQueueDB::doBatchPush	(	array	$jobs,
			$flags )

protected

See also: JobQueue::doBatchPush()

Parameters

IJobSpecification[]	$jobs
int	$flags

Exceptions

DBError|Exception

Returns: void

Reimplemented from JobQueue.

Definition at line 213 of file JobQueueDB.php.

References doBatchPushInternal(), getPrimaryDB(), and Profiler\instance().

◆ doBatchPushInternal()

JobQueueDB::doBatchPushInternal	(	IDatabase	$dbw,
		array	$jobs,
			$flags,
			$method )

This function should not be called outside of JobQueueDB.

Parameters

IDatabase	$dbw
IJobSpecification[]	$jobs
int	$flags
string	$method

Exceptions

DBError

Returns: void

Definition at line 248 of file JobQueueDB.php.

References $job, Wikimedia\Rdbms\IDatabase\endAtomic(), getDBException(), JobQueue\incrStats(), insertFields(), Profiler\instance(), Wikimedia\Rdbms\IDatabase\newInsertQueryBuilder(), Wikimedia\Rdbms\IReadableDatabase\newSelectQueryBuilder(), Wikimedia\Rdbms\IDatabase\startAtomic(), and wfDebug().

Referenced by doBatchPush().

◆ doDeduplicateRootJob()

JobQueueDB::doDeduplicateRootJob ( IJobSpecification $job )

protected

See also: JobQueue::doDeduplicateRootJob()

Parameters

IJobSpecification $job

Exceptions

JobQueueConnectionError

Returns: bool

Reimplemented from JobQueue.

Definition at line 542 of file JobQueueDB.php.

References $job, and getPrimaryDB().

◆ doDelete()

JobQueueDB::doDelete ( )

protected

See also: JobQueue::doDelete()

Returns: bool

Reimplemented from JobQueue.

Definition at line 563 of file JobQueueDB.php.

References getDBException(), and getPrimaryDB().

◆ doFlushCaches()

JobQueueDB::doFlushCaches ( )

protected

Returns: void

Reimplemented from JobQueue.

Definition at line 593 of file JobQueueDB.php.

References JobQueue\$type, and getCacheKey().

◆ doGetAbandonedCount()

JobQueueDB::doGetAbandonedCount ( )

protected

See also: JobQueue::doGetAbandonedCount()

Returns: int

Exceptions

JobQueueConnectionError
JobQueueError

Reimplemented from JobQueue.

Definition at line 173 of file JobQueueDB.php.

References getCacheKey(), getDBException(), and getReplicaDB().

◆ doGetAcquiredCount()

JobQueueDB::doGetAcquiredCount ( )

protected

See also: JobQueue::doGetAcquiredCount()

Returns: int

Reimplemented from JobQueue.

Definition at line 138 of file JobQueueDB.php.

References getCacheKey(), getDBException(), and getReplicaDB().

◆ doGetSiblingQueueSizes()

JobQueueDB::doGetSiblingQueueSizes ( array $types )

protected

Stability: stable: to override

See also: JobQueue::getSiblingQueuesSize()

Parameters

array $types List of queues types

Returns: array|null (list of queue types) or null if unsupported

Reimplemented from JobQueue.

Definition at line 682 of file JobQueueDB.php.

References getReplicaDB().

◆ doGetSiblingQueuesWithJobs()

JobQueueDB::doGetSiblingQueuesWithJobs ( array $types )

protected

Stability: stable: to override

See also: JobQueue::getSiblingQueuesWithJobs()

Parameters

array $types List of queues types

Returns: array|null (list of queue types) or null if unsupported

Reimplemented from JobQueue.

Definition at line 661 of file JobQueueDB.php.

References getReplicaDB().

◆ doGetSize()

JobQueueDB::doGetSize ( )

protected

See also: JobQueue::doGetSize()

Returns: int

Reimplemented from JobQueue.

Definition at line 112 of file JobQueueDB.php.

References getCacheKey(), getDBException(), and getReplicaDB().

◆ doIsEmpty()

JobQueueDB::doIsEmpty ( )

protected

See also: JobQueue::doIsEmpty()

Returns: bool

Reimplemented from JobQueue.

Definition at line 92 of file JobQueueDB.php.

References getDBException(), and getReplicaDB().

◆ doPop()

JobQueueDB::doPop ( )

protected

See also: JobQueue::doPop()

Returns: RunnableJob|false

Reimplemented from JobQueue.

Definition at line 316 of file JobQueueDB.php.

References $job, claimOldest(), claimRandom(), getDBException(), JobQueue\incrStats(), jobFromRow(), recycleAndDeleteStaleJobs(), and wfRandomString().

◆ doWaitForBackups()

JobQueueDB::doWaitForBackups ( )

protected

See also: JobQueue::doWaitForBackups()

Returns: void

Reimplemented from JobQueue.

Definition at line 581 of file JobQueueDB.php.

◆ getAllAbandonedJobs()

JobQueueDB::getAllAbandonedJobs ( )

See also: JobQueue::getAllAbandonedJobs()

Returns: Iterator<RunnableJob>

Reimplemented from JobQueue.

Definition at line 620 of file JobQueueDB.php.

References getJobIterator(), getReplicaDB(), and JobQueue\getType().

◆ getAllAcquiredJobs()

JobQueueDB::getAllAcquiredJobs ( )

See also: JobQueue::getAllAcquiredJobs()

Returns: Iterator<RunnableJob>

Reimplemented from JobQueue.

Definition at line 611 of file JobQueueDB.php.

References getJobIterator(), getReplicaDB(), and JobQueue\getType().

◆ getAllQueuedJobs()

JobQueueDB::getAllQueuedJobs ( )

See also: JobQueue::getAllQueuedJobs()

Returns: Iterator<RunnableJob>

Reimplemented from JobQueue.

Definition at line 603 of file JobQueueDB.php.

References getJobIterator(), and JobQueue\getType().

◆ getCoalesceLocationInternal()

JobQueueDB::getCoalesceLocationInternal ( )

Do not use this function outside of JobQueue/JobQueueGroup.

Stability: stable: to override

Returns: string|null

Since: 1.22

Reimplemented from JobQueue.

Definition at line 651 of file JobQueueDB.php.

◆ getDB()

JobQueueDB::getDB ( $index )

protected

Parameters

int $index (DB_REPLICA/DB_PRIMARY)

Returns: IMaintainableDatabase

Definition at line 852 of file JobQueueDB.php.

Referenced by getPrimaryDB(), and getReplicaDB().

◆ getDBException()

JobQueueDB::getDBException ( DBError $e )

protected

Parameters

DBError $e

Returns: JobQueueError

Definition at line 941 of file JobQueueDB.php.

Referenced by doAck(), doBatchPushInternal(), doDelete(), doGetAbandonedCount(), doGetAcquiredCount(), doGetSize(), doIsEmpty(), doPop(), getJobIterator(), and recycleAndDeleteStaleJobs().

◆ getJobIterator()

JobQueueDB::getJobIterator ( array $conds )

protected

Parameters

array $conds Query conditions

Returns: Iterator<RunnableJob>

Definition at line 633 of file JobQueueDB.php.

References getDBException(), getReplicaDB(), and jobFromRow().

Referenced by getAllAbandonedJobs(), getAllAcquiredJobs(), and getAllQueuedJobs().

◆ getPrimaryDB()

JobQueueDB::getPrimaryDB ( )

protected

Exceptions

JobQueueConnectionError

Returns: IMaintainableDatabase

Since: 1.37

Definition at line 840 of file JobQueueDB.php.

References DB_PRIMARY, and getDB().

Referenced by claimOldest(), claimRandom(), doAck(), doBatchPush(), doDeduplicateRootJob(), doDelete(), and recycleAndDeleteStaleJobs().

◆ getReplicaDB()

JobQueueDB::getReplicaDB ( )

protected

Exceptions

JobQueueConnectionError

Returns: IDatabase

Definition at line 827 of file JobQueueDB.php.

References DB_REPLICA, and getDB().

Referenced by doGetAbandonedCount(), doGetAcquiredCount(), doGetSiblingQueueSizes(), doGetSiblingQueuesWithJobs(), doGetSize(), doIsEmpty(), getAllAbandonedJobs(), getAllAcquiredJobs(), and getJobIterator().

◆ insertFields()

JobQueueDB::insertFields	(	IJobSpecification	$job,
		IReadableDatabase	$db )

protected

Parameters

IJobSpecification	$job
IReadableDatabase	$db

Returns: array

Definition at line 806 of file JobQueueDB.php.

References $job, NS_SPECIAL, and Wikimedia\Rdbms\Platform\ISQLPlatform\timestamp().

Referenced by doBatchPushInternal().

◆ jobFromRow()

JobQueueDB::jobFromRow ( $row )

protected

Parameters

stdClass $row

Returns: RunnableJob

Definition at line 922 of file JobQueueDB.php.

References $job, $params, and JobQueue\factoryJob().

Referenced by doPop(), and getJobIterator().

◆ makeBlob()

static JobQueueDB::makeBlob ( $params )

staticprotected

Parameters

array | false $params

Returns: string

Definition at line 910 of file JobQueueDB.php.

References $params.

◆ optimalOrder()

JobQueueDB::optimalOrder ( )

protected

Get the default queue order to use if configuration does not specify one.

Returns: string One of (random, timestamp, fifo, undefined)

Reimplemented from JobQueue.

Definition at line 84 of file JobQueueDB.php.

◆ recycleAndDeleteStaleJobs()

JobQueueDB::recycleAndDeleteStaleJobs ( )

Recycle or destroy any jobs that have been claimed for too long.

Returns: int Number of jobs recycled/deleted

Definition at line 705 of file JobQueueDB.php.

References getDBException(), getPrimaryDB(), and JobQueue\incrStats().

Referenced by doPop().

◆ selectFields()

static JobQueueDB::selectFields ( )

static

Return the list of job fields that should be selected.

Since: 1.23

Returns: array

Definition at line 950 of file JobQueueDB.php.

◆ supportedOrders()

JobQueueDB::supportedOrders ( )

protected

Get the allowed queue orders for configuration validation.

Returns: array Subset of (random, timestamp, fifo, undefined)

Reimplemented from JobQueue.

Definition at line 80 of file JobQueueDB.php.

Member Data Documentation

◆ $cluster

string null JobQueueDB::$cluster

protected

Name of an external DB cluster or null for the local DB cluster.

Definition at line 56 of file JobQueueDB.php.

◆ $conn

IMaintainableDatabase DBError null JobQueueDB::$conn

protected

Definition at line 51 of file JobQueueDB.php.

◆ $server

array null JobQueueDB::$server

protected

Server configuration array.

Definition at line 54 of file JobQueueDB.php.

The documentation for this class was generated from the following file:

includes/jobqueue/JobQueueDB.php

Public Member Functions

Static Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ claimOldest()

◆ claimRandom()

◆ doAck()

◆ doBatchPush()

◆ doBatchPushInternal()

◆ doDeduplicateRootJob()

◆ doDelete()

◆ doFlushCaches()

◆ doGetAbandonedCount()

◆ doGetAcquiredCount()

◆ doGetSiblingQueueSizes()

◆ doGetSiblingQueuesWithJobs()

◆ doGetSize()

◆ doIsEmpty()

◆ doPop()

◆ doWaitForBackups()

◆ getAllAbandonedJobs()

◆ getAllAcquiredJobs()

◆ getAllQueuedJobs()

◆ getCoalesceLocationInternal()

◆ getDB()

◆ getDBException()

◆ getJobIterator()

◆ getPrimaryDB()

◆ getReplicaDB()

◆ insertFields()

◆ jobFromRow()

◆ makeBlob()

◆ optimalOrder()

◆ recycleAndDeleteStaleJobs()

◆ selectFields()

◆ supportedOrders()

Member Data Documentation

◆ $cluster

◆ $conn

◆ $server