MediaWiki  master
JobQueueDB Class Reference

Class to handle job queues stored in the DB. More...

Inheritance diagram for JobQueueDB:
Collaboration diagram for JobQueueDB:

Public Member Functions

 doBatchPushInternal (IDatabase $dbw, array $jobs, $flags, $method)
 This function should not be called outside of JobQueueDB. More...
 
 getAllAcquiredJobs ()
 
 getAllQueuedJobs ()
 
 getCoalesceLocationInternal ()
 
 recycleAndDeleteStaleJobs ()
 Recycle or destroy any jobs that have been claimed for too long. More...
 
- Public Member Functions inherited from JobQueue
 ack (RunnableJob $job)
 Acknowledge that a job was completed. More...
 
 batchPush (array $jobs, $flags=0)
 Push a batch of jobs into the queue. More...
 
 deduplicateRootJob (IJobSpecification $job)
 Register the "root job" of a given job into the queue for de-duplication. More...
 
 delayedJobsEnabled ()
 
 delete ()
 Deleted all unclaimed and delayed jobs from the queue. More...
 
 flushCaches ()
 Clear any process and persistent caches. More...
 
 getAbandonedCount ()
 Get the number of acquired jobs that can no longer be attempted. More...
 
 getAcquiredCount ()
 Get the number of acquired jobs (these are temporarily out of the queue). More...
 
 getAllAbandonedJobs ()
 Get an iterator to traverse over all abandoned jobs in this queue. More...
 
 getAllAcquiredJobs ()
 Get an iterator to traverse over all claimed jobs in this queue. More...
 
 getAllDelayedJobs ()
 Get an iterator to traverse over all delayed jobs in this queue. More...
 
 getAllQueuedJobs ()
 Get an iterator to traverse over all available jobs in this queue. More...
 
 getCoalesceLocationInternal ()
 Do not use this function outside of JobQueue/JobQueueGroup. More...
 
 getDelayedCount ()
 Get the number of delayed jobs (these are temporarily out of the queue). More...
 
 getDomain ()
 
 getOrder ()
 
 getReadOnlyReason ()
 
 getSiblingQueueSizes (array $types)
 Check the size of each of the given queues. More...
 
 getSiblingQueuesWithJobs (array $types)
 Check whether each of the given queues are empty. More...
 
 getSize ()
 Get the number of available (unacquired, non-delayed) jobs in the queue. More...
 
 getType ()
 
 getWiki ()
 
 isEmpty ()
 Quickly check if the queue has no available (unacquired, non-delayed) jobs. More...
 
 pop ()
 Pop a job off of the queue. More...
 
 push ( $jobs, $flags=0)
 Push one or more jobs into the queue. More...
 
 waitForBackups ()
 Wait for any replica DBs or backup servers to catch up. More...
 

Static Public Member Functions

static selectFields ()
 Return the list of job fields that should be selected. More...
 
- Static Public Member Functions inherited from JobQueue
static factory (array $params)
 Get a job queue object of the specified type. More...
 

Public Attributes

const CACHE_TTL_SHORT = 30
 
const MAX_AGE_PRUNE = 604800
 
const MAX_JOB_RANDOM = 2147483647
 
const MAX_OFFSET = 255
 
- Public Attributes inherited from JobQueue
const QOS_ATOMIC = 1
 
const ROOTJOB_TTL = 2419200
 

Protected Member Functions

 __construct (array $params)
 Additional parameters include: More...
 
 claimOldest ( $uuid)
 Reserve a row with a single UPDATE without holding row locks over RTTs... More...
 
 claimRandom ( $uuid, $rand, $gte)
 Reserve a row with a single UPDATE without holding row locks over RTTs... More...
 
 doAck (RunnableJob $job)
 
 doBatchPush (array $jobs, $flags)
 
 doDeduplicateRootJob (IJobSpecification $job)
 
 doDelete ()
 
 doFlushCaches ()
 
 doGetAbandonedCount ()
 
 doGetAcquiredCount ()
 
 doGetSiblingQueueSizes (array $types)
 
 doGetSiblingQueuesWithJobs (array $types)
 
 doGetSize ()
 
 doIsEmpty ()
 
 doPop ()
 
 doWaitForBackups ()
 
 getDB ( $index)
 
 getDBException (DBError $e)
 
 getJobIterator (array $conds)
 
 getMasterDB ()
 
 getReplicaDB ()
 
 insertFields (IJobSpecification $job, IDatabase $db)
 
 jobFromRow ( $row)
 
 optimalOrder ()
 
 supportedOrders ()
 
- Protected Member Functions inherited from JobQueue
 __construct (array $params)
 
 assertNotReadOnly ()
 
 doAck (RunnableJob $job)
 
 doBatchPush (array $jobs, $flags)
 
 doDeduplicateRootJob (IJobSpecification $job)
 
 doDelete ()
 
 doFlushCaches ()
 
 doGetAbandonedCount ()
 
 doGetAcquiredCount ()
 
 doGetDelayedCount ()
 
 doGetSiblingQueueSizes (array $types)
 
 doGetSiblingQueuesWithJobs (array $types)
 
 doGetSize ()
 
 doIsEmpty ()
 
 doIsRootJobOldDuplicate (IJobSpecification $job)
 
 doPop ()
 
 doWaitForBackups ()
 
 factoryJob ( $command, $params)
 
 getRootJobCacheKey ( $signature)
 
 incrStats ( $key, $type, $delta=1)
 Call wfIncrStats() for the queue overall and for the queue type. More...
 
 isRootJobOldDuplicate (IJobSpecification $job)
 Check if the "root" job of a given job has been superseded by a newer one. More...
 
 optimalOrder ()
 Get the default queue order to use if configuration does not specify one. More...
 
 supportedOrders ()
 Get the allowed queue orders for configuration validation. More...
 
 supportsDelayedJobs ()
 Find out if delayed jobs are supported for configuration validation. More...
 

Static Protected Member Functions

static makeBlob ( $params)
 

Protected Attributes

WANObjectCache $cache
 
string null $cluster
 Name of an external DB cluster or null for the local DB cluster. More...
 
IDatabase DBError null $conn
 
array null $server
 Server configuration array. More...
 
- Protected Attributes inherited from JobQueue
int $claimTTL
 Time to live in seconds. More...
 
string $domain
 DB domain ID. More...
 
BagOStuff $dupCache
 
int $maxTries
 Maximum number of times to try a job. More...
 
string $order
 Job priority for pop() More...
 
string bool $readOnlyReason
 Read only rationale (or false if r/w) More...
 
StatsdDataFactoryInterface $stats
 
string $type
 Job type. More...
 

Private Member Functions

 getCacheKey ( $property)
 
 getScopedNoTrxFlag (IDatabase $db)
 

Detailed Description

Class to handle job queues stored in the DB.

Since
1.21

Definition at line 35 of file JobQueueDB.php.

Constructor & Destructor Documentation

◆ __construct()

JobQueueDB::__construct ( array  $params)
protected

Additional parameters include:

  • server : Server configuration array for Database::factory. Overrides "cluster".
  • cluster : The name of an external cluster registered via LBFactory. If not specified, the primary DB cluster for the wiki will be used. This can be overridden with a custom cluster so that DB handles will be retrieved via LBFactory::getExternalLB() and getConnection().
  • wanCache : An instance of WANObjectCache to use for caching.
    Parameters
    array$params

Definition at line 61 of file JobQueueDB.php.

References cache, WANObjectCache\newEmpty(), and server.

Member Function Documentation

◆ claimOldest()

JobQueueDB::claimOldest (   $uuid)
protected

Reserve a row with a single UPDATE without holding row locks over RTTs...

Parameters
string$uuid32 char hex string
Returns
stdClass|bool Row|false

Definition at line 421 of file JobQueueDB.php.

References getMasterDB(), getScopedNoTrxFlag(), type, and wfDebug().

Referenced by doPop().

◆ claimRandom()

JobQueueDB::claimRandom (   $uuid,
  $rand,
  $gte 
)
protected

Reserve a row with a single UPDATE without holding row locks over RTTs...

Parameters
string$uuid32 char hex string
int$randRandom unsigned integer (31 bits)
bool$gteSearch for job_random >= $random (otherwise job_random <= $random)
Returns
stdClass|bool Row|false

Definition at line 343 of file JobQueueDB.php.

References JobQueue\$type, cache, getCacheKey(), getMasterDB(), getScopedNoTrxFlag(), and type.

Referenced by doPop().

◆ doAck()

JobQueueDB::doAck ( RunnableJob  $job)
protected

◆ doBatchPush()

JobQueueDB::doBatchPush ( array  $jobs,
  $flags 
)
protected
See also
JobQueue::doBatchPush()
Parameters
IJobSpecification[]$jobs
int$flags
Exceptions
DBError|Exception
Returns
void

Definition at line 205 of file JobQueueDB.php.

References $fname, doBatchPushInternal(), getMasterDB(), getScopedNoTrxFlag(), and use.

◆ doBatchPushInternal()

JobQueueDB::doBatchPushInternal ( IDatabase  $dbw,
array  $jobs,
  $flags,
  $method 
)

This function should not be called outside of JobQueueDB.

SecurityCheck-SQLInjection Bug in phan-taint-check handling bulk inserts

Parameters
IDatabase$dbw
IJobSpecification[]$jobs
int$flags
string$method
Exceptions
DBError
Returns
void

Definition at line 238 of file JobQueueDB.php.

References $e, $job, $res, $rows, as, Wikimedia\Rdbms\IDatabase\endAtomic(), getDBException(), JobQueue\incrStats(), Wikimedia\Rdbms\IDatabase\insert(), insertFields(), Wikimedia\Rdbms\IDatabase\select(), Wikimedia\Rdbms\IDatabase\startAtomic(), type, and wfDebug().

Referenced by doBatchPush().

◆ doDeduplicateRootJob()

◆ doDelete()

JobQueueDB::doDelete ( )
protected
See also
JobQueue::doDelete()
Returns
bool

Definition at line 550 of file JobQueueDB.php.

References $e, getDBException(), getMasterDB(), getScopedNoTrxFlag(), and type.

◆ doFlushCaches()

JobQueueDB::doFlushCaches ( )
protected
Returns
void

Definition at line 582 of file JobQueueDB.php.

References JobQueue\$type, as, cache, and getCacheKey().

◆ doGetAbandonedCount()

JobQueueDB::doGetAbandonedCount ( )
protected
See also
JobQueue::doGetAbandonedCount()
Returns
int
Exceptions
MWException

Definition at line 165 of file JobQueueDB.php.

References $dbr, $e, cache, getCacheKey(), getDBException(), getReplicaDB(), and getScopedNoTrxFlag().

◆ doGetAcquiredCount()

JobQueueDB::doGetAcquiredCount ( )
protected

◆ doGetSiblingQueueSizes()

JobQueueDB::doGetSiblingQueueSizes ( array  $types)
protected

Definition at line 653 of file JobQueueDB.php.

References $dbr, $res, as, getReplicaDB(), and getScopedNoTrxFlag().

◆ doGetSiblingQueuesWithJobs()

JobQueueDB::doGetSiblingQueuesWithJobs ( array  $types)
protected

Definition at line 634 of file JobQueueDB.php.

References $dbr, $res, as, getReplicaDB(), and getScopedNoTrxFlag().

◆ doGetSize()

JobQueueDB::doGetSize ( )
protected
See also
JobQueue::doGetSize()
Returns
int

Definition at line 104 of file JobQueueDB.php.

References $dbr, $e, cache, getCacheKey(), getDBException(), getReplicaDB(), and getScopedNoTrxFlag().

◆ doIsEmpty()

JobQueueDB::doIsEmpty ( )
protected
See also
JobQueue::doIsEmpty()
Returns
bool

Definition at line 85 of file JobQueueDB.php.

References $dbr, $e, getDBException(), getReplicaDB(), getScopedNoTrxFlag(), and type.

◆ doPop()

JobQueueDB::doPop ( )
protected

◆ doWaitForBackups()

JobQueueDB::doWaitForBackups ( )
protected
See also
JobQueue::doWaitForBackups()
Returns
void

Definition at line 567 of file JobQueueDB.php.

References server.

◆ getAllAcquiredJobs()

JobQueueDB::getAllAcquiredJobs ( )
See also
JobQueue::getAllAcquiredJobs()
Returns
Iterator

Definition at line 600 of file JobQueueDB.php.

References getJobIterator(), and JobQueue\getType().

◆ getAllQueuedJobs()

JobQueueDB::getAllQueuedJobs ( )
See also
JobQueue::getAllQueuedJobs()
Returns
Iterator

Definition at line 592 of file JobQueueDB.php.

References getJobIterator(), and JobQueue\getType().

◆ getCacheKey()

JobQueueDB::getCacheKey (   $property)
private
Parameters
string$property
Returns
string

Definition at line 856 of file JobQueueDB.php.

References $property, cache, and type.

Referenced by claimRandom(), doFlushCaches(), doGetAbandonedCount(), doGetAcquiredCount(), and doGetSize().

◆ getCoalesceLocationInternal()

JobQueueDB::getCoalesceLocationInternal ( )

Definition at line 624 of file JobQueueDB.php.

References null, and server.

◆ getDB()

JobQueueDB::getDB (   $index)
protected
Parameters
int$index(DB_REPLICA/DB_MASTER)
Returns
IDatabase

Definition at line 806 of file JobQueueDB.php.

References $conn, $e, and server.

Referenced by getMasterDB(), and getReplicaDB().

◆ getDBException()

JobQueueDB::getDBException ( DBError  $e)
protected

◆ getJobIterator()

JobQueueDB::getJobIterator ( array  $conds)
protected
Parameters
array$condsQuery conditions
Returns
Iterator

Definition at line 608 of file JobQueueDB.php.

References $dbr, $e, getDBException(), getReplicaDB(), getScopedNoTrxFlag(), and jobFromRow().

Referenced by getAllAcquiredJobs(), and getAllQueuedJobs().

◆ getMasterDB()

JobQueueDB::getMasterDB ( )
protected
Exceptions
JobQueueConnectionError
Returns
IDatabase

Definition at line 794 of file JobQueueDB.php.

References $e, DB_MASTER, and getDB().

Referenced by claimOldest(), claimRandom(), doAck(), doBatchPush(), doDeduplicateRootJob(), doDelete(), doPop(), and recycleAndDeleteStaleJobs().

◆ getReplicaDB()

JobQueueDB::getReplicaDB ( )
protected

◆ getScopedNoTrxFlag()

◆ insertFields()

JobQueueDB::insertFields ( IJobSpecification  $job,
IDatabase  $db 
)
protected

◆ jobFromRow()

JobQueueDB::jobFromRow (   $row)
protected
Parameters
stdClass$row
Returns
RunnableJob|null

Definition at line 884 of file JobQueueDB.php.

References $job, $params, JobQueue\factoryJob(), string, and unserialize().

Referenced by doPop(), and getJobIterator().

◆ makeBlob()

static JobQueueDB::makeBlob (   $params)
staticprotected
Parameters
array | bool$params
Returns
string

Definition at line 872 of file JobQueueDB.php.

References $params, and serialize().

◆ optimalOrder()

JobQueueDB::optimalOrder ( )
protected

Definition at line 77 of file JobQueueDB.php.

◆ recycleAndDeleteStaleJobs()

JobQueueDB::recycleAndDeleteStaleJobs ( )

Recycle or destroy any jobs that have been claimed for too long.

Returns
int Number of jobs recycled/deleted

Definition at line 674 of file JobQueueDB.php.

References $e, $res, JobQueue\$type, getDBException(), getMasterDB(), getScopedNoTrxFlag(), JobQueue\incrStats(), and type.

Referenced by doPop().

◆ selectFields()

static JobQueueDB::selectFields ( )
static

Return the list of job fields that should be selected.

Since
1.23
Returns
array

Definition at line 912 of file JobQueueDB.php.

◆ supportedOrders()

JobQueueDB::supportedOrders ( )
protected

Definition at line 73 of file JobQueueDB.php.

Member Data Documentation

◆ $cache

WANObjectCache JobQueueDB::$cache
protected

Definition at line 42 of file JobQueueDB.php.

◆ $cluster

string null JobQueueDB::$cluster
protected

Name of an external DB cluster or null for the local DB cluster.

Definition at line 49 of file JobQueueDB.php.

◆ $conn

IDatabase DBError null JobQueueDB::$conn
protected

Definition at line 44 of file JobQueueDB.php.

Referenced by getDB().

◆ $server

array null JobQueueDB::$server
protected

Server configuration array.

Definition at line 47 of file JobQueueDB.php.

◆ CACHE_TTL_SHORT

const JobQueueDB::CACHE_TTL_SHORT = 30

Definition at line 36 of file JobQueueDB.php.

◆ MAX_AGE_PRUNE

const JobQueueDB::MAX_AGE_PRUNE = 604800

Definition at line 37 of file JobQueueDB.php.

◆ MAX_JOB_RANDOM

const JobQueueDB::MAX_JOB_RANDOM = 2147483647

Definition at line 38 of file JobQueueDB.php.

◆ MAX_OFFSET

const JobQueueDB::MAX_OFFSET = 255

Definition at line 39 of file JobQueueDB.php.


The documentation for this class was generated from the following file: