MediaWiki REL1_35
|
Class to handle enqueueing and running of background jobs. More...
Public Member Functions | |
ack (RunnableJob $job) | |
Acknowledge that a job was completed. | |
batchPush (array $jobs, $flags=0) | |
Push a batch of jobs into the queue. | |
deduplicateRootJob (IJobSpecification $job) | |
Register the "root job" of a given job into the queue for de-duplication. | |
delayedJobsEnabled () | |
delete () | |
Delete all unclaimed and delayed jobs from the queue. | |
flushCaches () | |
Clear any process and persistent caches. | |
getAbandonedCount () | |
Get the number of acquired jobs that can no longer be attempted. | |
getAcquiredCount () | |
Get the number of acquired jobs (these are temporarily out of the queue). | |
getAllAbandonedJobs () | |
Get an iterator to traverse over all abandoned jobs in this queue. | |
getAllAcquiredJobs () | |
Get an iterator to traverse over all claimed jobs in this queue. | |
getAllDelayedJobs () | |
Get an iterator to traverse over all delayed jobs in this queue. | |
getAllQueuedJobs () | |
Get an iterator to traverse over all available jobs in this queue. | |
getCoalesceLocationInternal () | |
Do not use this function outside of JobQueue/JobQueueGroup. | |
getDelayedCount () | |
Get the number of delayed jobs (these are temporarily out of the queue). | |
getDomain () | |
getOrder () | |
getReadOnlyReason () | |
getSiblingQueueSizes (array $types) | |
Check the size of each of the given queues. | |
getSiblingQueuesWithJobs (array $types) | |
Check whether each of the given queues are empty. | |
getSize () | |
Get the number of available (unacquired, non-delayed) jobs in the queue. | |
getType () | |
getWiki () | |
isEmpty () | |
Quickly check if the queue has no available (unacquired, non-delayed) jobs. | |
pop () | |
Pop a job off of the queue. | |
push ( $jobs, $flags=0) | |
Push one or more jobs into the queue. | |
waitForBackups () | |
Wait for any replica DBs or backup servers to catch up. | |
Static Public Member Functions | |
static | factory (array $params) |
Get a job queue object of the specified type. | |
Protected Member Functions | |
__construct (array $params) | |
Stable to call. | |
assertNotReadOnly () | |
doAck (RunnableJob $job) | |
doBatchPush (array $jobs, $flags) | |
doDeduplicateRootJob (IJobSpecification $job) | |
Stable to override. | |
doDelete () | |
Stable to override. | |
doFlushCaches () | |
Stable to override. | |
doGetAbandonedCount () | |
Stable to override. | |
doGetAcquiredCount () | |
doGetDelayedCount () | |
Stable to override. | |
doGetSiblingQueueSizes (array $types) | |
Stable to override. | |
doGetSiblingQueuesWithJobs (array $types) | |
Stable to override. | |
doGetSize () | |
doIsEmpty () | |
doIsRootJobOldDuplicate (IJobSpecification $job) | |
Stable to override. | |
doPop () | |
doWaitForBackups () | |
Stable to override. | |
factoryJob ( $command, $params) | |
getRootJobCacheKey ( $signature) | |
incrStats ( $key, $type, $delta=1) | |
Call StatsdDataFactoryInterface::updateCount() for the queue overall and for the queue type. | |
isRootJobOldDuplicate (IJobSpecification $job) | |
Check if the "root" job of a given job has been superseded by a newer one. | |
optimalOrder () | |
Get the default queue order to use if configuration does not specify one. | |
supportedOrders () | |
Get the allowed queue orders for configuration validation. | |
supportsDelayedJobs () | |
Find out if delayed jobs are supported for configuration validation. | |
Protected Attributes | |
int | $claimTTL |
Time to live in seconds. | |
string | $domain |
DB domain ID. | |
GlobalIdGenerator | $idGenerator |
int | $maxTries |
Maximum number of times to try a job. | |
string | $order |
Job priority for pop() | |
string bool | $readOnlyReason |
Read only rationale (or false if r/w) | |
StatsdDataFactoryInterface | $stats |
string | $type |
Job type. | |
WANObjectCache | $wanCache |
const | QOS_ATOMIC = 1 |
const | ROOTJOB_TTL = 2419200 |
Class to handle enqueueing and running of background jobs.
See the architecture doc for more information.
Definition at line 35 of file JobQueue.php.
|
protected |
Stable to call.
array | $params |
|
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Definition at line 76 of file JobQueue.php.
References optimalOrder(), and supportedOrders().
|
final |
Acknowledge that a job was completed.
This does nothing for certain queue classes or if "claimTTL" is not set. Outside callers should use JobQueueGroup::ack() instead of this function.
RunnableJob | $job |
JobQueueError |
Definition at line 432 of file JobQueue.php.
References assertNotReadOnly(), doAck(), and IJobSpecification\getType().
|
protected |
JobQueueReadOnlyError |
Definition at line 747 of file JobQueue.php.
Referenced by ack(), batchPush(), deduplicateRootJob(), delete(), and pop().
|
final |
Push a batch of jobs into the queue.
This does not require $wgJobClasses to be set for the given job type. Outside callers should use JobQueueGroup::push() instead of this function.
IJobSpecification[] | $jobs | |
int | $flags | Bitfield (supports JobQueue::QOS_ATOMIC) |
JobQueueError |
Definition at line 357 of file JobQueue.php.
References $job, assertNotReadOnly(), deduplicateRootJob(), and doBatchPush().
Referenced by push().
|
final |
Register the "root job" of a given job into the queue for de-duplication.
This should only be called right after all the new jobs have been inserted. This is used to turn older, duplicate, job entries into no-ops. The root job information will remain in the registry until it simply falls out of cache.
This requires that $job has two special fields in the "params" array:
A "root job" is a conceptual job that consist of potentially many smaller jobs that are actually inserted into the queue. For example, "refreshLinks" jobs are spawned when a template is edited. One can think of the task as "update links of pages that use template X" and an instance of that task as a "root job". However, what actually goes into the queue are range and leaf job subtypes. Since these jobs include things like page ID ranges and DB master positions, and can morph into smaller jobs recursively, simple duplicate detection for individual jobs being identical (like that of job_sha1) is not useful.
In the case of "refreshLinks", if these jobs are still in the queue when the template is edited again, we want all of these old refreshLinks jobs for that template to become no-ops. This can greatly reduce server load, since refreshLinks jobs involves parsing. Essentially, the new batch of jobs belong to a new "root job" and the older ones to a previous "root job" for the same task of "update links of pages that use template X".
This does nothing for certain queue classes.
IJobSpecification | $job |
JobQueueError |
Definition at line 478 of file JobQueue.php.
References assertNotReadOnly(), doDeduplicateRootJob(), and IJobSpecification\getType().
Referenced by batchPush().
|
final |
Definition at line 205 of file JobQueue.php.
References supportsDelayedJobs().
|
final |
Delete all unclaimed and delayed jobs from the queue.
JobQueueError |
Definition at line 574 of file JobQueue.php.
References assertNotReadOnly(), and doDelete().
|
abstractprotected |
RunnableJob | $job |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by ack().
|
abstractprotected |
IJobSpecification[] | $jobs | |
int | $flags |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by batchPush().
|
protected |
Stable to override.
IJobSpecification | $job |
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 494 of file JobQueue.php.
References $job, and getRootJobCacheKey().
Referenced by deduplicateRootJob().
|
protected |
Stable to override.
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Definition at line 585 of file JobQueue.php.
Referenced by delete().
|
protected |
Stable to override.
Reimplemented in JobQueueDB, and JobQueueFederated.
Definition at line 623 of file JobQueue.php.
Referenced by flushCaches().
|
protected |
Stable to override.
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 328 of file JobQueue.php.
Referenced by getAbandonedCount().
|
abstractprotected |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by getAcquiredCount().
|
protected |
Stable to override.
Reimplemented in JobQueueFederated, and JobQueueRedis.
Definition at line 304 of file JobQueue.php.
Referenced by getDelayedCount().
|
protected |
Stable to override.
array | $types | List of queues types |
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 730 of file JobQueue.php.
Referenced by getSiblingQueueSizes().
|
protected |
Stable to override.
array | $types | List of queues types |
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 706 of file JobQueue.php.
Referenced by getSiblingQueuesWithJobs().
|
abstractprotected |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by getSize().
|
abstractprotected |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by isEmpty().
|
protected |
Stable to override.
IJobSpecification | $job |
Reimplemented in JobQueueFederated, and JobQueueRedis.
Definition at line 535 of file JobQueue.php.
References $job, and getRootJobCacheKey().
Referenced by isRootJobOldDuplicate().
|
abstractprotected |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by pop().
|
protected |
Stable to override.
Reimplemented in JobQueueDB, and JobQueueFederated.
Definition at line 606 of file JobQueue.php.
Referenced by waitForBackups().
|
staticfinal |
Get a job queue object of the specified type.
$params includes:
Queue classes should throw an exception if they do not support the options given.
array | $params |
JobQueueError |
Definition at line 125 of file JobQueue.php.
References wfDeprecated().
Referenced by JobQueueFederated\__construct(), CopyJobQueue\execute(), and JobQueueGroup\factoryJobQueue().
|
protected |
string | $command | |
array | $params |
Definition at line 739 of file JobQueue.php.
References $command, and Job\factory().
Referenced by JobQueueRedis\getJobFromFields(), JobQueueRedis\getJobFromUidInternal(), JobQueueDB\jobFromRow(), and JobQueueMemory\jobFromSpecInternal().
|
final |
Clear any process and persistent caches.
Definition at line 614 of file JobQueue.php.
References doFlushCaches().
|
final |
Get the number of acquired jobs that can no longer be attempted.
Queue classes should use caching if they are any slower without memcached.
If caching is used, this number might be out of date for a minute.
JobQueueError |
Definition at line 317 of file JobQueue.php.
References $res, and doGetAbandonedCount().
|
final |
Get the number of acquired jobs (these are temporarily out of the queue).
Queue classes should use caching if they are any slower without memcached.
If caching is used, this number might be out of date for a minute.
JobQueueError |
Definition at line 271 of file JobQueue.php.
References $res, and doGetAcquiredCount().
Referenced by JobQueueMemory\doAck().
JobQueue::getAllAbandonedJobs | ( | ) |
Get an iterator to traverse over all abandoned jobs in this queue.
Stable to override
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 672 of file JobQueue.php.
JobQueue::getAllAcquiredJobs | ( | ) |
Get an iterator to traverse over all claimed jobs in this queue.
Callers should be quick to iterator over it or few results will be returned due to jobs being acknowledged and deleted
Stable to override
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Definition at line 660 of file JobQueue.php.
JobQueue::getAllDelayedJobs | ( | ) |
Get an iterator to traverse over all delayed jobs in this queue.
Note: results may be stale if the queue is concurrently modified.
Stable to override
JobQueueError |
Reimplemented in JobQueueFederated, and JobQueueRedis.
Definition at line 645 of file JobQueue.php.
|
abstract |
Get an iterator to traverse over all available jobs in this queue.
This does not include jobs that are currently acquired or delayed. Note: results may be stale if the queue is concurrently modified.
JobQueueError |
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
JobQueue::getCoalesceLocationInternal | ( | ) |
Do not use this function outside of JobQueue/JobQueueGroup.
Stable to override
Reimplemented in JobQueueDB, JobQueueFederated, and JobQueueRedis.
Definition at line 683 of file JobQueue.php.
|
final |
Get the number of delayed jobs (these are temporarily out of the queue).
Queue classes should use caching if they are any slower without memcached.
If caching is used, this number might be out of date for a minute.
JobQueueError |
Definition at line 293 of file JobQueue.php.
References $res, and doGetDelayedCount().
|
final |
|
final |
Definition at line 173 of file JobQueue.php.
References $order.
JobQueue::getReadOnlyReason | ( | ) |
Definition at line 213 of file JobQueue.php.
References $readOnlyReason.
|
protected |
string | $signature | Hash identifier of the root job |
Definition at line 557 of file JobQueue.php.
Referenced by doDeduplicateRootJob(), JobQueueRedis\doDeduplicateRootJob(), doIsRootJobOldDuplicate(), and JobQueueRedis\doIsRootJobOldDuplicate().
|
final |
Check the size of each of the given queues.
For queues not served by the same store as this one, 0 is returned. This is used for batching checks for queues stored at the same place.
array | $types | List of queues types |
JobQueueError |
Definition at line 720 of file JobQueue.php.
References doGetSiblingQueueSizes().
|
final |
Check whether each of the given queues are empty.
This is used for batching checks for queues stored at the same place.
array | $types | List of queues types |
JobQueueError |
Definition at line 696 of file JobQueue.php.
References doGetSiblingQueuesWithJobs().
|
final |
Get the number of available (unacquired, non-delayed) jobs in the queue.
Queue classes should use caching if they are any slower without memcached.
If caching is used, this number might be out of date for a minute.
JobQueueError |
Definition at line 250 of file JobQueue.php.
References $res, and doGetSize().
|
final |
Definition at line 166 of file JobQueue.php.
References $type.
Referenced by JobQueueDB\getAllAbandonedJobs(), JobQueueDB\getAllAcquiredJobs(), and JobQueueDB\getAllQueuedJobs().
|
final |
|
protected |
Call StatsdDataFactoryInterface::updateCount() for the queue overall and for the queue type.
string | $key | Event type |
string | $type | Job type |
int | $delta |
Definition at line 761 of file JobQueue.php.
Referenced by JobQueueDB\doAck(), JobQueueRedis\doAck(), JobQueueRedis\doBatchPush(), JobQueueDB\doBatchPushInternal(), JobQueueDB\doPop(), JobQueueRedis\doPop(), pop(), and JobQueueDB\recycleAndDeleteStaleJobs().
|
final |
Quickly check if the queue has no available (unacquired, non-delayed) jobs.
Queue classes should use caching if they are any slower without memcached.
If caching is used, this might return false when there are actually no jobs. If pop() is called and returns false then it should correct the cache. Also, calling flushCaches() first prevents this. However, this affect is typically not distinguishable from the race condition between isEmpty() and pop().
JobQueueError |
Definition at line 229 of file JobQueue.php.
References $res, and doIsEmpty().
|
finalprotected |
Check if the "root" job of a given job has been superseded by a newer one.
IJobSpecification | $job |
JobQueueError |
Definition at line 521 of file JobQueue.php.
References $job, and doIsRootJobOldDuplicate().
Referenced by pop().
|
abstractprotected |
Get the default queue order to use if configuration does not specify one.
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by __construct().
|
final |
Pop a job off of the queue.
This requires $wgJobClasses to be set for the given job type. Outside callers should use JobQueueGroup::pop() instead of this function.
JobQueueError |
Definition at line 398 of file JobQueue.php.
References $job, assertNotReadOnly(), doPop(), incrStats(), isRootJobOldDuplicate(), and DuplicateJob\newFromJob().
|
final |
Push one or more jobs into the queue.
This does not require $wgJobClasses to be set for the given job type. Outside callers should use JobQueueGroup::push() instead of this function.
IJobSpecification | IJobSpecification[] | $jobs | |
int | $flags | Bitfield (supports JobQueue::QOS_ATOMIC) |
JobQueueError |
Definition at line 342 of file JobQueue.php.
References batchPush().
Referenced by CopyJobQueue\copyJobs().
|
abstractprotected |
Get the allowed queue orders for configuration validation.
Reimplemented in JobQueueDB, JobQueueFederated, JobQueueMemory, and JobQueueRedis.
Referenced by __construct().
|
protected |
Find out if delayed jobs are supported for configuration validation.
Stable to override
Reimplemented in JobQueueFederated, and JobQueueRedis.
Definition at line 197 of file JobQueue.php.
Referenced by delayedJobsEnabled().
|
final |
Wait for any replica DBs or backup servers to catch up.
This does nothing for certain queue classes.
JobQueueError |
Definition at line 597 of file JobQueue.php.
References doWaitForBackups().
Referenced by CopyJobQueue\copyJobs(), and ManageJobs\repushAbandoned().
|
protected |
Time to live in seconds.
Definition at line 43 of file JobQueue.php.
|
protected |
DB domain ID.
Definition at line 37 of file JobQueue.php.
Referenced by JobQueueFederated\__construct(), getDomain(), and JobQueueMemory\getQueueData().
|
protected |
Definition at line 51 of file JobQueue.php.
|
protected |
Maximum number of times to try a job.
Definition at line 45 of file JobQueue.php.
|
protected |
|
protected |
Read only rationale (or false if r/w)
Definition at line 47 of file JobQueue.php.
Referenced by getReadOnlyReason().
|
protected |
Definition at line 49 of file JobQueue.php.
|
protected |
Job type.
Definition at line 39 of file JobQueue.php.
Referenced by JobQueueDB\doFlushCaches(), JobQueueFederated\doGetSiblingQueueSizes(), JobQueueRedis\doGetSiblingQueueSizes(), JobQueueMemory\getQueueData(), JobQueueRedis\getQueueKey(), getType(), and JobQueueDB\recycleAndDeleteStaleJobs().
|
protected |
Definition at line 54 of file JobQueue.php.
|
protected |
Definition at line 56 of file JobQueue.php.
|
protected |
Definition at line 58 of file JobQueue.php.