MediaWiki REL1_34
JobRunner Class Reference

Job queue runner utility methods. More...

Inheritance diagram for JobRunner:
Collaboration diagram for JobRunner:

Public Member Functions

 __construct (LoggerInterface $logger=null)
 
 run (array $options)
 Run jobs of the specified number/type for the specified time.
 
 setDebugHandler ( $debug)
 
 setLogger (LoggerInterface $logger)
 

Public Attributes

const ERROR_BACKOFF_TTL = 1
 
const LAG_CHECK_PERIOD = 1.0
 
const MAX_ALLOWED_LAG = 3
 
const READONLY_BACKOFF_TTL = 30
 

Protected Attributes

Config $config
 
callable null $debug
 Debug output handler.
 
 $logger
 

Private Member Functions

 checkMemoryOK ()
 Make sure that this script is not too close to the memory usage limit.
 
 commitMasterChanges (LBFactory $lbFactory, RunnableJob $job, $fnameTrxOwner)
 Issue a commit on all masters who are currently in a transaction and have made changes to the database.
 
 debugCallback ( $msg)
 Log the job message.
 
 executeJob (RunnableJob $job, LBFactory $lbFactory, $stats, $popTime)
 
 getBackoffTimeToWait (RunnableJob $job)
 
 getErrorBackoffTTL ( $error)
 
 getMaxRssKb ()
 
 loadBackoffs (array $backoffs, $mode='wait')
 Get the previous backoff expiries from persistent storage On I/O or lock acquisition failure this returns the original $backoffs.
 
 syncBackoffDeltas (array $backoffs, array &$deltas, $mode='wait')
 Merge the current backoff expiries from persistent storage.
 

Detailed Description

Job queue runner utility methods.

Since
1.24

Definition at line 39 of file JobRunner.php.

Constructor & Destructor Documentation

◆ __construct()

JobRunner::__construct ( LoggerInterface  $logger = null)
Parameters
LoggerInterface | null$logger

Definition at line 73 of file JobRunner.php.

References $logger, and setLogger().

Member Function Documentation

◆ checkMemoryOK()

JobRunner::checkMemoryOK ( )
private

Make sure that this script is not too close to the memory usage limit.

It is better to die in between jobs than OOM right in the middle of one.

Returns
bool

Definition at line 487 of file JobRunner.php.

References debugCallback().

Referenced by run().

◆ commitMasterChanges()

JobRunner::commitMasterChanges ( LBFactory  $lbFactory,
RunnableJob  $job,
  $fnameTrxOwner 
)
private

Issue a commit on all masters who are currently in a transaction and have made changes to the database.

It also supports sometimes waiting for the local wiki's replica DBs to catch up. See the documentation for $wgJobSerialCommitThreshold for more.

Parameters
LBFactory$lbFactory
RunnableJob$job
string$fnameTrxOwner
Exceptions
DBError

Definition at line 537 of file JobRunner.php.

References $job, Wikimedia\Rdbms\LBFactory\commitMasterChanges(), debugCallback(), and Wikimedia\Rdbms\ILBFactory\getMainLB().

Referenced by executeJob().

◆ debugCallback()

JobRunner::debugCallback (   $msg)
private

Log the job message.

Parameters
string$msgThe message to log

Definition at line 520 of file JobRunner.php.

References wfTimestamp().

Referenced by checkMemoryOK(), commitMasterChanges(), and executeJob().

◆ executeJob()

JobRunner::executeJob ( RunnableJob  $job,
LBFactory  $lbFactory,
  $stats,
  $popTime 
)
private
Parameters
RunnableJob$job
LBFactory$lbFactory
StatsdDataFactoryInterface$stats
float$popTime
Returns
array Map of status/error/timeMs

Definition at line 274 of file JobRunner.php.

References $job, Wikimedia\Rdbms\LBFactory\beginMasterChanges(), Wikimedia\Rdbms\LBFactory\commitMasterChanges(), commitMasterChanges(), debugCallback(), Wikimedia\Rdbms\LBFactory\flushReplicaSnapshots(), getMaxRssKb(), and wfTimestamp().

Referenced by run().

◆ getBackoffTimeToWait()

JobRunner::getBackoffTimeToWait ( RunnableJob  $job)
private
Parameters
RunnableJob$job
Returns
int Seconds for this runner to avoid doing more jobs of this type
See also
$wgJobBackoffThrottling

Definition at line 378 of file JobRunner.php.

References $job.

Referenced by run().

◆ getErrorBackoffTTL()

JobRunner::getErrorBackoffTTL (   $error)
private
Parameters
string$error
Returns
int TTL in seconds

Definition at line 261 of file JobRunner.php.

References ERROR_BACKOFF_TTL, and READONLY_BACKOFF_TTL.

Referenced by run().

◆ getMaxRssKb()

JobRunner::getMaxRssKb ( )
private
Returns
int|null Max memory RSS in kilobytes

Definition at line 367 of file JobRunner.php.

References wfGetRusage().

Referenced by executeJob().

◆ loadBackoffs()

JobRunner::loadBackoffs ( array  $backoffs,
  $mode = 'wait' 
)
private

Get the previous backoff expiries from persistent storage On I/O or lock acquisition failure this returns the original $backoffs.

Parameters
array$backoffsMap of (job type => UNIX timestamp)
string$modeLock wait mode - "wait" or "nowait"
Returns
array Map of (job type => backoff expiry timestamp)

Definition at line 410 of file JobRunner.php.

References $content, $file, $type, and wfTempDir().

Referenced by syncBackoffDeltas().

◆ run()

JobRunner::run ( array  $options)

Run jobs of the specified number/type for the specified time.

The response map has a 'job' field that lists status of each job, including:

  • type : the job type
  • status : ok/failed
  • error : any error message string
  • time : the job run time in ms The response map also has:
  • backoffs : the (job type => seconds) map of backoff times
  • elapsed : the total time spent running tasks in ms
  • reached : the reason the script finished, one of (none-ready, job-limit, time-limit, memory-limit)

This method outputs status information only if a debug handler was set. Any exceptions are caught and logged, but are not reported as output.

Parameters
array$optionsMap of parameters:
  • type : the job type (or false for the default types)
  • maxJobs : maximum number of jobs to run
  • maxTime : maximum time in seconds before stopping
  • throttle : whether to respect job backoff configuration
Returns
array Summary response that can easily be JSON serialized

Definition at line 105 of file JobRunner.php.

References $job, $response, $success, $type, checkMemoryOK(), executeJob(), getBackoffTimeToWait(), getErrorBackoffTTL(), syncBackoffDeltas(), and wfReadOnly().

◆ setDebugHandler()

JobRunner::setDebugHandler (   $debug)
Parameters
callable$debugOptional debug output handler

Definition at line 58 of file JobRunner.php.

References $debug.

◆ setLogger()

JobRunner::setLogger ( LoggerInterface  $logger)
Parameters
LoggerInterface$logger
Returns
void

Definition at line 66 of file JobRunner.php.

References $logger.

Referenced by __construct().

◆ syncBackoffDeltas()

JobRunner::syncBackoffDeltas ( array  $backoffs,
array &  $deltas,
  $mode = 'wait' 
)
private

Merge the current backoff expiries from persistent storage.

The $deltas map is set to an empty array on success. On I/O or lock acquisition failure this returns the original $backoffs.

Parameters
array$backoffsMap of (job type => UNIX timestamp)
array$deltasMap of (job type => seconds)
string$modeLock wait mode - "wait" or "nowait"
Returns
array The new backoffs account for $backoffs and the latest file data

Definition at line 447 of file JobRunner.php.

References $content, $file, $type, loadBackoffs(), and wfTempDir().

Referenced by run().

Member Data Documentation

◆ $config

Config JobRunner::$config
protected

Definition at line 41 of file JobRunner.php.

◆ $debug

callable null JobRunner::$debug
protected

Debug output handler.

Definition at line 43 of file JobRunner.php.

Referenced by setDebugHandler().

◆ $logger

LoggerInterface JobRunner::$logger
protected

Definition at line 48 of file JobRunner.php.

Referenced by __construct(), and setLogger().

◆ ERROR_BACKOFF_TTL

const JobRunner::ERROR_BACKOFF_TTL = 1

Definition at line 52 of file JobRunner.php.

Referenced by getErrorBackoffTTL().

◆ LAG_CHECK_PERIOD

const JobRunner::LAG_CHECK_PERIOD = 1.0

Definition at line 51 of file JobRunner.php.

◆ MAX_ALLOWED_LAG

const JobRunner::MAX_ALLOWED_LAG = 3

Definition at line 50 of file JobRunner.php.

◆ READONLY_BACKOFF_TTL

const JobRunner::READONLY_BACKOFF_TTL = 30

Definition at line 53 of file JobRunner.php.

Referenced by getErrorBackoffTTL().


The documentation for this class was generated from the following file: