MediaWiki master
MediaWiki\Maintenance\OrderedStreamingForkController Class Reference

Apply a transformation to values via a pool of sub processes. More...

Inherits MediaWiki\Maintenance\ForkController.

Collaboration diagram for MediaWiki\Maintenance\OrderedStreamingForkController:

Public Member Functions

 __construct ( $numProcs, $workCallback, $input, $output)
 
 start ()
 Start the child processes.This should only be called from the command line. It should be called as early as possible during execution.This will return 'child' in the child processes. In the parent process, it will run until all the child processes exit or a TERM signal is received. It will then return 'done'.
Returns
string

 
- Public Member Functions inherited from MediaWiki\Maintenance\ForkController
 __construct ( $numProcs, $flags=0)
 
 allSuccessful ()
 Return true if all completed child processes exited with an exit status / return code of 0.
 
 getChildNumber ()
 Get the number of the child currently running.
 

Protected Member Functions

 consume ()
 Child worker process.
 
 consumeNoFork ()
 Special cased version of self::consume() when no forking occurs.
 
 feedChildren (array $sockets)
 Reads lines of work from $this->input and farms them out to the provided socket.
 
 forkWorkers ( $numProcs)
 
 receive ( $id, $data)
 
 updateAvailableSockets (&$sockets, &$used, $timeout)
 Moves sockets from $used to $sockets when they are available for more work.
 
- Protected Member Functions inherited from MediaWiki\Maintenance\ForkController
 handleTermSignal ( $signal)
 
 initChild ()
 
 prepareEnvironment ()
 

Protected Attributes

string[] $delayedOutputData = []
 Int key indicates order, value is data.
 
resource $input
 
int $nextOutputId
 
resource $output
 
callable $workCallback
 
- Protected Attributes inherited from MediaWiki\Maintenance\ForkController
int $childNumber = 0
 
array null $children = []
 
int[] $exitStatuses = []
 
int $flags = 0
 
int $procsToStart = 0
 
bool $termReceived = false
 

Additional Inherited Members

- Static Protected Attributes inherited from MediaWiki\Maintenance\ForkController
static int[] $RESTARTABLE_SIGNALS = []
 

Detailed Description

Apply a transformation to values via a pool of sub processes.

The controller reads lines from a given input stream, where each line describes work to be done. This work is then farmed out to multiple child streams that correspond to child procesess. Each child has exactly one piece of work in-flight at a given moment. The result of each work is written to an output stream.

If numProcs is zero, the fallback is to perform work in-process instead.

This class guarantees that the output is produced in the same exact order as input values were.

Currently used by CirrusSearch extension to implement CLI search script.

Since
1.30

Definition at line 42 of file OrderedStreamingForkController.php.

Constructor & Destructor Documentation

◆ __construct()

MediaWiki\Maintenance\OrderedStreamingForkController::__construct ( $numProcs,
$workCallback,
$input,
$output )
Parameters
int$numProcsThe number of worker processes to fork
callable$workCallbackA callback to call in the child process once for each line of work to process.
resource$inputA socket to read work lines from
resource$outputA socket to write the result of work to.

Definition at line 61 of file OrderedStreamingForkController.php.

References MediaWiki\Maintenance\OrderedStreamingForkController\$input, MediaWiki\Maintenance\OrderedStreamingForkController\$output, and MediaWiki\Maintenance\OrderedStreamingForkController\$workCallback.

Member Function Documentation

◆ consume()

MediaWiki\Maintenance\OrderedStreamingForkController::consume ( )
protected

Child worker process.

Reads work from $this->input and writes the result of that work to $this->output when completed.

Definition at line 127 of file OrderedStreamingForkController.php.

Referenced by MediaWiki\Maintenance\OrderedStreamingForkController\start().

◆ consumeNoFork()

MediaWiki\Maintenance\OrderedStreamingForkController::consumeNoFork ( )
protected

Special cased version of self::consume() when no forking occurs.

Definition at line 141 of file OrderedStreamingForkController.php.

Referenced by MediaWiki\Maintenance\OrderedStreamingForkController\start().

◆ feedChildren()

MediaWiki\Maintenance\OrderedStreamingForkController::feedChildren ( array $sockets)
protected

Reads lines of work from $this->input and farms them out to the provided socket.

Parameters
resource[]$sockets

Definition at line 163 of file OrderedStreamingForkController.php.

References MediaWiki\Maintenance\OrderedStreamingForkController\updateAvailableSockets().

Referenced by MediaWiki\Maintenance\OrderedStreamingForkController\forkWorkers().

◆ forkWorkers()

MediaWiki\Maintenance\OrderedStreamingForkController::forkWorkers ( $numProcs)
protected

◆ receive()

MediaWiki\Maintenance\OrderedStreamingForkController::receive ( $id,
$data )
protected
Parameters
int$id
string$data

Definition at line 219 of file OrderedStreamingForkController.php.

Referenced by MediaWiki\Maintenance\OrderedStreamingForkController\updateAvailableSockets().

◆ start()

MediaWiki\Maintenance\OrderedStreamingForkController::start ( )

Start the child processes.This should only be called from the command line. It should be called as early as possible during execution.This will return 'child' in the child processes. In the parent process, it will run until all the child processes exit or a TERM signal is received. It will then return 'done'.

Returns
string

Reimplemented from MediaWiki\Maintenance\ForkController.

Definition at line 71 of file OrderedStreamingForkController.php.

References MediaWiki\Maintenance\OrderedStreamingForkController\consume(), and MediaWiki\Maintenance\OrderedStreamingForkController\consumeNoFork().

◆ updateAvailableSockets()

MediaWiki\Maintenance\OrderedStreamingForkController::updateAvailableSockets ( & $sockets,
& $used,
$timeout )
protected

Moves sockets from $used to $sockets when they are available for more work.

Parameters
resource[]&$socketsList of sockets that are waiting for work
resource[]&$usedList of sockets currently performing work
int$timeoutThe number of seconds to block waiting. 0 for non-blocking operation.

Definition at line 201 of file OrderedStreamingForkController.php.

References MediaWiki\Maintenance\OrderedStreamingForkController\receive().

Referenced by MediaWiki\Maintenance\OrderedStreamingForkController\feedChildren().

Member Data Documentation

◆ $delayedOutputData

string [] MediaWiki\Maintenance\OrderedStreamingForkController::$delayedOutputData = []
protected

Int key indicates order, value is data.

Definition at line 52 of file OrderedStreamingForkController.php.

◆ $input

resource MediaWiki\Maintenance\OrderedStreamingForkController::$input
protected

◆ $nextOutputId

int MediaWiki\Maintenance\OrderedStreamingForkController::$nextOutputId
protected

Definition at line 50 of file OrderedStreamingForkController.php.

◆ $output

resource MediaWiki\Maintenance\OrderedStreamingForkController::$output
protected

◆ $workCallback

callable MediaWiki\Maintenance\OrderedStreamingForkController::$workCallback
protected

The documentation for this class was generated from the following file: